2024-09-15T22:18:20.601 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T22:18:20.628 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T22:18:20.721 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906253' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNNJVUUx0ah3G4PRAKz5ba9TXNGoXJtjR2NIaTS71TKAVRfh3fRsrTavMl04iUUqLGQtqWuNlnLuLgk9XcRLgSI= smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF+jRECDcPaWisi083jZREvbj3RI5g4RbeWfnqo8nUwXheUh1yNmAJtZGRFaDI1EMNXya6EiN+A9YB7rqQq0RXY= smithi130.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJysa0vLBiyqXnNxD/PHR+sRjnjV670TLX7DR52YL8f6ixaFtA0kyiD+TJzARe0JiU0k13+MyeQBLpmwpFV080s= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T22:18:20.721 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T22:18:20.722 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T22:18:20.722 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T22:18:20.723 INFO:teuthology.task.internal:Checking packages... 2024-09-15T22:18:20.744 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T22:18:20.744 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T22:18:20.744 INFO:teuthology.packaging:ref: None 2024-09-15T22:18:20.744 INFO:teuthology.packaging:tag: None 2024-09-15T22:18:20.744 INFO:teuthology.packaging:branch: squid 2024-09-15T22:18:20.744 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:18:20.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T22:18:20.960 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-15T22:18:20.961 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T22:18:20.968 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T22:18:20.968 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T22:18:20.990 INFO:teuthology.task.internal:Saving configuration 2024-09-15T22:18:21.011 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T22:18:21.021 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T22:18:21.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 22:13:31.410708', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNNJVUUx0ah3G4PRAKz5ba9TXNGoXJtjR2NIaTS71TKAVRfh3fRsrTavMl04iUUqLGQtqWuNlnLuLgk9XcRLgSI='} 2024-09-15T22:18:21.067 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 22:13:31.408787', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF+jRECDcPaWisi083jZREvbj3RI5g4RbeWfnqo8nUwXheUh1yNmAJtZGRFaDI1EMNXya6EiN+A9YB7rqQq0RXY='} 2024-09-15T22:18:21.088 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi130.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 22:13:31.412495', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJysa0vLBiyqXnNxD/PHR+sRjnjV670TLX7DR52YL8f6ixaFtA0kyiD+TJzARe0JiU0k13+MyeQBLpmwpFV080s='} 2024-09-15T22:18:21.088 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T22:18:21.094 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-15T22:18:21.094 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-15T22:18:21.094 INFO:teuthology.task.internal:roles: ubuntu@smithi130.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-15T22:18:21.094 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T22:18:21.213 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb1980dbbe0>, signals=[15]) 2024-09-15T22:18:21.214 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T22:18:21.219 INFO:teuthology.task.internal:Opening connections... 2024-09-15T22:18:21.220 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:18:21.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:18:21.300 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-09-15T22:18:21.301 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:18:21.376 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi130.front.sepia.ceph.com 2024-09-15T22:18:21.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:18:21.450 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T22:18:21.457 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-15T22:18:21.479 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-15T22:18:21.479 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-15T22:18:21.537 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-15T22:18:21.537 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-15T22:18:21.537 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-15T22:18:21.537 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-15T22:18:21.537 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-15T22:18:21.537 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T22:18:21.538 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T22:18:21.539 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-15T22:18:21.562 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-09-15T22:18:21.580 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-09-15T22:18:21.580 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-09-15T22:18:21.638 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-09-15T22:18:21.639 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T22:18:21.640 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T22:18:21.640 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T22:18:21.640 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-09-15T22:18:21.662 DEBUG:teuthology.orchestra.run.smithi130:> uname -m 2024-09-15T22:18:21.680 INFO:teuthology.orchestra.run.smithi130.stdout:x86_64 2024-09-15T22:18:21.680 DEBUG:teuthology.orchestra.run.smithi130:> cat /etc/os-release 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:NAME="CentOS Stream" 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION="9" 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:ID="centos" 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:ID_LIKE="rhel fedora" 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:VERSION_ID="9" 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:PLATFORM_ID="platform:el9" 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T22:18:21.738 INFO:teuthology.orchestra.run.smithi130.stdout:ANSI_COLOR="0;31" 2024-09-15T22:18:21.739 INFO:teuthology.orchestra.run.smithi130.stdout:LOGO="fedora-logo-icon" 2024-09-15T22:18:21.739 INFO:teuthology.orchestra.run.smithi130.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T22:18:21.739 INFO:teuthology.orchestra.run.smithi130.stdout:HOME_URL="https://centos.org/" 2024-09-15T22:18:21.739 INFO:teuthology.orchestra.run.smithi130.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T22:18:21.739 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T22:18:21.739 INFO:teuthology.orchestra.run.smithi130.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T22:18:21.739 INFO:teuthology.lock.ops:Updating smithi130.front.sepia.ceph.com on lock server 2024-09-15T22:18:21.761 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T22:18:21.772 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T22:18:21.777 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T22:18:21.777 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-15T22:18:21.780 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-09-15T22:18:21.782 DEBUG:teuthology.orchestra.run.smithi130:> test '!' -e /home/ubuntu/cephtest 2024-09-15T22:18:21.798 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T22:18:21.804 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T22:18:21.804 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-15T22:18:21.838 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-09-15T22:18:21.840 DEBUG:teuthology.orchestra.run.smithi130:> test -z $(ls -A /var/lib/ceph) 2024-09-15T22:18:21.865 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T22:18:21.942 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T22:18:21.956 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-15T22:18:21.957 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-15T22:18:21.957 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-15T22:18:21.957 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T22:18:21.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:18:21.974 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-15T22:18:22.032 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-15T22:18:22.033 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-15T22:18:22.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-15T22:18:27.545 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.2 MB 00:03 2024-09-15T22:18:33.554 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-15T22:18:38.188 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-15T22:18:39.872 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-15T22:18:41.026 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-15T22:18:46.699 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T22:18:48.805 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T22:18:48.806 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T22:18:48.902 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T22:18:48.912 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:18:48.913 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-15T22:18:48.913 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-15T22:18:48.913 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-15T22:18:48.913 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:18:48.913 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-15T22:18:48.913 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-15T22:18:48.914 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-15T22:18:49.175 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 185 kB/s | 23 kB 00:00 2024-09-15T22:18:49.842 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-15T22:18:50.501 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 22 MB/s | 29 MB 00:01 2024-09-15T22:18:50.810 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-15T22:18:50.810 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:18:50.810 INFO:teuthology.orchestra.run.smithi016.stdout:Total 42 MB/s | 79 MB 00:01 2024-09-15T22:18:51.524 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-15T22:18:51.921 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-15T22:18:51.922 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-15T22:18:52.875 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-15T22:18:52.876 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-15T22:18:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-15T22:18:56.145 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T22:18:56.226 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T22:18:59.032 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T22:18:59.164 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T22:19:04.287 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T22:19:04.331 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T22:19:09.475 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T22:19:48.637 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T22:19:48.679 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T22:19:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T22:19:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T22:19:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T22:19:49.001 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:19:49.864 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-15T22:19:50.214 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T22:19:50.314 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T22:19:50.315 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T22:19:50.315 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T22:19:50.315 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T22:19:50.315 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T22:19:50.315 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-15T22:19:50.315 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T22:19:50.315 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T22:19:50.373 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T22:19:50.374 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T22:19:50.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-15T22:19:50.407 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T22:19:50.408 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:19:50.409 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T22:19:50.410 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T22:19:50.410 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-505.el9.x86_64 2024-09-15T22:19:50.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T22:19:51.074 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-15T22:19:52.383 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T22:19:52.402 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-15T22:19:52.404 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-15T22:19:52.419 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.Sw45k5kJaC 2024-09-15T22:19:52.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Sw45k5kJaC 2024-09-15T22:19:52.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.Sw45k5kJaC 2024-09-15T22:19:52.617 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.Sw45k5kJaC is 6KB 2024-09-15T22:19:52.637 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.Sw45k5kJaC 2024-09-15T22:19:52.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T22:19:52.714 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T22:19:52.714 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T22:19:52.714 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T22:19:52.714 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T22:19:52.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T22:19:52.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-15T22:20:22.870 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-15T22:20:22.871 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T22:20:22.872 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:21:22.874 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T22:21:31.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T22:21:31.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:21:32.281 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-15T22:21:32.660 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T22:21:32.660 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-15T22:21:32.660 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-15T22:21:32.675 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-505.el9.x86_64 2024-09-15T22:21:32.675 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T22:21:32.676 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T22:21:32.676 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T22:21:33.677 DEBUG:teuthology.parallel:result is None 2024-09-15T22:21:33.678 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T22:21:33.685 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T22:21:33.685 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T22:21:33.688 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T22:21:33.690 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T22:21:33.709 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T22:21:33.731 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T22:21:33.770 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T22:21:33.770 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T22:21:33.772 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T22:21:33.775 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T22:21:33.814 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T22:21:33.820 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T22:21:33.820 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T22:21:33.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:21:33.864 DEBUG:teuthology.orchestra.run.smithi130:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T22:21:33.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:21:33.882 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T22:21:33.885 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T22:21:33.907 DEBUG:teuthology.orchestra.run.smithi130:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T22:21:33.945 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:21:33.962 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:21:33.976 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:21:33.993 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:21:33.999 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:21:34.014 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:21:34.015 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T22:21:34.023 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T22:21:34.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T22:21:34.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T22:21:34.028 DEBUG:teuthology.orchestra.run.smithi130:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T22:21:34.056 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T22:21:34.064 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T22:21:34.065 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T22:21:34.094 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T22:21:34.097 DEBUG:teuthology.orchestra.run.smithi130:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T22:21:34.113 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:21:34.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:21:34.247 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:21:34.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:21:34.370 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:21:34.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T22:21:34.451 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:21:34.496 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:21:34.548 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:21:34.630 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:21:34.677 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:21:34.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T22:21:34.746 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:21:34.792 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:21:34.848 DEBUG:teuthology.orchestra.run.smithi130:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:21:34.934 DEBUG:teuthology.orchestra.run.smithi130:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:21:34.975 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:21:34.975 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T22:21:35.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-15T22:21:35.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-09-15T22:21:35.047 DEBUG:teuthology.orchestra.run.smithi130:> sudo service rsyslog restart 2024-09-15T22:21:35.085 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:21:35.097 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:21:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:21:35.567 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T22:21:35.575 INFO:teuthology.task.internal:Starting timer... 2024-09-15T22:21:35.575 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T22:21:35.654 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T22:21:35.696 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T22:21:35.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-15T22:21:35.749 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-15T22:21:35.752 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-09-15T22:21:35.828 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-09-15T22:21:35.830 DEBUG:teuthology.orchestra.run.smithi130:> sudo service auditd rotate 2024-09-15T22:21:35.890 INFO:teuthology.orchestra.run.smithi130.stdout:Rotating logs: 2024-09-15T22:21:35.892 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T22:21:35.892 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-15T22:21:35.925 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-15T22:21:35.925 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-09-15T22:21:35.952 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-09-15T22:21:35.953 DEBUG:teuthology.orchestra.run.smithi130:> /usr/sbin/getenforce 2024-09-15T22:21:35.973 INFO:teuthology.orchestra.run.smithi130.stdout:Permissive 2024-09-15T22:21:35.973 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi130.front.sepia.ceph.com': 'permissive'} 2024-09-15T22:21:35.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:21:35.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:21:36.000 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:21:36.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:21:36.028 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:21:36.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:21:36.054 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T22:21:36.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-15T22:21:36.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-09-15T22:21:36.219 DEBUG:teuthology.orchestra.run.smithi130:> sudo /usr/sbin/setenforce permissive 2024-09-15T22:21:36.248 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T22:21:36.257 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T22:21:36.269 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T22:21:36.270 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi059.front.sepia.ceph.com,smithi130.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T22:25:38.532 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi130.front.sepia.ceph.com')] 2024-09-15T22:25:38.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T22:25:38.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:25:38.612 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-15T22:25:38.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-15T22:25:38.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-15T22:25:38.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:25:38.766 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-09-15T22:25:38.853 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-09-15T22:25:38.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-15T22:25:38.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi130.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:25:38.931 DEBUG:teuthology.orchestra.run.smithi130:> true 2024-09-15T22:25:39.016 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi130.front.sepia.ceph.com' 2024-09-15T22:25:39.016 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T22:25:39.031 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T22:25:39.031 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T22:25:39.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:25:39.034 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T22:25:39.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:25:39.037 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T22:25:39.037 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:25:39.066 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T22:25:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T22:25:39.082 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T22:25:39.084 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T22:25:39.085 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T22:25:39.098 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T22:25:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-09-15T22:25:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-15T22:25:39.119 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-09-15T22:25:39.125 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-15T22:25:39.131 INFO:teuthology.orchestra.run.smithi130.stderr:sudo: ntpd: command not found 2024-09-15T22:25:39.134 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T22:25:39.140 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T22:25:39.143 INFO:teuthology.orchestra.run.smithi130.stdout:506 Cannot talk to daemon 2024-09-15T22:25:39.148 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T22:25:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T22:25:39.157 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T22:25:39.171 INFO:teuthology.orchestra.run.smithi130.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T22:25:39.208 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:25:39.212 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:25:39.212 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-09-15T22:25:39.212 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.212 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.212 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.212 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.213 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:25:39.216 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:25:39.216 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-15T22:25:39.216 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.216 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.217 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.217 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.227 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:25:39.231 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:25:39.231 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-09-15T22:25:39.231 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.231 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.231 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.231 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:25:39.232 INFO:teuthology.run_tasks:Running task install... 2024-09-15T22:25:39.240 DEBUG:teuthology.task.install:project ceph 2024-09-15T22:25:39.240 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T22:25:39.240 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T22:25:39.240 INFO:teuthology.task.install:Using flavor: default 2024-09-15T22:25:39.246 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T22:25:39.246 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-15T22:25:39.246 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T22:25:39.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:25:39.249 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T22:25:39.249 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:25:39.250 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T22:25:39.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:25:39.402 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T22:25:39.402 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T22:25:39.406 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T22:25:39.406 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T22:25:39.409 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T22:25:39.410 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T22:25:39.511 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T22:25:39.511 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:25:39.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T22:25:39.520 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T22:25:39.521 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:25:39.521 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T22:25:39.525 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T22:25:39.526 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:25:39.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T22:25:39.540 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T22:25:39.540 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T22:25:39.553 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T22:25:39.553 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T22:25:39.556 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T22:25:39.556 DEBUG:teuthology.orchestra.run.smithi130:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T22:25:39.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T22:25:39.619 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T22:25:39.622 DEBUG:teuthology.orchestra.run.smithi130:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T22:25:39.690 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T22:25:39.694 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T22:25:39.697 DEBUG:teuthology.orchestra.run.smithi130:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T22:25:39.756 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-15T22:25:39.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-15T22:25:39.759 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-09-15T22:25:39.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-09-15T22:25:39.761 INFO:teuthology.orchestra.run.smithi130.stdout:check_obsoletes = 1 2024-09-15T22:25:39.763 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum clean all 2024-09-15T22:25:39.988 INFO:teuthology.orchestra.run.smithi059.stdout:45 files removed 2024-09-15T22:25:39.992 INFO:teuthology.orchestra.run.smithi130.stdout:45 files removed 2024-09-15T22:25:40.003 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-15T22:25:40.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T22:25:40.021 DEBUG:teuthology.orchestra.run.smithi130:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T22:25:40.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T22:25:40.573 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 304 kB/s | 83 kB 00:00 2024-09-15T22:25:40.574 INFO:teuthology.orchestra.run.smithi130.stdout:ceph packages for x86_64 315 kB/s | 83 kB 00:00 2024-09-15T22:25:40.601 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 309 kB/s | 83 kB 00:00 2024-09-15T22:25:40.850 INFO:teuthology.orchestra.run.smithi130.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-15T22:25:40.857 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T22:25:40.885 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T22:25:41.117 INFO:teuthology.orchestra.run.smithi130.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-15T22:25:41.132 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T22:25:41.161 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T22:25:41.642 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-15T22:25:41.678 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-15T22:25:43.660 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-15T22:25:43.679 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-15T22:25:44.049 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-15T22:25:47.459 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 20 MB 00:02 2024-09-15T22:25:48.028 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T22:25:48.157 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T22:25:49.600 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 109 kB/s | 19 kB 00:00 2024-09-15T22:25:50.311 INFO:teuthology.orchestra.run.smithi130.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-15T22:25:50.671 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T22:25:51.724 INFO:teuthology.orchestra.run.smithi130.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-15T22:25:52.017 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-15T22:25:53.743 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-15T22:25:54.905 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-15T22:25:56.343 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T22:25:57.129 INFO:teuthology.orchestra.run.smithi130.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T22:25:58.822 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:25:58.822 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:25:58.835 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:25:58.836 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:25:58.836 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:25:58.837 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:25:58.839 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:25:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:25:58.842 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:25:58.911 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:25:58.926 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:25:58.927 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:25:58.928 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:25:58.929 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:25:58.930 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:25:58.931 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:25:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:25:58.933 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:25:58.933 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:25:58.933 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:25:58.933 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:25:58.933 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:25:58.933 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:25:58.936 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:25:58.936 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:25:58.936 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:25:58.936 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:25:58.937 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:25:58.938 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:25:58.939 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:25:58.940 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:25:58.941 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:Install 148 Packages 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-15T22:25:58.942 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:25:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 226 M 2024-09-15T22:25:58.943 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-15T22:25:59.440 INFO:teuthology.orchestra.run.smithi130.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:25:59.441 INFO:teuthology.orchestra.run.smithi130.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:25:59.452 INFO:teuthology.orchestra.run.smithi130.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:25:59.453 INFO:teuthology.orchestra.run.smithi130.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:25:59.454 INFO:teuthology.orchestra.run.smithi130.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:25:59.455 INFO:teuthology.orchestra.run.smithi130.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:25:59.456 INFO:teuthology.orchestra.run.smithi130.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:25:59.458 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:25:59.459 INFO:teuthology.orchestra.run.smithi130.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:25:59.521 INFO:teuthology.orchestra.run.smithi130.stdout:Dependencies resolved. 2024-09-15T22:25:59.531 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-09-15T22:25:59.531 INFO:teuthology.orchestra.run.smithi130.stdout: Package Arch Version Repository Size 2024-09-15T22:25:59.531 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-09-15T22:25:59.531 INFO:teuthology.orchestra.run.smithi130.stdout:Installing: 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:25:59.532 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:25:59.533 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:25:59.534 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrading: 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:25:59.535 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout:Installing dependencies: 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:25:59.536 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:25:59.537 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:25:59.538 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:25:59.539 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:25:59.540 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:25:59.541 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:25:59.541 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:25:59.541 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:25:59.541 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:25:59.541 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:25:59.541 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:25:59.542 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:25:59.543 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:25:59.544 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:25:59.545 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:25:59.546 INFO:teuthology.orchestra.run.smithi130.stdout:Installing weak dependencies: 2024-09-15T22:25:59.547 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:25:59.547 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:25:59.547 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction Summary 2024-09-15T22:25:59.547 INFO:teuthology.orchestra.run.smithi130.stdout:====================================================================================== 2024-09-15T22:25:59.547 INFO:teuthology.orchestra.run.smithi130.stdout:Install 148 Packages 2024-09-15T22:25:59.547 INFO:teuthology.orchestra.run.smithi130.stdout:Upgrade 8 Packages 2024-09-15T22:25:59.547 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:25:59.548 INFO:teuthology.orchestra.run.smithi016.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-15T22:25:59.548 INFO:teuthology.orchestra.run.smithi130.stdout:Total download size: 226 M 2024-09-15T22:25:59.548 INFO:teuthology.orchestra.run.smithi130.stdout:Downloading Packages: 2024-09-15T22:25:59.770 INFO:teuthology.orchestra.run.smithi016.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.8 MB/s | 896 kB 00:00 2024-09-15T22:25:59.853 INFO:teuthology.orchestra.run.smithi016.stdout:(3/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-15T22:25:59.879 INFO:teuthology.orchestra.run.smithi016.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 1.3 MB/s | 145 kB 00:00 2024-09-15T22:25:59.979 INFO:teuthology.orchestra.run.smithi016.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 19 MB/s | 2.4 MB 00:00 2024-09-15T22:26:00.055 INFO:teuthology.orchestra.run.smithi016.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-15T22:26:00.347 INFO:teuthology.orchestra.run.smithi016.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-15T22:26:00.364 INFO:teuthology.orchestra.run.smithi130.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 33 kB/s | 6.5 kB 00:00 2024-09-15T22:26:00.545 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T22:26:00.572 INFO:teuthology.orchestra.run.smithi016.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 21 MB 00:01 2024-09-15T22:26:00.589 INFO:teuthology.orchestra.run.smithi130.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.9 MB/s | 896 kB 00:00 2024-09-15T22:26:00.614 INFO:teuthology.orchestra.run.smithi016.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 609 kB/s | 25 kB 00:00 2024-09-15T22:26:00.656 INFO:teuthology.orchestra.run.smithi130.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.1 MB/s | 145 kB 00:00 2024-09-15T22:26:01.031 INFO:teuthology.orchestra.run.smithi016.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 16 MB/s | 11 MB 00:00 2024-09-15T22:26:01.073 INFO:teuthology.orchestra.run.smithi130.stdout:(4/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 5.7 MB/s | 2.4 MB 00:00 2024-09-15T22:26:01.182 INFO:teuthology.orchestra.run.smithi130.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 1.5 MB 00:00 2024-09-15T22:26:01.214 INFO:teuthology.orchestra.run.smithi016.stdout:(11/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 15 MB/s | 17 MB 00:01 2024-09-15T22:26:01.240 INFO:teuthology.orchestra.run.smithi016.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.1 MB/s | 228 kB 00:00 2024-09-15T22:26:01.265 INFO:teuthology.orchestra.run.smithi016.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 541 kB/s | 27 kB 00:00 2024-09-15T22:26:01.290 INFO:teuthology.orchestra.run.smithi016.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 632 kB/s | 32 kB 00:00 2024-09-15T22:26:01.332 INFO:teuthology.orchestra.run.smithi016.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 751 kB 00:00 2024-09-15T22:26:01.366 INFO:teuthology.orchestra.run.smithi016.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.1 MB/s | 163 kB 00:00 2024-09-15T22:26:01.416 INFO:teuthology.orchestra.run.smithi016.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.5 MB/s | 127 kB 00:00 2024-09-15T22:26:01.450 INFO:teuthology.orchestra.run.smithi016.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-15T22:26:01.499 INFO:teuthology.orchestra.run.smithi130.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-15T22:26:01.525 INFO:teuthology.orchestra.run.smithi016.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 602 kB/s | 45 kB 00:00 2024-09-15T22:26:01.566 INFO:teuthology.orchestra.run.smithi130.stdout:(7/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-09-15T22:26:01.609 INFO:teuthology.orchestra.run.smithi016.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-15T22:26:01.709 INFO:teuthology.orchestra.run.smithi016.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.6 MB/s | 161 kB 00:00 2024-09-15T22:26:01.784 INFO:teuthology.orchestra.run.smithi016.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-15T22:26:01.818 INFO:teuthology.orchestra.run.smithi016.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.9 MB/s | 320 kB 00:00 2024-09-15T22:26:01.860 INFO:teuthology.orchestra.run.smithi016.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.9 MB/s | 300 kB 00:00 2024-09-15T22:26:01.894 INFO:teuthology.orchestra.run.smithi016.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.3 MB/s | 99 kB 00:00 2024-09-15T22:26:01.936 INFO:teuthology.orchestra.run.smithi016.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-15T22:26:02.036 INFO:teuthology.orchestra.run.smithi016.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-09-15T22:26:02.169 INFO:teuthology.orchestra.run.smithi016.stdout:(28/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-15T22:26:02.195 INFO:teuthology.orchestra.run.smithi016.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 200 kB/s | 32 kB 00:00 2024-09-15T22:26:02.254 INFO:teuthology.orchestra.run.smithi016.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.7 MB/s | 146 kB 00:00 2024-09-15T22:26:02.604 INFO:teuthology.orchestra.run.smithi016.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 8.7 MB/s | 3.6 MB 00:00 2024-09-15T22:26:02.696 INFO:teuthology.orchestra.run.smithi016.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.7 MB/s | 249 kB 00:00 2024-09-15T22:26:02.779 INFO:teuthology.orchestra.run.smithi016.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 593 kB/s | 49 kB 00:00 2024-09-15T22:26:02.863 INFO:teuthology.orchestra.run.smithi016.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 197 kB/s | 16 kB 00:00 2024-09-15T22:26:02.889 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:26:02.890 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:26:02.902 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:26:02.903 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:26:02.904 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:26:02.905 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:26:02.906 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:26:02.909 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:26:02.909 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:26:02.916 INFO:teuthology.orchestra.run.smithi130.stdout:(8/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 7.9 MB/s | 11 MB 00:01 2024-09-15T22:26:02.974 INFO:teuthology.orchestra.run.smithi130.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 434 kB/s | 25 kB 00:00 2024-09-15T22:26:02.977 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-09-15T22:26:02.989 INFO:teuthology.orchestra.run.smithi016.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 10 MB/s | 7.4 MB 00:00 2024-09-15T22:26:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:====================================================================================== 2024-09-15T22:26:03.005 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-09-15T22:26:03.005 INFO:teuthology.orchestra.run.smithi059.stdout:====================================================================================== 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:26:03.006 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:26:03.007 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:26:03.008 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:26:03.009 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:26:03.010 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:26:03.011 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:26:03.012 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:26:03.013 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:26:03.014 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:26:03.015 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:26:03.016 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:26:03.017 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout:====================================================================================== 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout:Install 148 Packages 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 8 Packages 2024-09-15T22:26:03.018 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:03.019 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 226 M 2024-09-15T22:26:03.019 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-09-15T22:26:03.022 INFO:teuthology.orchestra.run.smithi016.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 1.8 MB/s | 285 kB 00:00 2024-09-15T22:26:03.056 INFO:teuthology.orchestra.run.smithi016.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 11 MB/s | 765 kB 00:00 2024-09-15T22:26:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 311 kB/s | 18 kB 00:00 2024-09-15T22:26:03.165 INFO:teuthology.orchestra.run.smithi016.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-15T22:26:03.191 INFO:teuthology.orchestra.run.smithi130.stdout:(10/156): ceph-common-19.1.1-326.g8921c8ef.el9. 7.0 MB/s | 21 MB 00:03 2024-09-15T22:26:03.223 INFO:teuthology.orchestra.run.smithi016.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T22:26:03.249 INFO:teuthology.orchestra.run.smithi016.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-09-15T22:26:03.282 INFO:teuthology.orchestra.run.smithi016.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-15T22:26:03.333 INFO:teuthology.orchestra.run.smithi016.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-15T22:26:03.358 INFO:teuthology.orchestra.run.smithi016.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-15T22:26:03.375 INFO:teuthology.orchestra.run.smithi130.stdout:(11/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 9.0 MB/s | 17 MB 00:01 2024-09-15T22:26:03.383 INFO:teuthology.orchestra.run.smithi016.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-15T22:26:03.400 INFO:teuthology.orchestra.run.smithi130.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.1 MB/s | 228 kB 00:00 2024-09-15T22:26:03.409 INFO:teuthology.orchestra.run.smithi016.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T22:26:03.417 INFO:teuthology.orchestra.run.smithi130.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 650 kB/s | 27 kB 00:00 2024-09-15T22:26:03.429 INFO:teuthology.orchestra.run.smithi059.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-15T22:26:03.442 INFO:teuthology.orchestra.run.smithi130.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 759 kB/s | 32 kB 00:00 2024-09-15T22:26:03.442 INFO:teuthology.orchestra.run.smithi016.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-15T22:26:03.501 INFO:teuthology.orchestra.run.smithi130.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 8.8 MB/s | 751 kB 00:00 2024-09-15T22:26:03.526 INFO:teuthology.orchestra.run.smithi130.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.9 MB/s | 163 kB 00:00 2024-09-15T22:26:03.551 INFO:teuthology.orchestra.run.smithi130.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-15T22:26:03.585 INFO:teuthology.orchestra.run.smithi130.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 8.5 MB/s | 506 kB 00:00 2024-09-15T22:26:03.604 INFO:teuthology.orchestra.run.smithi059.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 5.0 MB/s | 896 kB 00:00 2024-09-15T22:26:03.644 INFO:teuthology.orchestra.run.smithi130.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 799 kB/s | 45 kB 00:00 2024-09-15T22:26:03.688 INFO:teuthology.orchestra.run.smithi059.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.7 MB/s | 145 kB 00:00 2024-09-15T22:26:03.725 INFO:teuthology.orchestra.run.smithi130.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-15T22:26:03.817 INFO:teuthology.orchestra.run.smithi130.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.7 MB/s | 161 kB 00:00 2024-09-15T22:26:03.893 INFO:teuthology.orchestra.run.smithi016.stdout:(48/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 15 MB/s | 49 MB 00:03 2024-09-15T22:26:03.901 INFO:teuthology.orchestra.run.smithi130.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-15T22:26:03.914 INFO:teuthology.orchestra.run.smithi059.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 8.9 MB/s | 5.4 MB 00:00 2024-09-15T22:26:03.968 INFO:teuthology.orchestra.run.smithi130.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 4.4 MB/s | 300 kB 00:00 2024-09-15T22:26:03.972 INFO:teuthology.orchestra.run.smithi059.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 8.4 MB/s | 2.4 MB 00:00 2024-09-15T22:26:04.026 INFO:teuthology.orchestra.run.smithi130.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-15T22:26:04.056 INFO:teuthology.orchestra.run.smithi059.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-15T22:26:04.102 INFO:teuthology.orchestra.run.smithi130.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-15T22:26:04.185 INFO:teuthology.orchestra.run.smithi130.stdout:(26/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 8.4 MB/s | 5.3 MB 00:00 2024-09-15T22:26:04.431 INFO:teuthology.orchestra.run.smithi059.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-15T22:26:04.519 INFO:teuthology.orchestra.run.smithi130.stdout:(27/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 7.4 MB/s | 3.1 MB 00:00 2024-09-15T22:26:04.576 INFO:teuthology.orchestra.run.smithi016.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 156 kB/s | 107 kB 00:00 2024-09-15T22:26:04.594 INFO:teuthology.orchestra.run.smithi130.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 426 kB/s | 32 kB 00:00 2024-09-15T22:26:04.677 INFO:teuthology.orchestra.run.smithi130.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.7 MB/s | 146 kB 00:00 2024-09-15T22:26:04.693 INFO:teuthology.orchestra.run.smithi016.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 537 kB/s | 689 kB 00:01 2024-09-15T22:26:04.818 INFO:teuthology.orchestra.run.smithi016.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 685 kB/s | 943 kB 00:01 2024-09-15T22:26:04.919 INFO:teuthology.orchestra.run.smithi130.stdout:(30/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 232 kB/s | 170 kB 00:00 2024-09-15T22:26:05.045 INFO:teuthology.orchestra.run.smithi130.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 9.7 MB/s | 3.6 MB 00:00 2024-09-15T22:26:05.048 INFO:teuthology.orchestra.run.smithi059.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 21 MB 00:01 2024-09-15T22:26:05.103 INFO:teuthology.orchestra.run.smithi130.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.2 MB/s | 249 kB 00:00 2024-09-15T22:26:05.106 INFO:teuthology.orchestra.run.smithi059.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 433 kB/s | 25 kB 00:00 2024-09-15T22:26:05.154 INFO:teuthology.orchestra.run.smithi130.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 990 kB/s | 49 kB 00:00 2024-09-15T22:26:05.169 INFO:teuthology.orchestra.run.smithi016.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 37 kB/s | 22 kB 00:00 2024-09-15T22:26:05.204 INFO:teuthology.orchestra.run.smithi130.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 328 kB/s | 16 kB 00:00 2024-09-15T22:26:05.254 INFO:teuthology.orchestra.run.smithi130.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.6 MB/s | 285 kB 00:00 2024-09-15T22:26:05.307 INFO:teuthology.orchestra.run.smithi059.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 13 MB/s | 17 MB 00:01 2024-09-15T22:26:05.363 INFO:teuthology.orchestra.run.smithi130.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.9 MB/s | 765 kB 00:00 2024-09-15T22:26:05.377 INFO:teuthology.orchestra.run.smithi016.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-15T22:26:05.413 INFO:teuthology.orchestra.run.smithi130.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 365 kB/s | 18 kB 00:00 2024-09-15T22:26:05.424 INFO:teuthology.orchestra.run.smithi059.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 11 MB/s | 11 MB 00:00 2024-09-15T22:26:05.457 INFO:teuthology.orchestra.run.smithi059.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.5 MB/s | 228 kB 00:00 2024-09-15T22:26:05.482 INFO:teuthology.orchestra.run.smithi059.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 463 kB/s | 27 kB 00:00 2024-09-15T22:26:05.502 INFO:teuthology.orchestra.run.smithi016.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 387 kB/s | 313 kB 00:00 2024-09-15T22:26:05.507 INFO:teuthology.orchestra.run.smithi059.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 633 kB/s | 32 kB 00:00 2024-09-15T22:26:05.558 INFO:teuthology.orchestra.run.smithi059.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 3.2 MB/s | 163 kB 00:00 2024-09-15T22:26:05.583 INFO:teuthology.orchestra.run.smithi059.stdout:(16/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 7.3 MB/s | 751 kB 00:00 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi130.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi059.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.5 MB/s | 127 kB 00:00 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi059.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 8.4 MB/s | 506 kB 00:00 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi059.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 675 kB/s | 45 kB 00:00 2024-09-15T22:26:05.776 INFO:teuthology.orchestra.run.smithi059.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.0 MB/s | 136 kB 00:00 2024-09-15T22:26:05.843 INFO:teuthology.orchestra.run.smithi059.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.4 MB/s | 161 kB 00:00 2024-09-15T22:26:05.853 INFO:teuthology.orchestra.run.smithi016.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-15T22:26:05.872 INFO:teuthology.orchestra.run.smithi130.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 174 kB/s | 80 kB 00:00 2024-09-15T22:26:05.952 INFO:teuthology.orchestra.run.smithi059.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-15T22:26:05.985 INFO:teuthology.orchestra.run.smithi059.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.2 MB/s | 320 kB 00:00 2024-09-15T22:26:05.986 INFO:teuthology.orchestra.run.smithi016.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-15T22:26:06.011 INFO:teuthology.orchestra.run.smithi059.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-15T22:26:06.044 INFO:teuthology.orchestra.run.smithi059.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-15T22:26:06.072 INFO:teuthology.orchestra.run.smithi130.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-15T22:26:06.078 INFO:teuthology.orchestra.run.smithi059.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-15T22:26:06.137 INFO:teuthology.orchestra.run.smithi059.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.9 MB/s | 170 kB 00:00 2024-09-15T22:26:06.198 INFO:teuthology.orchestra.run.smithi130.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-15T22:26:06.203 INFO:teuthology.orchestra.run.smithi059.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 479 kB/s | 32 kB 00:00 2024-09-15T22:26:06.262 INFO:teuthology.orchestra.run.smithi059.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-15T22:26:06.287 INFO:teuthology.orchestra.run.smithi059.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.7 MB/s | 146 kB 00:00 2024-09-15T22:26:06.329 INFO:teuthology.orchestra.run.smithi016.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-15T22:26:06.331 INFO:teuthology.orchestra.run.smithi130.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-15T22:26:06.407 INFO:teuthology.orchestra.run.smithi130.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-09-15T22:26:06.473 INFO:teuthology.orchestra.run.smithi130.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 751 kB/s | 106 kB 00:00 2024-09-15T22:26:06.529 INFO:teuthology.orchestra.run.smithi059.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 13 MB/s | 3.6 MB 00:00 2024-09-15T22:26:06.532 INFO:teuthology.orchestra.run.smithi130.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-15T22:26:06.582 INFO:teuthology.orchestra.run.smithi130.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-15T22:26:06.588 INFO:teuthology.orchestra.run.smithi059.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.2 MB/s | 249 kB 00:00 2024-09-15T22:26:06.595 INFO:teuthology.orchestra.run.smithi016.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 216 kB/s | 160 kB 00:00 2024-09-15T22:26:06.637 INFO:teuthology.orchestra.run.smithi016.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 69 kB/s | 45 kB 00:00 2024-09-15T22:26:06.649 INFO:teuthology.orchestra.run.smithi130.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-15T22:26:06.655 INFO:teuthology.orchestra.run.smithi059.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 739 kB/s | 49 kB 00:00 2024-09-15T22:26:06.741 INFO:teuthology.orchestra.run.smithi130.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-09-15T22:26:06.780 INFO:teuthology.orchestra.run.smithi059.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-15T22:26:06.783 INFO:teuthology.orchestra.run.smithi130.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-15T22:26:06.806 INFO:teuthology.orchestra.run.smithi059.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 109 kB/s | 16 kB 00:00 2024-09-15T22:26:06.825 INFO:teuthology.orchestra.run.smithi130.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-15T22:26:06.839 INFO:teuthology.orchestra.run.smithi059.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.8 MB/s | 285 kB 00:00 2024-09-15T22:26:06.867 INFO:teuthology.orchestra.run.smithi130.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-09-15T22:26:06.881 INFO:teuthology.orchestra.run.smithi059.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-15T22:26:06.892 INFO:teuthology.orchestra.run.smithi130.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-15T22:26:06.906 INFO:teuthology.orchestra.run.smithi059.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 273 kB/s | 18 kB 00:00 2024-09-15T22:26:06.909 INFO:teuthology.orchestra.run.smithi130.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 780 kB/s | 33 kB 00:00 2024-09-15T22:26:06.943 INFO:teuthology.orchestra.run.smithi130.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 550 kB/s | 18 kB 00:00 2024-09-15T22:26:07.002 INFO:teuthology.orchestra.run.smithi130.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-09-15T22:26:07.019 INFO:teuthology.orchestra.run.smithi130.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-15T22:26:07.023 INFO:teuthology.orchestra.run.smithi059.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-15T22:26:07.044 INFO:teuthology.orchestra.run.smithi130.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-09-15T22:26:07.069 INFO:teuthology.orchestra.run.smithi130.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-15T22:26:07.099 INFO:teuthology.orchestra.run.smithi059.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-15T22:26:07.111 INFO:teuthology.orchestra.run.smithi130.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-15T22:26:07.124 INFO:teuthology.orchestra.run.smithi059.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-15T22:26:07.145 INFO:teuthology.orchestra.run.smithi130.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-15T22:26:07.149 INFO:teuthology.orchestra.run.smithi059.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-15T22:26:07.162 INFO:teuthology.orchestra.run.smithi016.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 47 kB/s | 24 kB 00:00 2024-09-15T22:26:07.170 INFO:teuthology.orchestra.run.smithi130.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-15T22:26:07.195 INFO:teuthology.orchestra.run.smithi130.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-15T22:26:07.198 INFO:teuthology.orchestra.run.smithi059.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-15T22:26:07.213 INFO:teuthology.orchestra.run.smithi016.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 750 kB/s | 662 kB 00:00 2024-09-15T22:26:07.232 INFO:teuthology.orchestra.run.smithi059.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-15T22:26:07.257 INFO:teuthology.orchestra.run.smithi059.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-15T22:26:07.282 INFO:teuthology.orchestra.run.smithi059.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T22:26:07.308 INFO:teuthology.orchestra.run.smithi059.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-15T22:26:07.447 INFO:teuthology.orchestra.run.smithi016.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 683 kB/s | 581 kB 00:00 2024-09-15T22:26:07.646 INFO:teuthology.orchestra.run.smithi130.stdout:(63/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 10 MB/s | 49 MB 00:04 2024-09-15T22:26:07.687 INFO:teuthology.orchestra.run.smithi130.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 565 kB/s | 292 kB 00:00 2024-09-15T22:26:07.721 INFO:teuthology.orchestra.run.smithi130.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 359 kB/s | 188 kB 00:00 2024-09-15T22:26:07.725 INFO:teuthology.orchestra.run.smithi059.stdout:(48/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 19 MB/s | 49 MB 00:02 2024-09-15T22:26:07.738 INFO:teuthology.orchestra.run.smithi016.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-15T22:26:07.754 INFO:teuthology.orchestra.run.smithi130.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 355 kB/s | 38 kB 00:00 2024-09-15T22:26:07.867 INFO:teuthology.orchestra.run.smithi059.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 756 kB/s | 107 kB 00:00 2024-09-15T22:26:07.872 INFO:teuthology.orchestra.run.smithi130.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-15T22:26:07.917 INFO:teuthology.orchestra.run.smithi059.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 436 kB/s | 22 kB 00:00 2024-09-15T22:26:07.939 INFO:teuthology.orchestra.run.smithi130.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-15T22:26:07.981 INFO:teuthology.orchestra.run.smithi130.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-15T22:26:08.000 INFO:teuthology.orchestra.run.smithi059.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 960 kB/s | 689 kB 00:00 2024-09-15T22:26:08.006 INFO:teuthology.orchestra.run.smithi130.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-15T22:26:08.014 INFO:teuthology.orchestra.run.smithi016.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 365 kB/s | 292 kB 00:00 2024-09-15T22:26:08.048 INFO:teuthology.orchestra.run.smithi130.stdout:(71/156): postgresql-13.16-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-09-15T22:26:08.051 INFO:teuthology.orchestra.run.smithi059.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-15T22:26:08.090 INFO:teuthology.orchestra.run.smithi130.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-15T22:26:08.093 INFO:teuthology.orchestra.run.smithi059.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-09-15T22:26:08.118 INFO:teuthology.orchestra.run.smithi059.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 313 kB 00:00 2024-09-15T22:26:08.143 INFO:teuthology.orchestra.run.smithi059.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-15T22:26:08.169 INFO:teuthology.orchestra.run.smithi059.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-15T22:26:08.172 INFO:teuthology.orchestra.run.smithi130.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 705 kB/s | 87 kB 00:00 2024-09-15T22:26:08.180 INFO:teuthology.orchestra.run.smithi016.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 260 kB/s | 188 kB 00:00 2024-09-15T22:26:08.215 INFO:teuthology.orchestra.run.smithi130.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-15T22:26:08.219 INFO:teuthology.orchestra.run.smithi059.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-09-15T22:26:08.244 INFO:teuthology.orchestra.run.smithi059.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-15T22:26:08.291 INFO:teuthology.orchestra.run.smithi130.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-15T22:26:08.298 INFO:teuthology.orchestra.run.smithi016.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-15T22:26:08.316 INFO:teuthology.orchestra.run.smithi130.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-15T22:26:08.341 INFO:teuthology.orchestra.run.smithi130.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-15T22:26:08.366 INFO:teuthology.orchestra.run.smithi130.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-15T22:26:08.392 INFO:teuthology.orchestra.run.smithi130.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-15T22:26:08.417 INFO:teuthology.orchestra.run.smithi130.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-15T22:26:08.420 INFO:teuthology.orchestra.run.smithi059.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-15T22:26:08.442 INFO:teuthology.orchestra.run.smithi130.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-15T22:26:08.478 INFO:teuthology.orchestra.run.smithi059.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-09-15T22:26:08.503 INFO:teuthology.orchestra.run.smithi059.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-15T22:26:08.529 INFO:teuthology.orchestra.run.smithi059.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-15T22:26:08.534 INFO:teuthology.orchestra.run.smithi130.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-15T22:26:08.559 INFO:teuthology.orchestra.run.smithi130.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-15T22:26:08.584 INFO:teuthology.orchestra.run.smithi130.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-15T22:26:08.610 INFO:teuthology.orchestra.run.smithi130.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-15T22:26:08.612 INFO:teuthology.orchestra.run.smithi059.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-15T22:26:08.635 INFO:teuthology.orchestra.run.smithi130.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-15T22:26:08.638 INFO:teuthology.orchestra.run.smithi059.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-15T22:26:08.660 INFO:teuthology.orchestra.run.smithi130.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-15T22:26:08.663 INFO:teuthology.orchestra.run.smithi059.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-15T22:26:08.685 INFO:teuthology.orchestra.run.smithi130.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-15T22:26:08.711 INFO:teuthology.orchestra.run.smithi130.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-15T22:26:08.728 INFO:teuthology.orchestra.run.smithi130.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-15T22:26:08.744 INFO:teuthology.orchestra.run.smithi130.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-15T22:26:08.820 INFO:teuthology.orchestra.run.smithi130.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-15T22:26:08.837 INFO:teuthology.orchestra.run.smithi130.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-15T22:26:08.854 INFO:teuthology.orchestra.run.smithi130.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-15T22:26:08.863 INFO:teuthology.orchestra.run.smithi059.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-09-15T22:26:08.887 INFO:teuthology.orchestra.run.smithi130.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-15T22:26:08.904 INFO:teuthology.orchestra.run.smithi130.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-09-15T22:26:08.921 INFO:teuthology.orchestra.run.smithi130.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-15T22:26:08.947 INFO:teuthology.orchestra.run.smithi130.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-15T22:26:08.972 INFO:teuthology.orchestra.run.smithi130.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-15T22:26:08.997 INFO:teuthology.orchestra.run.smithi130.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-15T22:26:09.022 INFO:teuthology.orchestra.run.smithi130.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T22:26:09.048 INFO:teuthology.orchestra.run.smithi130.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-15T22:26:09.123 INFO:teuthology.orchestra.run.smithi130.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-15T22:26:09.148 INFO:teuthology.orchestra.run.smithi016.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-09-15T22:26:09.213 INFO:teuthology.orchestra.run.smithi059.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-09-15T22:26:09.298 INFO:teuthology.orchestra.run.smithi016.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-15T22:26:09.307 INFO:teuthology.orchestra.run.smithi130.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-15T22:26:09.365 INFO:teuthology.orchestra.run.smithi130.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-09-15T22:26:09.390 INFO:teuthology.orchestra.run.smithi016.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-09-15T22:26:09.391 INFO:teuthology.orchestra.run.smithi130.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-15T22:26:09.414 INFO:teuthology.orchestra.run.smithi059.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-15T22:26:09.415 INFO:teuthology.orchestra.run.smithi130.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-15T22:26:09.441 INFO:teuthology.orchestra.run.smithi130.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-15T22:26:09.472 INFO:teuthology.orchestra.run.smithi059.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-15T22:26:09.474 INFO:teuthology.orchestra.run.smithi130.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-15T22:26:09.500 INFO:teuthology.orchestra.run.smithi130.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-09-15T22:26:09.542 INFO:teuthology.orchestra.run.smithi130.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-15T22:26:09.567 INFO:teuthology.orchestra.run.smithi130.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 731 kB/s | 67 kB 00:00 2024-09-15T22:26:09.617 INFO:teuthology.orchestra.run.smithi130.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-15T22:26:09.648 INFO:teuthology.orchestra.run.smithi059.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-09-15T22:26:09.651 INFO:teuthology.orchestra.run.smithi130.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-15T22:26:09.676 INFO:teuthology.orchestra.run.smithi130.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-09-15T22:26:09.701 INFO:teuthology.orchestra.run.smithi130.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-09-15T22:26:09.726 INFO:teuthology.orchestra.run.smithi130.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-15T22:26:09.752 INFO:teuthology.orchestra.run.smithi130.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-15T22:26:09.794 INFO:teuthology.orchestra.run.smithi130.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-15T22:26:09.819 INFO:teuthology.orchestra.run.smithi130.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-15T22:26:09.844 INFO:teuthology.orchestra.run.smithi130.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-15T22:26:09.848 INFO:teuthology.orchestra.run.smithi059.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-09-15T22:26:09.906 INFO:teuthology.orchestra.run.smithi059.stdout:(72/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.9 MB/s | 4.9 MB 00:01 2024-09-15T22:26:09.911 INFO:teuthology.orchestra.run.smithi130.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-09-15T22:26:09.931 INFO:teuthology.orchestra.run.smithi059.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-15T22:26:09.936 INFO:teuthology.orchestra.run.smithi130.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-15T22:26:09.962 INFO:teuthology.orchestra.run.smithi130.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-09-15T22:26:09.978 INFO:teuthology.orchestra.run.smithi130.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-09-15T22:26:09.981 INFO:teuthology.orchestra.run.smithi059.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-15T22:26:10.004 INFO:teuthology.orchestra.run.smithi130.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 555 kB/s | 23 kB 00:00 2024-09-15T22:26:10.015 INFO:teuthology.orchestra.run.smithi016.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 192 kB/s | 137 kB 00:00 2024-09-15T22:26:10.029 INFO:teuthology.orchestra.run.smithi130.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-15T22:26:10.054 INFO:teuthology.orchestra.run.smithi130.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-15T22:26:10.065 INFO:teuthology.orchestra.run.smithi059.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-15T22:26:10.080 INFO:teuthology.orchestra.run.smithi130.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T22:26:10.082 INFO:teuthology.orchestra.run.smithi016.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 975 kB/s | 910 kB 00:00 2024-09-15T22:26:10.113 INFO:teuthology.orchestra.run.smithi130.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T22:26:10.149 INFO:teuthology.orchestra.run.smithi059.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-15T22:26:10.155 INFO:teuthology.orchestra.run.smithi130.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-09-15T22:26:10.172 INFO:teuthology.orchestra.run.smithi130.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 979 kB/s | 58 kB 00:00 2024-09-15T22:26:10.189 INFO:teuthology.orchestra.run.smithi130.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 985 kB/s | 16 kB 00:00 2024-09-15T22:26:10.199 INFO:teuthology.orchestra.run.smithi059.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-15T22:26:10.215 INFO:teuthology.orchestra.run.smithi130.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-15T22:26:10.240 INFO:teuthology.orchestra.run.smithi130.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-15T22:26:10.249 INFO:teuthology.orchestra.run.smithi059.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-15T22:26:10.257 INFO:teuthology.orchestra.run.smithi130.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-09-15T22:26:10.274 INFO:teuthology.orchestra.run.smithi130.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 917 kB/s | 31 kB 00:00 2024-09-15T22:26:10.308 INFO:teuthology.orchestra.run.smithi059.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-15T22:26:10.324 INFO:teuthology.orchestra.run.smithi130.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-15T22:26:10.359 INFO:teuthology.orchestra.run.smithi059.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-15T22:26:10.374 INFO:teuthology.orchestra.run.smithi130.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-09-15T22:26:10.400 INFO:teuthology.orchestra.run.smithi130.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 477 kB/s | 36 kB 00:00 2024-09-15T22:26:10.425 INFO:teuthology.orchestra.run.smithi130.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-09-15T22:26:10.450 INFO:teuthology.orchestra.run.smithi130.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-15T22:26:10.475 INFO:teuthology.orchestra.run.smithi130.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-15T22:26:10.501 INFO:teuthology.orchestra.run.smithi130.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-15T22:26:10.534 INFO:teuthology.orchestra.run.smithi130.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-15T22:26:10.560 INFO:teuthology.orchestra.run.smithi130.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T22:26:10.593 INFO:teuthology.orchestra.run.smithi130.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-15T22:26:10.633 INFO:teuthology.orchestra.run.smithi016.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-09-15T22:26:10.643 INFO:teuthology.orchestra.run.smithi130.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-15T22:26:10.749 INFO:teuthology.orchestra.run.smithi016.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 131 kB/s | 87 kB 00:00 2024-09-15T22:26:10.784 INFO:teuthology.orchestra.run.smithi059.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-09-15T22:26:10.867 INFO:teuthology.orchestra.run.smithi059.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-15T22:26:10.869 INFO:teuthology.orchestra.run.smithi130.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-15T22:26:10.925 INFO:teuthology.orchestra.run.smithi016.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-15T22:26:10.927 INFO:teuthology.orchestra.run.smithi130.stdout:(150/156): librados2-19.1.1-326.g8921c8ef.el9.x 9.3 MB/s | 3.4 MB 00:00 2024-09-15T22:26:10.960 INFO:teuthology.orchestra.run.smithi059.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-15T22:26:11.035 INFO:teuthology.orchestra.run.smithi059.stdout:(84/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 6.0 MB 00:01 2024-09-15T22:26:11.060 INFO:teuthology.orchestra.run.smithi059.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 771 kB/s | 77 kB 00:00 2024-09-15T22:26:11.085 INFO:teuthology.orchestra.run.smithi059.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-15T22:26:11.135 INFO:teuthology.orchestra.run.smithi059.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-15T22:26:11.169 INFO:teuthology.orchestra.run.smithi059.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-15T22:26:11.212 INFO:teuthology.orchestra.run.smithi130.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 144 kB/s | 41 kB 00:00 2024-09-15T22:26:11.236 INFO:teuthology.orchestra.run.smithi059.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.8 MB/s | 5.1 MB 00:00 2024-09-15T22:26:11.261 INFO:teuthology.orchestra.run.smithi059.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-09-15T22:26:11.287 INFO:teuthology.orchestra.run.smithi059.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-15T22:26:11.312 INFO:teuthology.orchestra.run.smithi059.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-15T22:26:11.337 INFO:teuthology.orchestra.run.smithi059.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-15T22:26:11.362 INFO:teuthology.orchestra.run.smithi059.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 878 kB/s | 44 kB 00:00 2024-09-15T22:26:11.396 INFO:teuthology.orchestra.run.smithi059.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-15T22:26:11.421 INFO:teuthology.orchestra.run.smithi059.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-15T22:26:11.436 INFO:teuthology.orchestra.run.smithi130.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.9 MB/s | 2.3 MB 00:00 2024-09-15T22:26:11.446 INFO:teuthology.orchestra.run.smithi059.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-15T22:26:11.450 INFO:teuthology.orchestra.run.smithi016.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 87 kB/s | 61 kB 00:00 2024-09-15T22:26:11.497 INFO:teuthology.orchestra.run.smithi059.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-15T22:26:11.522 INFO:teuthology.orchestra.run.smithi059.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-15T22:26:11.564 INFO:teuthology.orchestra.run.smithi059.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-15T22:26:11.578 INFO:teuthology.orchestra.run.smithi130.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-15T22:26:11.603 INFO:teuthology.orchestra.run.smithi130.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 634 kB/s | 249 kB 00:00 2024-09-15T22:26:11.614 INFO:teuthology.orchestra.run.smithi059.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 692 kB/s | 64 kB 00:00 2024-09-15T22:26:11.640 INFO:teuthology.orchestra.run.smithi059.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-15T22:26:11.645 INFO:teuthology.orchestra.run.smithi130.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 2.3 MB/s | 1.8 MB 00:00 2024-09-15T22:26:11.665 INFO:teuthology.orchestra.run.smithi059.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-09-15T22:26:11.700 INFO:teuthology.orchestra.run.smithi016.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 316 kB/s | 245 kB 00:00 2024-09-15T22:26:11.737 INFO:teuthology.orchestra.run.smithi130.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-15T22:26:11.743 INFO:teuthology.orchestra.run.smithi130.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:26:11.743 INFO:teuthology.orchestra.run.smithi130.stdout:Total 19 MB/s | 226 MB 00:12 2024-09-15T22:26:11.757 INFO:teuthology.orchestra.run.smithi059.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-09-15T22:26:11.799 INFO:teuthology.orchestra.run.smithi059.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-15T22:26:11.801 INFO:teuthology.orchestra.run.smithi016.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-15T22:26:11.824 INFO:teuthology.orchestra.run.smithi059.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-09-15T22:26:11.849 INFO:teuthology.orchestra.run.smithi059.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-15T22:26:11.883 INFO:teuthology.orchestra.run.smithi059.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-15T22:26:11.917 INFO:teuthology.orchestra.run.smithi059.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-15T22:26:11.942 INFO:teuthology.orchestra.run.smithi059.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T22:26:11.975 INFO:teuthology.orchestra.run.smithi059.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-15T22:26:12.043 INFO:teuthology.orchestra.run.smithi059.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-15T22:26:12.110 INFO:teuthology.orchestra.run.smithi059.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-15T22:26:12.135 INFO:teuthology.orchestra.run.smithi059.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T22:26:12.160 INFO:teuthology.orchestra.run.smithi059.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T22:26:12.185 INFO:teuthology.orchestra.run.smithi059.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 569 kB/s | 14 kB 00:00 2024-09-15T22:26:12.219 INFO:teuthology.orchestra.run.smithi059.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-15T22:26:12.259 INFO:teuthology.orchestra.run.smithi016.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 307 kB/s | 248 kB 00:00 2024-09-15T22:26:12.284 INFO:teuthology.orchestra.run.smithi016.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 44 kB/s | 26 kB 00:00 2024-09-15T22:26:12.286 INFO:teuthology.orchestra.run.smithi059.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-09-15T22:26:12.311 INFO:teuthology.orchestra.run.smithi059.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-15T22:26:12.326 INFO:teuthology.orchestra.run.smithi016.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 37 kB/s | 19 kB 00:00 2024-09-15T22:26:12.337 INFO:teuthology.orchestra.run.smithi059.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-15T22:26:12.362 INFO:teuthology.orchestra.run.smithi059.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-15T22:26:12.396 INFO:teuthology.orchestra.run.smithi059.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-15T22:26:12.421 INFO:teuthology.orchestra.run.smithi059.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-15T22:26:12.446 INFO:teuthology.orchestra.run.smithi059.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-15T22:26:12.471 INFO:teuthology.orchestra.run.smithi059.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-09-15T22:26:12.497 INFO:teuthology.orchestra.run.smithi059.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-15T22:26:12.522 INFO:teuthology.orchestra.run.smithi059.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-15T22:26:12.572 INFO:teuthology.orchestra.run.smithi059.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-15T22:26:12.598 INFO:teuthology.orchestra.run.smithi059.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T22:26:12.631 INFO:teuthology.orchestra.run.smithi059.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-09-15T22:26:12.657 INFO:teuthology.orchestra.run.smithi059.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-15T22:26:12.682 INFO:teuthology.orchestra.run.smithi059.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-15T22:26:12.707 INFO:teuthology.orchestra.run.smithi059.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-15T22:26:12.741 INFO:teuthology.orchestra.run.smithi059.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-15T22:26:12.774 INFO:teuthology.orchestra.run.smithi059.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-09-15T22:26:12.791 INFO:teuthology.orchestra.run.smithi059.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-15T22:26:12.808 INFO:teuthology.orchestra.run.smithi059.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 920 kB/s | 31 kB 00:00 2024-09-15T22:26:12.825 INFO:teuthology.orchestra.run.smithi059.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-15T22:26:12.842 INFO:teuthology.orchestra.run.smithi059.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-09-15T22:26:12.843 INFO:teuthology.orchestra.run.smithi016.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-15T22:26:12.867 INFO:teuthology.orchestra.run.smithi059.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-15T22:26:12.893 INFO:teuthology.orchestra.run.smithi059.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-15T22:26:12.918 INFO:teuthology.orchestra.run.smithi059.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-15T22:26:12.952 INFO:teuthology.orchestra.run.smithi059.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-15T22:26:12.971 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction check 2024-09-15T22:26:12.977 INFO:teuthology.orchestra.run.smithi059.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-15T22:26:13.002 INFO:teuthology.orchestra.run.smithi059.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-15T22:26:13.010 INFO:teuthology.orchestra.run.smithi016.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 230 kB/s | 172 kB 00:00 2024-09-15T22:26:13.028 INFO:teuthology.orchestra.run.smithi059.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-15T22:26:13.095 INFO:teuthology.orchestra.run.smithi059.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-15T22:26:13.121 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction check succeeded. 2024-09-15T22:26:13.121 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction test 2024-09-15T22:26:13.270 INFO:teuthology.orchestra.run.smithi059.stdout:(148/156): librados2-19.1.1-326.g8921c8ef.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-15T22:26:13.370 INFO:teuthology.orchestra.run.smithi059.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-15T22:26:13.452 INFO:teuthology.orchestra.run.smithi016.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-15T22:26:13.454 INFO:teuthology.orchestra.run.smithi059.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-15T22:26:13.496 INFO:teuthology.orchestra.run.smithi059.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 984 kB/s | 41 kB 00:00 2024-09-15T22:26:13.538 INFO:teuthology.orchestra.run.smithi059.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.8 MB/s | 249 kB 00:00 2024-09-15T22:26:13.580 INFO:teuthology.orchestra.run.smithi059.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 8.6 MB/s | 1.8 MB 00:00 2024-09-15T22:26:13.619 INFO:teuthology.orchestra.run.smithi016.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 323 kB/s | 250 kB 00:00 2024-09-15T22:26:13.622 INFO:teuthology.orchestra.run.smithi059.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 16 MB/s | 1.4 MB 00:00 2024-09-15T22:26:13.680 INFO:teuthology.orchestra.run.smithi059.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-15T22:26:13.769 INFO:teuthology.orchestra.run.smithi016.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 293 kB/s | 222 kB 00:00 2024-09-15T22:26:14.120 INFO:teuthology.orchestra.run.smithi016.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 116 kB/s | 77 kB 00:00 2024-09-15T22:26:14.203 INFO:teuthology.orchestra.run.smithi016.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 72 kB/s | 42 kB 00:00 2024-09-15T22:26:14.497 INFO:teuthology.orchestra.run.smithi059.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.0 MB/s | 16 MB 00:03 2024-09-15T22:26:14.503 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:26:14.503 INFO:teuthology.orchestra.run.smithi059.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-15T22:26:14.537 INFO:teuthology.orchestra.run.smithi016.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 351 kB/s | 269 kB 00:00 2024-09-15T22:26:14.846 INFO:teuthology.orchestra.run.smithi016.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 219 kB/s | 159 kB 00:00 2024-09-15T22:26:14.913 INFO:teuthology.orchestra.run.smithi130.stdout:Transaction test succeeded. 2024-09-15T22:26:14.913 INFO:teuthology.orchestra.run.smithi130.stdout:Running transaction 2024-09-15T22:26:15.088 INFO:teuthology.orchestra.run.smithi016.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 315 kB/s | 279 kB 00:00 2024-09-15T22:26:15.154 INFO:teuthology.orchestra.run.smithi016.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-15T22:26:15.588 INFO:teuthology.orchestra.run.smithi016.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 72 kB/s | 54 kB 00:00 2024-09-15T22:26:15.754 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-09-15T22:26:15.805 INFO:teuthology.orchestra.run.smithi016.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 64 kB/s | 42 kB 00:00 2024-09-15T22:26:15.918 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-09-15T22:26:15.918 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-09-15T22:26:16.172 INFO:teuthology.orchestra.run.smithi016.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 76 kB/s | 44 kB 00:00 2024-09-15T22:26:16.664 INFO:teuthology.orchestra.run.smithi016.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 260 kB/s | 221 kB 00:00 2024-09-15T22:26:16.773 INFO:teuthology.orchestra.run.smithi016.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 60 kB/s | 36 kB 00:00 2024-09-15T22:26:17.240 INFO:teuthology.orchestra.run.smithi016.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-15T22:26:17.565 INFO:teuthology.orchestra.run.smithi016.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-09-15T22:26:17.699 INFO:teuthology.orchestra.run.smithi016.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 329 kB/s | 304 kB 00:00 2024-09-15T22:26:17.818 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-09-15T22:26:17.819 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-09-15T22:26:17.982 INFO:teuthology.orchestra.run.smithi016.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 78 kB/s | 58 kB 00:00 2024-09-15T22:26:18.316 INFO:teuthology.orchestra.run.smithi016.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-09-15T22:26:18.341 INFO:teuthology.orchestra.run.smithi016.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 125 kB/s | 96 kB 00:00 2024-09-15T22:26:18.408 INFO:teuthology.orchestra.run.smithi016.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 246 kB/s | 22 kB 00:00 2024-09-15T22:26:18.491 INFO:teuthology.orchestra.run.smithi016.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-15T22:26:18.533 INFO:teuthology.orchestra.run.smithi016.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-15T22:26:18.559 INFO:teuthology.orchestra.run.smithi016.stdout:(106/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-15T22:26:18.584 INFO:teuthology.orchestra.run.smithi016.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 396 kB/s | 19 kB 00:00 2024-09-15T22:26:18.626 INFO:teuthology.orchestra.run.smithi016.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-15T22:26:18.651 INFO:teuthology.orchestra.run.smithi016.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-09-15T22:26:18.685 INFO:teuthology.orchestra.run.smithi016.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T22:26:18.718 INFO:teuthology.orchestra.run.smithi016.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 540 kB/s | 49 kB 00:00 2024-09-15T22:26:18.743 INFO:teuthology.orchestra.run.smithi016.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-15T22:26:18.819 INFO:teuthology.orchestra.run.smithi016.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-15T22:26:18.844 INFO:teuthology.orchestra.run.smithi016.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T22:26:18.870 INFO:teuthology.orchestra.run.smithi016.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-09-15T22:26:18.895 INFO:teuthology.orchestra.run.smithi016.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 636 kB/s | 32 kB 00:00 2024-09-15T22:26:18.920 INFO:teuthology.orchestra.run.smithi016.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-15T22:26:18.954 INFO:teuthology.orchestra.run.smithi016.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-15T22:26:18.988 INFO:teuthology.orchestra.run.smithi016.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-15T22:26:19.013 INFO:teuthology.orchestra.run.smithi016.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-15T22:26:19.055 INFO:teuthology.orchestra.run.smithi016.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-15T22:26:19.089 INFO:teuthology.orchestra.run.smithi016.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-15T22:26:19.114 INFO:teuthology.orchestra.run.smithi016.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-15T22:26:19.139 INFO:teuthology.orchestra.run.smithi016.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-15T22:26:19.215 INFO:teuthology.orchestra.run.smithi016.stdout:(125/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-09-15T22:26:19.240 INFO:teuthology.orchestra.run.smithi016.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-09-15T22:26:19.265 INFO:teuthology.orchestra.run.smithi016.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-15T22:26:19.291 INFO:teuthology.orchestra.run.smithi016.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-15T22:26:19.324 INFO:teuthology.orchestra.run.smithi016.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-15T22:26:19.358 INFO:teuthology.orchestra.run.smithi016.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T22:26:19.383 INFO:teuthology.orchestra.run.smithi016.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T22:26:19.425 INFO:teuthology.orchestra.run.smithi016.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-15T22:26:19.459 INFO:teuthology.orchestra.run.smithi016.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-15T22:26:19.484 INFO:teuthology.orchestra.run.smithi016.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-15T22:26:19.509 INFO:teuthology.orchestra.run.smithi016.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-15T22:26:19.560 INFO:teuthology.orchestra.run.smithi016.stdout:(136/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-15T22:26:19.585 INFO:teuthology.orchestra.run.smithi016.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-09-15T22:26:19.610 INFO:teuthology.orchestra.run.smithi016.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-09-15T22:26:19.635 INFO:teuthology.orchestra.run.smithi016.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-15T22:26:19.661 INFO:teuthology.orchestra.run.smithi016.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-15T22:26:19.686 INFO:teuthology.orchestra.run.smithi016.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-15T22:26:19.711 INFO:teuthology.orchestra.run.smithi016.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-15T22:26:19.737 INFO:teuthology.orchestra.run.smithi016.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-15T22:26:19.762 INFO:teuthology.orchestra.run.smithi016.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T22:26:19.787 INFO:teuthology.orchestra.run.smithi016.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-15T22:26:19.813 INFO:teuthology.orchestra.run.smithi016.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-15T22:26:19.841 INFO:teuthology.orchestra.run.smithi016.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.2 MB/s | 427 kB 00:00 2024-09-15T22:26:19.908 INFO:teuthology.orchestra.run.smithi016.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-15T22:26:20.084 INFO:teuthology.orchestra.run.smithi016.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-15T22:26:20.142 INFO:teuthology.orchestra.run.smithi016.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-15T22:26:20.238 INFO:teuthology.orchestra.run.smithi016.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 445 kB/s | 41 kB 00:00 2024-09-15T22:26:20.276 INFO:teuthology.orchestra.run.smithi016.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 9.4 MB/s | 1.8 MB 00:00 2024-09-15T22:26:20.343 INFO:teuthology.orchestra.run.smithi016.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-15T22:26:20.368 INFO:teuthology.orchestra.run.smithi016.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.8 MB/s | 249 kB 00:00 2024-09-15T22:26:20.427 INFO:teuthology.orchestra.run.smithi016.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-15T22:26:20.748 INFO:teuthology.orchestra.run.smithi016.stdout:(156/156): NetworkManager-1.49.90-1.el9.x86_64. 2.8 MB/s | 2.3 MB 00:00 2024-09-15T22:26:20.755 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:26:20.755 INFO:teuthology.orchestra.run.smithi016.stdout:Total 10 MB/s | 226 MB 00:21 2024-09-15T22:26:21.146 INFO:teuthology.orchestra.run.smithi130.stdout: Preparing : 1/1 2024-09-15T22:26:21.249 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:26:21.371 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:26:21.550 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:26:21.592 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:26:21.617 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:26:21.668 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:26:21.998 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:26:22.006 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-15T22:26:22.129 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:26:22.176 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-15T22:26:22.176 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-15T22:26:22.257 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:26:22.285 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:26:22.376 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:26:22.406 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:26:22.563 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:26:22.620 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:26:22.647 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:26:22.743 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:26:22.773 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:26:22.827 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:26:22.858 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:26:22.886 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:26:23.075 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:26:23.203 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:26:23.331 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:26:23.442 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:23.527 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:23.749 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:23.860 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:26:23.954 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:26:23.982 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:26:24.034 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:26:24.115 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:26:24.159 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:26:24.160 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-09-15T22:26:24.199 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:26:24.243 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:26:24.275 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:26:24.301 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:26:24.303 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:26:24.373 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:26:24.415 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:26:24.426 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:26:24.464 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:26:24.499 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:26:24.510 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-15T22:26:24.510 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-15T22:26:24.575 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:26:24.587 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:26:24.610 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:26:24.628 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:26:24.653 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:26:24.661 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:26:24.682 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:26:24.706 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:26:24.754 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:26:24.797 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:26:24.872 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:26:25.014 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:26:25.052 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:26:25.074 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:26:25.153 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:26:25.278 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:26:25.304 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:26:25.396 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:26:25.455 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:26:25.601 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:26:25.628 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:26:25.684 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:26:25.713 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:26:25.730 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:26:25.757 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:26:25.764 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:26:25.802 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:26:25.844 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:26:25.863 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:26:25.892 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:26:25.955 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:26:25.976 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:26:26.000 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:26:26.008 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:26:26.055 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:26:26.110 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:26:26.183 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:26:26.212 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:26:26.214 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:26:26.259 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:26:26.314 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:26:26.350 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:26:26.364 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:26:26.393 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:26:26.435 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:26:26.466 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:26:26.557 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:26.572 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:26:26.651 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:26.762 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:26:26.820 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:26:26.893 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:26.923 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:26:26.962 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:26:27.001 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:26:27.004 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:26:27.027 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:26:27.051 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:26:27.106 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:26:27.127 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:26:27.139 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:26:27.174 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:26:27.252 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:26:27.258 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:26:27.300 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:26:27.308 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:26:27.328 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:26:27.360 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:26:27.362 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:26:27.401 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:26:27.412 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:26:27.433 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:26:27.444 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:26:27.463 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:26:27.472 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:26:27.505 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:26:27.539 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:26:27.544 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:26:27.580 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:26:27.595 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:26:27.615 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:26:27.633 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:26:27.668 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:26:27.701 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:26:27.745 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:26:27.747 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:26:27.776 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:26:27.779 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:26:27.831 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:26:27.867 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:26:27.879 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:26:27.935 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:26:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:26:27.991 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:26:28.069 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:26:28.216 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:26:28.278 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:26:28.660 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:26:28.707 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:26:28.741 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:26:28.839 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:26:28.839 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:26:28.896 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:26:28.924 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:26:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:26:28.959 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:26:29.002 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:26:29.038 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:26:29.040 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:26:29.076 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:26:29.082 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:26:29.237 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:26:29.261 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:26:29.291 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:26:29.301 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:26:29.368 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:26:29.373 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:26:29.447 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:26:29.484 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:26:29.604 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:26:29.745 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:26:29.778 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:26:29.823 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:26:29.873 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:26:30.017 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:26:30.025 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:26:30.073 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:26:30.115 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:26:30.132 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:26:30.221 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:26:30.256 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:26:30.295 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:26:30.321 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:26:30.346 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:26:30.433 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:26:30.568 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:26:30.620 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:26:30.647 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:26:30.681 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:26:30.721 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:26:30.761 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:26:30.790 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:26:30.825 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:26:30.859 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:26:30.900 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:26:30.934 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:26:31.033 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:26:31.090 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:26:31.121 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:26:31.229 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:26:31.282 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:26:31.362 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:26:31.439 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-15T22:26:31.445 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:26:31.473 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:26:31.506 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:26:31.540 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:26:31.581 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:26:31.644 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:26:31.682 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:26:31.708 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:26:31.766 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:26:31.815 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:26:31.869 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:26:31.960 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:26:31.990 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:26:32.027 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:26:32.055 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:26:32.103 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:26:32.123 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:26:32.136 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:26:32.234 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:26:32.243 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:26:32.292 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:26:32.345 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:26:32.351 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:26:32.377 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:26:32.436 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:26:32.452 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:26:32.472 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:26:32.487 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:26:32.508 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:26:32.555 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:26:32.627 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:26:32.657 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:26:32.661 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:26:32.681 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:26:32.694 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:26:32.738 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:26:32.764 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:26:32.773 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:26:32.870 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:26:32.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:26:32.962 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:26:32.983 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:26:33.006 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:26:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:26:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:26:33.463 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:26:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:26:33.583 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:26:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:26:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:26:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:26:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:26:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:26:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:26:34.441 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:26:34.493 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:26:34.526 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:26:34.553 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:26:34.624 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:26:34.676 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:26:34.706 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:26:34.750 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:26:34.828 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:26:34.869 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:26:34.897 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:26:34.921 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:26:34.957 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:26:34.982 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:26:35.011 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:26:35.030 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:26:35.044 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:26:35.073 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:26:35.153 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:26:35.264 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:35.294 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:35.295 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:26:35.351 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:26:35.379 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:26:35.423 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:26:35.506 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:26:35.536 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:26:35.572 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:26:35.841 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:35.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:26:35.906 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:26:35.976 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:26:35.976 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:26:35.976 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:26:35.976 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:36.006 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:26:36.007 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:26:36.041 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:26:36.080 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:26:36.085 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:26:36.109 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:26:36.139 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:26:36.181 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:26:36.190 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:26:36.219 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:26:36.358 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:26:36.426 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:26:36.482 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:26:36.556 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:26:36.594 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:26:36.630 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:26:36.686 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:26:36.730 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:26:36.773 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:26:36.815 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:26:36.958 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:26:37.001 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:26:37.042 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:26:37.148 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:26:37.192 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:26:37.238 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:26:37.265 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:26:37.294 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:26:37.386 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:26:37.525 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:26:37.568 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:26:37.597 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:26:37.638 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:26:37.687 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:26:37.735 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:26:37.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:26:37.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:26:37.832 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:26:37.875 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:26:37.908 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:26:37.998 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:26:38.048 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:26:38.087 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:26:38.195 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:26:38.248 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:26:38.615 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:38.649 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:39.017 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:26:39.060 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:26:39.090 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:26:39.192 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:26:39.224 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:39.245 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:26:39.285 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:26:39.290 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:26:39.380 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:26:39.380 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:26:39.380 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:26:39.380 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:39.381 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:26:39.411 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:26:39.416 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:26:39.611 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:26:39.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:26:40.081 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:26:40.776 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:26:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:26:42.236 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:26:42.319 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:26:42.348 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:26:42.381 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:26:42.707 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:26:42.752 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:26:42.852 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:26:42.882 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:26:42.918 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:26:43.427 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:26:43.455 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:26:43.526 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:26:43.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:26:45.986 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:46.020 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:46.598 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:26:46.651 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:26:46.743 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:26:46.743 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:26:46.743 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:26:46.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:26:46.785 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:26:47.669 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:26:47.669 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /sys 2024-09-15T22:26:47.669 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /proc 2024-09-15T22:26:47.669 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /mnt 2024-09-15T22:26:47.670 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /var/tmp 2024-09-15T22:26:47.670 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /home 2024-09-15T22:26:47.670 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /root 2024-09-15T22:26:47.670 INFO:teuthology.orchestra.run.smithi130.stdout:skipping the directory /tmp 2024-09-15T22:26:47.670 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:47.849 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:26:47.866 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:26:47.866 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:47.866 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:26:47.866 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:49.344 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:26:49.371 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:26:49.371 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:49.371 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:26:49.371 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:26:49.372 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:26:49.372 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:49.603 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:26:49.626 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:26:49.626 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:49.626 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:26:49.626 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:26:49.627 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:26:49.627 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:50.070 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:26:50.094 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:26:50.094 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:50.094 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:26:50.094 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:26:50.094 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:26:50.094 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:50.200 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:26:50.257 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:26:50.312 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:26:50.497 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:26:50.545 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:26:50.615 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:26:50.703 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:26:50.835 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:26:50.887 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:26:50.919 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:26:50.946 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:26:51.392 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:26:51.420 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:26:51.451 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:26:51.515 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:26:51.544 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-09-15T22:26:51.593 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:51.772 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:26:51.788 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:26:51.788 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:51.788 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:26:51.788 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:52.583 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:26:52.619 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:26:53.316 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:26:53.343 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:26:53.343 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:53.343 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:26:53.343 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:26:53.343 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:26:53.343 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:53.582 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:26:53.584 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:26:53.607 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:26:53.607 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:53.608 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:26:53.608 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:26:53.608 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:26:53.608 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:53.611 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:26:53.715 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:26:53.872 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:26:53.905 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:26:53.929 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:26:53.929 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:53.929 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:26:53.929 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:26:53.929 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:26:53.930 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:53.981 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:26:53.993 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:26:54.067 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:26:54.089 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:26:54.091 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:26:54.091 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:54.091 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:26:54.091 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:26:54.091 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:26:54.091 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:54.146 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:26:54.207 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:26:54.265 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:26:54.328 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:26:54.531 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:26:54.577 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:26:54.658 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:26:54.755 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:26:54.915 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:26:54.961 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:26:54.991 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:26:55.019 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:26:55.069 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:26:55.100 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:26:55.123 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:26:55.123 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:55.124 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:26:55.124 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:26:55.124 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:26:55.124 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:55.167 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:26:55.190 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:26:55.190 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:55.190 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:26:55.190 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:55.243 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:26:55.266 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:26:55.266 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:55.266 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:26:55.267 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:26:55.267 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:26:55.267 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:55.407 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:26:55.434 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:26:55.466 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:26:55.530 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:26:55.557 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:26:55.585 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:26:55.609 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:26:55.609 INFO:teuthology.orchestra.run.smithi130.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:55.609 INFO:teuthology.orchestra.run.smithi130.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:26:55.609 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:26:55.609 INFO:teuthology.orchestra.run.smithi130.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:26:55.609 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:26:56.662 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:26:56.709 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:26:57.723 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:26:57.750 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:26:57.859 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:26:58.028 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:26:58.061 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:26:58.085 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:26:58.085 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:58.085 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:26:58.085 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:26:58.085 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:26:58.085 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:58.137 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:26:58.149 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:26:58.260 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:26:58.312 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:26:59.100 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:26:59.100 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-15T22:26:59.100 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-15T22:26:59.100 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-15T22:26:59.100 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-15T22:26:59.101 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-15T22:26:59.101 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-15T22:26:59.101 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-15T22:26:59.101 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:26:59.287 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:26:59.290 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:26:59.307 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:26:59.307 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:59.307 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:26:59.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:26:59.323 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:26:59.347 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:26:59.347 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:59.347 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:26:59.347 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:26:59.347 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:26:59.347 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:59.398 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:26:59.421 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:26:59.421 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:59.422 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:26:59.422 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:59.482 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:26:59.505 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:26:59.506 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:59.506 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:26:59.506 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:26:59.506 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:26:59.506 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:26:59.825 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:26:59.848 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:26:59.848 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:26:59.848 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:26:59.848 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:26:59.848 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:26:59.848 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:27:00.197 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:27:00.237 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:27:00.279 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:27:00.311 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:27:00.338 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:27:00.521 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:27:00.567 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:27:00.598 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:27:00.627 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:27:00.674 INFO:teuthology.orchestra.run.smithi130.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:27:00.835 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:27:00.864 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:27:00.864 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:00.864 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:27:00.864 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:27:00.864 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:27:00.864 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:01.111 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:27:01.136 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:27:01.136 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:01.136 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:27:01.137 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:27:01.137 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:27:01.137 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:01.247 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:01.287 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:01.308 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:01.595 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:27:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:27:01.620 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:01.620 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:27:01.620 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:27:01.620 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:27:01.620 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:01.726 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:27:01.784 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:27:01.848 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:27:02.050 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:27:02.097 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:27:02.145 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:27:02.194 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:27:02.225 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:27:02.265 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:27:02.284 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:27:02.346 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:27:02.403 INFO:teuthology.orchestra.run.smithi130.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:27:02.427 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:27:02.428 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:27:02.454 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:27:02.454 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:27:02.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:27:02.520 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:27:02.538 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:27:02.538 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:27:02.547 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:27:02.605 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:27:02.630 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:27:02.636 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:02.636 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:02.815 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:02.815 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:27:02.908 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:27:02.971 INFO:teuthology.orchestra.run.smithi130.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:02.979 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:27:03.005 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:27:03.037 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:27:03.101 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:27:03.128 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:27:04.238 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:27:04.271 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:27:04.653 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:27:04.695 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:27:04.737 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:27:04.768 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:27:04.795 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:27:04.997 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:27:05.041 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:27:05.071 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:27:05.101 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:27:05.155 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:27:05.289 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:27:05.314 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:27:05.426 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:27:05.613 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:27:05.641 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:27:05.666 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:27:05.667 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:05.667 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:27:05.667 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:27:05.667 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:27:05.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:05.716 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:27:05.728 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:27:05.754 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:05.794 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:05.815 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:05.830 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:27:05.886 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:27:06.724 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:27:06.807 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:27:06.849 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:27:06.869 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:27:06.903 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:27:06.928 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:27:06.928 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:06.928 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:27:06.928 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:27:06.928 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:27:06.929 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:06.934 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:27:06.970 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:27:06.976 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:27:06.994 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:27:06.994 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:06.994 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:27:06.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:07.002 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:27:07.027 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:27:07.027 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:27:07.046 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:27:07.070 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:27:07.070 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:07.070 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:27:07.070 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:27:07.071 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:27:07.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:07.103 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:27:07.104 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:27:07.171 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:27:07.195 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:27:07.201 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:07.201 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:07.381 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:07.381 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:27:07.388 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:27:07.413 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:27:07.413 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:27:07.414 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:27:07.414 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:27:07.414 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:27:07.414 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:07.465 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:27:07.540 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:08.583 INFO:teuthology.orchestra.run.smithi130.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:08.584 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:27:08.584 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:27:08.584 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:27:08.584 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:27:08.584 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:27:08.584 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:27:08.585 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:27:08.585 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:27:08.585 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:27:08.585 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:27:08.585 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:27:08.585 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:27:08.585 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:27:08.586 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:27:08.587 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:27:08.588 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:27:08.589 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:27:08.590 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:27:08.591 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:27:08.592 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:27:08.593 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:27:08.594 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:27:08.595 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:27:08.596 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:27:08.597 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:27:08.598 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:27:08.598 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:27:08.598 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:27:08.598 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:27:08.598 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout:Upgraded: 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.220 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout:Installed: 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.221 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.222 INFO:teuthology.orchestra.run.smithi130.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:27:11.223 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.224 INFO:teuthology.orchestra.run.smithi130.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:27:11.225 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.226 INFO:teuthology.orchestra.run.smithi130.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:27:11.227 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:27:11.228 INFO:teuthology.orchestra.run.smithi130.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:27:11.229 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:27:11.230 INFO:teuthology.orchestra.run.smithi130.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:27:11.231 INFO:teuthology.orchestra.run.smithi130.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:27:11.232 INFO:teuthology.orchestra.run.smithi130.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:27:11.232 INFO:teuthology.orchestra.run.smithi130.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:27:11.232 INFO:teuthology.orchestra.run.smithi130.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:27:11.232 INFO:teuthology.orchestra.run.smithi130.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:27:11.232 INFO:teuthology.orchestra.run.smithi130.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:27:11.232 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:27:11.232 INFO:teuthology.orchestra.run.smithi130.stdout:Complete! 2024-09-15T22:27:11.641 DEBUG:teuthology.parallel:result is None 2024-09-15T22:27:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:27:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:27:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:27:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:27:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:27:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:27:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:27:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:27:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:27:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:27:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:27:13.010 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:27:13.011 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:27:13.012 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:27:13.013 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:27:13.014 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:27:13.015 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:27:13.016 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:27:13.017 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:27:13.018 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:27:13.019 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:27:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:27:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:27:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:27:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:27:13.020 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:27:13.021 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:27:13.022 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:27:13.023 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:27:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:27:13.024 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:27:13.394 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:13.435 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:13.456 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:27:14.376 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:27:14.465 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:27:14.518 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:27:14.613 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:27:14.643 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:27:14.669 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:27:14.695 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:27:14.695 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:27:14.787 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:27:14.787 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:27:14.863 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:27:14.888 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:27:14.893 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:15.073 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:27:15.073 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:27:15.149 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:27:15.217 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:15.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:15.517 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.519 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:27:15.520 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.521 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:27:15.522 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.523 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:27:15.524 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:27:15.525 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:27:15.525 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:27:15.525 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:27:15.525 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:27:15.525 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:27:15.526 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:27:15.527 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:27:15.528 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:27:15.529 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:27:15.529 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:27:15.529 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:27:15.529 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:27:15.529 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:27:15.530 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:27:15.531 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:27:15.531 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:27:15.531 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-09-15T22:27:15.908 DEBUG:teuthology.parallel:result is None 2024-09-15T22:27:21.062 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:21.062 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:27:21.062 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:27:21.062 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:27:21.063 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:27:21.064 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:27:21.076 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:27:21.076 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:27:21.076 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:27:21.076 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:27:21.076 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:27:21.076 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:27:21.076 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:27:21.077 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:27:21.078 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:27:21.079 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:27:21.080 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:27:21.081 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:27:21.082 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:27:21.083 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:27:21.084 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:27:21.085 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:27:21.086 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:27:23.805 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.806 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.807 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.808 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.809 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:27:23.810 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:27:23.811 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:27:23.812 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:27:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:27:23.814 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:27:23.815 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:27:23.816 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:27:23.817 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.818 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:27:23.819 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.820 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:27:23.821 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:27:23.822 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:27:23.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:27:23.822 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-15T22:27:24.247 DEBUG:teuthology.parallel:result is None 2024-09-15T22:27:24.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:27:24.401 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:27:24.425 INFO:teuthology.orchestra.run.smithi016.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:27:24.426 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:27:24.426 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:27:24.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:27:24.577 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:27:24.603 INFO:teuthology.orchestra.run.smithi059.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:27:24.603 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:27:24.603 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:27:24.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:27:24.757 DEBUG:teuthology.orchestra.run.smithi130:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:27:24.784 INFO:teuthology.orchestra.run.smithi130.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:27:24.785 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:27:24.785 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:27:24.787 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T22:27:24.787 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:27:24.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:27:24.820 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:27:24.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:27:24.854 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:27:24.854 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:27:24.886 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T22:27:24.886 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:27:24.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:27:24.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:27:24.986 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:27:24.986 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:27:25.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:27:25.085 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:27:25.085 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:27:25.115 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:27:25.184 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T22:27:25.185 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:27:25.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:27:25.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:27:25.286 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:27:25.286 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:27:25.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:27:25.384 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:27:25.384 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:27:25.409 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:27:25.474 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T22:27:25.475 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:27:25.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:27:25.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:27:25.579 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:27:25.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:27:25.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:27:25.677 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:27:25.678 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:27:25.702 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:27:25.769 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-15T22:27:25.864 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-15T22:27:25.864 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:27:25.865 INFO:tasks.cephadm:Cluster fsid is af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:27:25.865 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-15T22:27:25.865 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.59', 'mon.c': '172.21.15.130'} 2024-09-15T22:27:25.866 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-09-15T22:27:25.866 INFO:tasks.cephadm:First mgr is x 2024-09-15T22:27:25.866 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-15T22:27:25.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-09-15T22:27:25.897 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2024-09-15T22:27:25.927 DEBUG:teuthology.orchestra.run.smithi130:> sudo hostname $(hostname -s) 2024-09-15T22:27:25.953 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-15T22:27:25.953 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:27:26.101 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-15T22:27:26.240 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:27:26.241 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T22:27:26.241 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T22:27:26.241 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:26.598 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:27 /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:26.598 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:26.969 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:27 /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:26.970 DEBUG:teuthology.orchestra.run.smithi130:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:27.332 INFO:teuthology.orchestra.run.smithi130.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:27 /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:27.333 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:27.353 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:27.373 DEBUG:teuthology.orchestra.run.smithi130:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:27:27.407 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-15T22:27:27.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:27:27.409 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:27:27.415 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:27:29.110 INFO:teuthology.orchestra.run.smithi130.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:27:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:27:29.134 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:27:55.771 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-15T22:27:55.771 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:27:55.771 INFO:teuthology.orchestra.run.smithi130.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:27:55.771 INFO:teuthology.orchestra.run.smithi130.stdout: "repo_digests": [ 2024-09-15T22:27:55.771 INFO:teuthology.orchestra.run.smithi130.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:27:55.771 INFO:teuthology.orchestra.run.smithi130.stdout: ] 2024-09-15T22:27:55.771 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-15T22:27:56.779 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:27:56.780 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:27:56.780 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:27:56.780 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-09-15T22:27:56.780 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:27:56.780 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:27:56.780 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:28:02.912 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-15T22:28:02.912 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:28:02.912 INFO:teuthology.orchestra.run.smithi059.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:28:02.912 INFO:teuthology.orchestra.run.smithi059.stdout: "repo_digests": [ 2024-09-15T22:28:02.912 INFO:teuthology.orchestra.run.smithi059.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:28:02.912 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-09-15T22:28:02.912 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-15T22:28:02.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-09-15T22:28:02.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2024-09-15T22:28:03.001 DEBUG:teuthology.orchestra.run.smithi130:> sudo mkdir -p /etc/ceph 2024-09-15T22:28:03.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-09-15T22:28:03.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2024-09-15T22:28:03.090 DEBUG:teuthology.orchestra.run.smithi130:> sudo chmod 777 /etc/ceph 2024-09-15T22:28:03.119 INFO:tasks.cephadm:Writing seed config... 2024-09-15T22:28:03.120 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-15T22:28:03.120 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-15T22:28:03.120 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-15T22:28:03.120 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-15T22:28:03.120 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-15T22:28:03.120 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-15T22:28:03.121 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-15T22:28:03.122 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-15T22:28:03.123 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-15T22:28:03.124 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:28:03.124 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-15T22:28:03.143 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = af43fc90-73b1-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-15T22:28:03.143 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a.service 2024-09-15T22:28:03.187 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.x.service 2024-09-15T22:28:03.230 INFO:tasks.cephadm:Bootstrapping... 2024-09-15T22:28:03.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid af43fc90-73b1-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:28:03.446 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:28:03.446 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', 'af43fc90-73b1-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-09-15T22:28:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-15T22:28:03.447 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-09-15T22:28:03.477 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-15T22:28:03.477 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-09-15T22:28:03.477 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-09-15T22:28:03.485 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T22:28:03.485 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T22:28:03.492 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T22:28:03.492 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-15T22:28:03.499 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-15T22:28:03.506 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-15T22:28:03.506 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-15T22:28:03.506 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-09-15T22:28:03.529 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-15T22:28:03.529 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-15T22:28:03.529 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-09-15T22:28:03.529 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-09-15T22:28:03.536 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T22:28:03.536 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T22:28:03.543 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T22:28:03.543 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-15T22:28:03.549 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-15T22:28:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-15T22:28:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-15T22:28:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-09-15T22:28:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140411661198336 on /run/cephadm/af43fc90-73b1-11ef-bceb-c7b262605968.lock 2024-09-15T22:28:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140411661198336 acquired on /run/cephadm/af43fc90-73b1-11ef-bceb-c7b262605968.lock 2024-09-15T22:28:03.557 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-09-15T22:28:03.558 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-09-15T22:28:03.558 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-15T22:28:03.561 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-09-15T22:28:03.561 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-09-15T22:28:03.563 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-15T22:28:03.564 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-15T22:28:03.564 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1795sec hoplimit 64 pref medium 2024-09-15T22:28:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-15T22:28:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-15T22:28:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T22:28:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-15T22:28:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-15T22:28:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T22:28:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-15T22:28:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-15T22:28:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-15T22:28:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-15T22:28:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:28:04.014 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T22:28:04.014 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:28:04.014 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-09-15T22:28:04.014 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-15T22:28:04.014 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-15T22:28:04.014 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T22:28:04.014 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-15T22:28:04.969 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T22:28:04.969 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T22:28:04.969 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-09-15T22:28:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-09-15T22:28:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-09-15T22:28:06.982 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB2X+dmypzFERAAHUBKExlSaqZH38T2MWBMGA== 2024-09-15T22:28:15.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQB5X+dm/1ScFxAAF407wwpz+LsSsy+ksk7y+w== 2024-09-15T22:28:19.599 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQCCX+dmWyPZLxAA1jnjC3D08Xa0ZUggRoWsVw== 2024-09-15T22:28:19.600 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-09-15T22:28:20.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:28:20.514 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-09-15T22:28:21.921 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-09-15T22:28:22.141 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-15T22:28:22.279 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-15T22:28:22.446 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-af43fc90-73b1-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-af43fc90-73b1-11ef-bceb-c7b262605968.target. 2024-09-15T22:28:22.447 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-af43fc90-73b1-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-af43fc90-73b1-11ef-bceb-c7b262605968.target. 2024-09-15T22:28:22.677 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a 2024-09-15T22:28:22.678 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-15T22:28:22.842 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af43fc90-73b1-11ef-bceb-c7b262605968.target.wants/ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-af43fc90-73b1-11ef-bceb-c7b262605968@.service. 2024-09-15T22:28:23.277 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 podman[31109]: 2024-09-15 22:28:22.98060363 +0000 UTC m=+0.016469597 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:28:23.278 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 podman[31109]: 2024-09-15 22:28:23.276554835 +0000 UTC m=+0.312420826 container create 1b7eb7e6c6c4ed0ff502a2933c9985d40654cad10c7ec655cfb2f3b0fa1ffeba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:28:23.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 podman[31109]: 2024-09-15 22:28:23.487313189 +0000 UTC m=+0.523179137 container init 1b7eb7e6c6c4ed0ff502a2933c9985d40654cad10c7ec655cfb2f3b0fa1ffeba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-15T22:28:23.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 podman[31109]: 2024-09-15 22:28:23.490568311 +0000 UTC m=+0.526434267 container start 1b7eb7e6c6c4ed0ff502a2933c9985d40654cad10c7ec655cfb2f3b0fa1ffeba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-15T22:28:23.586 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:28:23.586 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-15T22:28:23.593 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:28:23.593 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-15T22:28:23.593 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-15T22:28:23.594 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T22:28:23.594 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-09-15T22:28:23.594 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-09-15T22:28:23.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 ceph-mon[31128]: mkfs af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:23.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 ceph-mon[31128]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T22:28:23.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 bash[31109]: 1b7eb7e6c6c4ed0ff502a2933c9985d40654cad10c7ec655cfb2f3b0fa1ffeba 2024-09-15T22:28:23.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:23 smithi016 systemd[1]: Started Ceph mon.a for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:28:24.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T22:28:24.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: monmap epoch 1 2024-09-15T22:28:24.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:24.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: last_changed 2024-09-15T22:28:19.896069+0000 2024-09-15T22:28:24.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: created 2024-09-15T22:28:19.896069+0000 2024-09-15T22:28:24.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: min_mon_release 19 (squid) 2024-09-15T22:28:24.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: election_strategy: 1 2024-09-15T22:28:24.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-15T22:28:24.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: fsmap 2024-09-15T22:28:24.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: osdmap e1: 0 total, 0 up, 0 in 2024-09-15T22:28:24.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: mgrmap e1: no daemons active 2024-09-15T22:28:24.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:24 smithi016 ceph-mon[31128]: from='client.? 172.21.15.16:0/3881654328' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-15T22:28:25.340 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-09-15T22:28:25.346 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.670721s) 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-09-15T22:28:25.347 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.127+0000 7fa6debbb640 1 Processor -- start 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.127+0000 7fa6debbb640 1 -- start start 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.127+0000 7fa6debbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8106870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.127+0000 7fa6debbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8106e40 con 0x7fa6d8106470 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.128+0000 7fa6dc957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.128+0000 7fa6dc957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8106870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48052/0 (socket says 172.21.15.16:48052) 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.128+0000 7fa6dc957640 1 -- 172.21.15.16:0/4287052739 learned_addr learned my addr 172.21.15.16:0/4287052739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.128+0000 7fa6dc957640 1 -- 172.21.15.16:0/4287052739 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d8107670 con 0x7fa6d8106470 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.130+0000 7fa6dc957640 1 --2- 172.21.15.16:0/4287052739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8106870 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009820 tx=0x7fa6d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98b28c2581921d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.130+0000 7fa6cf7fe640 1 -- 172.21.15.16:0/4287052739 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d4037070 con 0x7fa6d8106470 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.130+0000 7fa6cf7fe640 1 -- 172.21.15.16:0/4287052739 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d4032440 con 0x7fa6d8106470 2024-09-15T22:28:25.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.130+0000 7fa6cf7fe640 1 -- 172.21.15.16:0/4287052739 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d4032740 con 0x7fa6d8106470 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.131+0000 7fa6debbb640 1 -- 172.21.15.16:0/4287052739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 msgr2=0x7fa6d8106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.131+0000 7fa6debbb640 1 --2- 172.21.15.16:0/4287052739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8106870 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009820 tx=0x7fa6d402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.131+0000 7fa6debbb640 1 -- 172.21.15.16:0/4287052739 shutdown_connections 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.131+0000 7fa6debbb640 1 --2- 172.21.15.16:0/4287052739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8106870 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.131+0000 7fa6debbb640 1 -- 172.21.15.16:0/4287052739 >> 172.21.15.16:0/4287052739 conn(0x7fa6d8101c00 msgr2=0x7fa6d8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.131+0000 7fa6debbb640 1 -- 172.21.15.16:0/4287052739 shutdown_connections 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.131+0000 7fa6debbb640 1 -- 172.21.15.16:0/4287052739 wait complete. 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6debbb640 1 Processor -- start 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6debbb640 1 -- start start 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6debbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6debbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8107c10 con 0x7fa6d8106470 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6dc957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:25.349 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6dc957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48060/0 (socket says 172.21.15.16:48060) 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6dc957640 1 -- 172.21.15.16:0/3881654328 learned_addr learned my addr 172.21.15.16:0/3881654328 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6dc957640 1 -- 172.21.15.16:0/3881654328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d819a180 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.132+0000 7fa6dc957640 1 --2- 172.21.15.16:0/3881654328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8199c40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40090b0 tx=0x7fa6d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.133+0000 7fa6cdffb640 1 -- 172.21.15.16:0/3881654328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d4039600 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.133+0000 7fa6cdffb640 1 -- 172.21.15.16:0/3881654328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d40397a0 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.133+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d819a410 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.133+0000 7fa6cdffb640 1 -- 172.21.15.16:0/3881654328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6d4039ac0 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.133+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d8074930 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.133+0000 7fa6cdffb640 1 -- 172.21.15.16:0/3881654328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa6d404a060 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.133+0000 7fa6cdffb640 1 -- 172.21.15.16:0/3881654328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa6d4043730 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.134+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6d819ab80 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.136+0000 7fa6cdffb640 1 -- 172.21.15.16:0/3881654328 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa6d819ab80 con 0x7fa6d8106470 2024-09-15T22:28:25.350 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.197+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa6d8074f80 con 0x7fa6d8106470 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6cdffb640 1 -- 172.21.15.16:0/3881654328 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa6d403e070 con 0x7fa6d8106470 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 msgr2=0x7fa6d8199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6debbb640 1 --2- 172.21.15.16:0/3881654328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8199c40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40090b0 tx=0x7fa6d40046c0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 shutdown_connections 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6debbb640 1 --2- 172.21.15.16:0/3881654328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8106470 0x7fa6d8199c40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 >> 172.21.15.16:0/3881654328 conn(0x7fa6d8101c00 msgr2=0x7fa6d818df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 shutdown_connections 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:24.198+0000 7fa6debbb640 1 -- 172.21.15.16:0/3881654328 wait complete. 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-09-15T22:28:25.351 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-09-15T22:28:26.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:25 smithi016 ceph-mon[31128]: from='client.? 172.21.15.16:0/4000751164' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T22:28:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:25 smithi016 ceph-mon[31128]: from='client.? 172.21.15.16:0/4000751164' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-15T22:28:26.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:26.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T22:28:26.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:26.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-15T22:28:26.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T22:28:26.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T22:28:26.416 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.755+0000 7fcfc42a9640 1 Processor -- start 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.756+0000 7fcfc42a9640 1 -- start start 2024-09-15T22:28:26.417 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.756+0000 7fcfc42a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc10a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.756+0000 7fcfc42a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc07be10 con 0x7fcfbc107fe0 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.756+0000 7fcfc2045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc10a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.756+0000 7fcfc2045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc10a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48066/0 (socket says 172.21.15.16:48066) 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.756+0000 7fcfc2045640 1 -- 172.21.15.16:0/1230412757 learned_addr learned my addr 172.21.15.16:0/1230412757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.756+0000 7fcfc2045640 1 -- 172.21.15.16:0/1230412757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc10a910 con 0x7fcfbc107fe0 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.757+0000 7fcfc2045640 1 --2- 172.21.15.16:0/1230412757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc10a3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009820 tx=0x7fcfb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6862d24cc114bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.757+0000 7fcfc1043640 1 -- 172.21.15.16:0/1230412757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfb8037070 con 0x7fcfbc107fe0 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.757+0000 7fcfc1043640 1 -- 172.21.15.16:0/1230412757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb8032440 con 0x7fcfbc107fe0 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.757+0000 7fcfc1043640 1 -- 172.21.15.16:0/1230412757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfb8032740 con 0x7fcfbc107fe0 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.758+0000 7fcfc42a9640 1 -- 172.21.15.16:0/1230412757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 msgr2=0x7fcfbc10a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.758+0000 7fcfc42a9640 1 --2- 172.21.15.16:0/1230412757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc10a3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009820 tx=0x7fcfb802a270 comp rx=0 tx=0).stop 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.758+0000 7fcfc42a9640 1 -- 172.21.15.16:0/1230412757 shutdown_connections 2024-09-15T22:28:26.418 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.758+0000 7fcfc42a9640 1 --2- 172.21.15.16:0/1230412757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc10a3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.758+0000 7fcfc42a9640 1 -- 172.21.15.16:0/1230412757 >> 172.21.15.16:0/1230412757 conn(0x7fcfbc101bc0 msgr2=0x7fcfbc104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.758+0000 7fcfc42a9640 1 -- 172.21.15.16:0/1230412757 shutdown_connections 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.758+0000 7fcfc42a9640 1 -- 172.21.15.16:0/1230412757 wait complete. 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc42a9640 1 Processor -- start 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc42a9640 1 -- start start 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc42a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc42a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc10adf0 con 0x7fcfbc107fe0 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc2045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc2045640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc19ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48080/0 (socket says 172.21.15.16:48080) 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc2045640 1 -- 172.21.15.16:0/4000751164 learned_addr learned my addr 172.21.15.16:0/4000751164 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.759+0000 7fcfc2045640 1 -- 172.21.15.16:0/4000751164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc19b220 con 0x7fcfbc107fe0 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfc2045640 1 --2- 172.21.15.16:0/4000751164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc19ace0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009820 tx=0x7fcfb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:26.419 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfb8041070 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb803c7a0 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc19b4b0 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfb8037040 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc19e190 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcfb8044070 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.760+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcfb8046dd0 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.761+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc108440 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.763+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fcfb8046450 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.823+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fcfbc02d980 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.826+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fcfb8046630 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfb37fe640 1 -- 172.21.15.16:0/4000751164 <== mon.0 v2:172.21.15.16:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcfb8055020 con 0x7fcfbc107fe0 2024-09-15T22:28:26.420 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 msgr2=0x7fcfbc19ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:26.421 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfc42a9640 1 --2- 172.21.15.16:0/4000751164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc19ace0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009820 tx=0x7fcfb80046c0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.421 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 shutdown_connections 2024-09-15T22:28:26.421 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfc42a9640 1 --2- 172.21.15.16:0/4000751164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfbc107fe0 0x7fcfbc19ace0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:26.421 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 >> 172.21.15.16:0/4000751164 conn(0x7fcfbc101bc0 msgr2=0x7fcfbc102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:26.421 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 shutdown_connections 2024-09-15T22:28:26.421 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:25.827+0000 7fcfc42a9640 1 -- 172.21.15.16:0/4000751164 wait complete. 2024-09-15T22:28:26.421 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-09-15T22:28:27.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:26 smithi016 ceph-mon[31128]: from='client.? 172.21.15.16:0/213960790' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:28:27.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.827+0000 7efd4415d640 1 Processor -- start 2024-09-15T22:28:27.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.827+0000 7efd4415d640 1 -- start start 2024-09-15T22:28:27.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.827+0000 7efd4415d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:27.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.827+0000 7efd4415d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c107f00 con 0x7efd3c107530 2024-09-15T22:28:27.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.828+0000 7efd41ef9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.828+0000 7efd41ef9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48086/0 (socket says 172.21.15.16:48086) 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.828+0000 7efd41ef9640 1 -- 172.21.15.16:0/2813448965 learned_addr learned my addr 172.21.15.16:0/2813448965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.828+0000 7efd41ef9640 1 -- 172.21.15.16:0/2813448965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c108730 con 0x7efd3c107530 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.829+0000 7efd41ef9640 1 --2- 172.21.15.16:0/2813448965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c107930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efd34009820 tx=0x7efd3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b3d11c91fdbc68 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.829+0000 7efd40ef7640 1 -- 172.21.15.16:0/2813448965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd34037070 con 0x7efd3c107530 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.829+0000 7efd40ef7640 1 -- 172.21.15.16:0/2813448965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd34032900 con 0x7efd3c107530 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.829+0000 7efd40ef7640 1 -- 172.21.15.16:0/2813448965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd34030850 con 0x7efd3c107530 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.829+0000 7efd4415d640 1 -- 172.21.15.16:0/2813448965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 msgr2=0x7efd3c107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.829+0000 7efd4415d640 1 --2- 172.21.15.16:0/2813448965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c107930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efd34009820 tx=0x7efd3402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.829+0000 7efd4415d640 1 -- 172.21.15.16:0/2813448965 shutdown_connections 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 --2- 172.21.15.16:0/2813448965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c107930 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 -- 172.21.15.16:0/2813448965 >> 172.21.15.16:0/2813448965 conn(0x7efd3c102cc0 msgr2=0x7efd3c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:27.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 -- 172.21.15.16:0/2813448965 shutdown_connections 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 -- 172.21.15.16:0/2813448965 wait complete. 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 Processor -- start 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 -- start start 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.830+0000 7efd4415d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c10bae0 con 0x7efd3c107530 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd41ef9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd41ef9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c19b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48098/0 (socket says 172.21.15.16:48098) 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd41ef9640 1 -- 172.21.15.16:0/213960790 learned_addr learned my addr 172.21.15.16:0/213960790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd41ef9640 1 -- 172.21.15.16:0/213960790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c19bac0 con 0x7efd3c107530 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd41ef9640 1 --2- 172.21.15.16:0/213960790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c19b580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd340097f0 tx=0x7efd34030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd3affd640 1 -- 172.21.15.16:0/213960790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd34037040 con 0x7efd3c107530 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd3affd640 1 -- 172.21.15.16:0/213960790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd34040070 con 0x7efd3c107530 2024-09-15T22:28:27.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c19bd50 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd3affd640 1 -- 172.21.15.16:0/213960790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd3403a390 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c19ea40 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.831+0000 7efd3affd640 1 -- 172.21.15.16:0/213960790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd3403a5e0 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.832+0000 7efd3affd640 1 -- 172.21.15.16:0/213960790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd34045950 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.832+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c10bc60 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.835+0000 7efd3affd640 1 -- 172.21.15.16:0/213960790 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efd34045460 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.895+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efd3c19ec10 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.895+0000 7efd3affd640 1 -- 172.21.15.16:0/213960790 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7efd3403b430 con 0x7efd3c107530 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.896+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 msgr2=0x7efd3c19b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.896+0000 7efd4415d640 1 --2- 172.21.15.16:0/213960790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c19b580 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efd340097f0 tx=0x7efd34030d80 comp rx=0 tx=0).stop 2024-09-15T22:28:27.490 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.896+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 shutdown_connections 2024-09-15T22:28:27.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.896+0000 7efd4415d640 1 --2- 172.21.15.16:0/213960790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd3c107530 0x7efd3c19b580 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:27.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.896+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 >> 172.21.15.16:0/213960790 conn(0x7efd3c102cc0 msgr2=0x7efd3c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:27.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.896+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 shutdown_connections 2024-09-15T22:28:27.491 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:26.896+0000 7efd4415d640 1 -- 172.21.15.16:0/213960790 wait complete. 2024-09-15T22:28:27.491 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-09-15T22:28:27.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:27 smithi016 systemd[1]: Stopping Ceph mon.a for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:28:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a[31124]: 2024-09-15T22:28:27.773+0000 7fe379fa4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a[31124]: 2024-09-15T22:28:27.773+0000 7fe379fa4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-15T22:28:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:27 smithi016 podman[31370]: 2024-09-15 22:28:27.907666944 +0000 UTC m=+0.255138728 container died 1b7eb7e6c6c4ed0ff502a2933c9985d40654cad10c7ec655cfb2f3b0fa1ffeba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-15T22:28:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 podman[31370]: 2024-09-15 22:28:28.232052515 +0000 UTC m=+0.579524281 container cleanup 1b7eb7e6c6c4ed0ff502a2933c9985d40654cad10c7ec655cfb2f3b0fa1ffeba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-15T22:28:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 bash[31370]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a 2024-09-15T22:28:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 podman[31383]: 2024-09-15 22:28:28.408121527 +0000 UTC m=+0.501533377 container remove 1b7eb7e6c6c4ed0ff502a2933c9985d40654cad10c7ec655cfb2f3b0fa1ffeba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-15T22:28:28.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-15T22:28:28.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 systemd[1]: Stopped Ceph mon.a for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:28:28.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 systemd[1]: Starting Ceph mon.a for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:28:28.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 podman[31473]: 2024-09-15 22:28:28.783475632 +0000 UTC m=+0.016465225 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:28:28.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:28 smithi016 podman[31473]: 2024-09-15 22:28:28.883986681 +0000 UTC m=+0.116976329 container create 7ea2261f5ae741770b0f78053f0a334e6f8726ac64b9ffa27081cc7d64a28aaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-15T22:28:29.133 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-15T22:28:29.350 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 podman[31473]: 2024-09-15 22:28:29.069743903 +0000 UTC m=+0.302733507 container init 7ea2261f5ae741770b0f78053f0a334e6f8726ac64b9ffa27081cc7d64a28aaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 podman[31473]: 2024-09-15 22:28:29.072893867 +0000 UTC m=+0.305883452 container start 7ea2261f5ae741770b0f78053f0a334e6f8726ac64b9ffa27081cc7d64a28aaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True) 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: pidfile_write: ignore empty --pid-file 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: load: jerasure load: lrc 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: RocksDB version: 7.9.2 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Git sha 0 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: DB SUMMARY 2024-09-15T22:28:29.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: DB Session ID: 1Y4ABV3D0MWBIDSAQEGB 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: CURRENT file: CURRENT 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104676 ; 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.error_if_exists: 0 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.create_if_missing: 0 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.paranoid_checks: 1 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T22:28:29.352 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.env: 0x561f1972d940 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.info_log: 0x561f1b5016e0 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.statistics: (nil) 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.use_fsync: 0 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_log_file_size: 0 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T22:28:29.353 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.allow_fallocate: 1 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.use_direct_reads: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.db_log_dir: 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.wal_dir: 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T22:28:29.354 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.write_buffer_manager: 0x561f1b510dc0 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.unordered_write: 0 2024-09-15T22:28:29.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T22:28:29.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T22:28:29.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T22:28:29.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T22:28:29.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.row_cache: None 2024-09-15T22:28:29.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.wal_filter: None 2024-09-15T22:28:29.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T22:28:29.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T22:28:29.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.two_write_queues: 0 2024-09-15T22:28:29.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T22:28:29.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.wal_compression: 0 2024-09-15T22:28:29.357 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.atomic_flush: 0 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.log_readahead_size: 0 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T22:28:29.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_background_jobs: 2 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_background_compactions: -1 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_subcompactions: 1 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_open_files: -1 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T22:28:29.359 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_background_flushes: -1 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Compression algorithms supported: 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kZSTD supported: 0 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kXpressCompression supported: 0 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kBZip2Compression supported: 0 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kLZ4Compression supported: 1 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kZlibCompression supported: 1 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: kSnappyCompression supported: 1 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T22:28:29.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.merge_operator: 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_filter: None 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561f1b501320) 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-09-15T22:28:29.361 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x561f1b4f51f0 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-09-15T22:28:29.362 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-09-15T22:28:29.363 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T22:28:29.364 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression: NoCompression 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.num_levels: 7 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:28:29.365 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T22:28:29.366 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T22:28:29.367 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T22:28:29.368 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.inplace_update_support: 0 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T22:28:29.369 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.bloom_locality: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.max_successive_merges: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.ttl: 2592000 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T22:28:29.370 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.enable_blob_files: false 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.min_blob_size: 0 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-15T22:28:29.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0fde73aa-e8ab-457d-9b02-5afad43222d3 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439309099895, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439309100909, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99831, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97713, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14159, "raw_average_key_size": 49, "raw_value_size": 90022, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726439309, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0fde73aa-e8ab-457d-9b02-5afad43222d3", "db_session_id": "1Y4ABV3D0MWBIDSAQEGB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439309101026, "job": 1, "event": "recovery_finished"} 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561f1b522e00 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: DB pointer 0x561f1b61e000 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-09-15T22:28:29.372 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 99.35 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 99.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:28:29.373 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T22:28:29.374 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 29.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 29.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x561f1b4f51f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:29.375 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???) e1 preinit fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???).mds e1 new map 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???).mds e1 print_map 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout: btime 2024-09-15T22:28:23:528693+0000 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-09-15T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 bash[31473]: 7ea2261f5ae741770b0f78053f0a334e6f8726ac64b9ffa27081cc7d64a28aaf 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 systemd[1]: Started Ceph mon.a for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: monmap epoch 1 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: last_changed 2024-09-15T22:28:19.896069+0000 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: created 2024-09-15T22:28:19.896069+0000 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: min_mon_release 19 (squid) 2024-09-15T22:28:29.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: election_strategy: 1 2024-09-15T22:28:29.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-15T22:28:29.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: fsmap 2024-09-15T22:28:29.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: osdmap e1: 0 total, 0 up, 0 in 2024-09-15T22:28:29.378 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:29 smithi016 ceph-mon[31492]: mgrmap e1: no daemons active 2024-09-15T22:28:30.505 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.589+0000 7f1435d96640 1 Processor -- start 2024-09-15T22:28:30.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.589+0000 7f1435d96640 1 -- start start 2024-09-15T22:28:30.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.589+0000 7f1435d96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f1430107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:30.511 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.589+0000 7f1435d96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1430107f00 con 0x7f1430107530 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.590+0000 7f142f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f1430107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.590+0000 7f142f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f1430107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48106/0 (socket says 172.21.15.16:48106) 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.590+0000 7f142f7fe640 1 -- 172.21.15.16:0/2090169262 learned_addr learned my addr 172.21.15.16:0/2090169262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.590+0000 7f142f7fe640 1 -- 172.21.15.16:0/2090169262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1430108730 con 0x7f1430107530 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.592+0000 7f142f7fe640 1 --2- 172.21.15.16:0/2090169262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f1430107930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1420009e80 tx=0x7f142002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2faf9432a1d21bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.592+0000 7f142e7fc640 1 -- 172.21.15.16:0/2090169262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420036070 con 0x7f1430107530 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.592+0000 7f142e7fc640 1 -- 172.21.15.16:0/2090169262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14200318c0 con 0x7f1430107530 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.592+0000 7f142e7fc640 1 -- 172.21.15.16:0/2090169262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420007ab0 con 0x7f1430107530 2024-09-15T22:28:30.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 -- 172.21.15.16:0/2090169262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 msgr2=0x7f1430107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 --2- 172.21.15.16:0/2090169262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f1430107930 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1420009e80 tx=0x7f142002a7a0 comp rx=0 tx=0).stop 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 -- 172.21.15.16:0/2090169262 shutdown_connections 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 --2- 172.21.15.16:0/2090169262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f1430107930 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 -- 172.21.15.16:0/2090169262 >> 172.21.15.16:0/2090169262 conn(0x7f1430102cc0 msgr2=0x7f1430105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 -- 172.21.15.16:0/2090169262 shutdown_connections 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 -- 172.21.15.16:0/2090169262 wait complete. 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.593+0000 7f1435d96640 1 Processor -- start 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f1435d96640 1 -- start start 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f1435d96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f143019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f1435d96640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f143010bae0 con 0x7f1430107530 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f142f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f143019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:30.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f142f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f143019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48122/0 (socket says 172.21.15.16:48122) 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f142f7fe640 1 -- 172.21.15.16:0/835046561 learned_addr learned my addr 172.21.15.16:0/835046561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f142f7fe640 1 -- 172.21.15.16:0/835046561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143019ba90 con 0x7f1430107530 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.594+0000 7f142f7fe640 1 --2- 172.21.15.16:0/835046561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f143019b550 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f142002ad90 tx=0x7f1420031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.595+0000 7f142cff9640 1 -- 172.21.15.16:0/835046561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420036050 con 0x7f1430107530 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.595+0000 7f142cff9640 1 -- 172.21.15.16:0/835046561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1420042070 con 0x7f1430107530 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.595+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143019bd20 con 0x7f1430107530 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.595+0000 7f142cff9640 1 -- 172.21.15.16:0/835046561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f142003d7e0 con 0x7f1430107530 2024-09-15T22:28:30.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.595+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143019ea10 con 0x7f1430107530 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.595+0000 7f142cff9640 1 -- 172.21.15.16:0/835046561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f142003dae0 con 0x7f1430107530 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.595+0000 7f142cff9640 1 -- 172.21.15.16:0/835046561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1420007e00 con 0x7f1430107530 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.596+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143010bc60 con 0x7f1430107530 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.598+0000 7f142cff9640 1 -- 172.21.15.16:0/835046561 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f14200072f0 con 0x7f1430107530 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.658+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f14301079b0 con 0x7f1430107530 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.660+0000 7f142cff9640 1 -- 172.21.15.16:0/835046561 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f1420047460 con 0x7f1430107530 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.660+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 msgr2=0x7f143019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.660+0000 7f1435d96640 1 --2- 172.21.15.16:0/835046561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f143019b550 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f142002ad90 tx=0x7f1420031ca0 comp rx=0 tx=0).stop 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.661+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 shutdown_connections 2024-09-15T22:28:30.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.661+0000 7f1435d96640 1 --2- 172.21.15.16:0/835046561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430107530 0x7f143019b550 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:30.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.661+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 >> 172.21.15.16:0/835046561 conn(0x7f1430102cc0 msgr2=0x7f14301038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:30.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.661+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 shutdown_connections 2024-09-15T22:28:30.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:29.661+0000 7f1435d96640 1 -- 172.21.15.16:0/835046561 wait complete. 2024-09-15T22:28:30.516 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-15T22:28:30.535 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:28:30.535 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-09-15T22:28:30.536 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-15T22:28:30.536 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-15T22:28:30.793 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.x 2024-09-15T22:28:30.793 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-15T22:28:30.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:30 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/835046561' entity='client.admin' 2024-09-15T22:28:30.943 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-af43fc90-73b1-11ef-bceb-c7b262605968.target.wants/ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-af43fc90-73b1-11ef-bceb-c7b262605968@.service. 2024-09-15T22:28:31.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:30 smithi016 systemd[1]: Starting Ceph mgr.x for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:28:31.465 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 podman[31702]: 2024-09-15 22:28:31.079781133 +0000 UTC m=+0.016299596 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:28:31.465 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 podman[31702]: 2024-09-15 22:28:31.186864902 +0000 UTC m=+0.123383366 container create c77097c54c1fc68eab0a55343f83d3fc8f8652ccc242fafb54f2b4796cce87a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e) 2024-09-15T22:28:31.465 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 podman[31702]: 2024-09-15 22:28:31.389236725 +0000 UTC m=+0.325755191 container init c77097c54c1fc68eab0a55343f83d3fc8f8652ccc242fafb54f2b4796cce87a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-15T22:28:31.465 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 podman[31702]: 2024-09-15 22:28:31.391976408 +0000 UTC m=+0.328494879 container start c77097c54c1fc68eab0a55343f83d3fc8f8652ccc242fafb54f2b4796cce87a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-15T22:28:31.465 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 bash[31702]: c77097c54c1fc68eab0a55343f83d3fc8f8652ccc242fafb54f2b4796cce87a1 2024-09-15T22:28:31.480 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:28:31.481 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-15T22:28:31.489 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:28:31.489 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-15T22:28:31.489 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-15T22:28:31.489 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T22:28:31.498 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:28:31.498 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-15T22:28:31.506 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:28:31.506 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-15T22:28:31.506 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-15T22:28:31.506 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-15T22:28:31.506 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-09-15T22:28:31.506 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-09-15T22:28:31.740 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 systemd[1]: Started Ceph mgr.x for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:28:31.741 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:31.521+0000 7fc56546d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:28:31.741 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:31.581+0000 7fc56546d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:28:31.741 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:31.739+0000 7fc56546d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:28:32.010 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:31.808+0000 7fc56546d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:28:32.010 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:31 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:31.966+0000 7fc56546d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:28:32.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:32 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3138873028' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:28:32.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:32 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:32.116+0000 7fc56546d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:28:32.551 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:32.551 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:28:32.551 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:28:32.551 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-15T22:28:32.552 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:28:32.553 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:28:32.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:28:32.560 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:28:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:28:23:528693+0000", 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:28:32.562 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:28:23.529340+0000", 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.938+0000 7f7e7051a640 1 Processor -- start 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.938+0000 7f7e7051a640 1 -- start start 2024-09-15T22:28:32.563 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.938+0000 7f7e7051a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6810a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.938+0000 7f7e7051a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6807bdf0 con 0x7f7e68108000 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.939+0000 7f7e6e2b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6810a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.939+0000 7f7e6e2b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6810a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46420/0 (socket says 172.21.15.16:46420) 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.939+0000 7f7e6e2b6640 1 -- 172.21.15.16:0/841150179 learned_addr learned my addr 172.21.15.16:0/841150179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.939+0000 7f7e6e2b6640 1 -- 172.21.15.16:0/841150179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6810a930 con 0x7f7e68108000 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.940+0000 7f7e6e2b6640 1 --2- 172.21.15.16:0/841150179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6810a3f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009820 tx=0x7f7e6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416c4fc5562227a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.940+0000 7f7e6d2b4640 1 -- 172.21.15.16:0/841150179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64037070 con 0x7f7e68108000 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.940+0000 7f7e6d2b4640 1 -- 172.21.15.16:0/841150179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e64032900 con 0x7f7e68108000 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.940+0000 7f7e6d2b4640 1 -- 172.21.15.16:0/841150179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64030850 con 0x7f7e68108000 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 -- 172.21.15.16:0/841150179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 msgr2=0x7f7e6810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 --2- 172.21.15.16:0/841150179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6810a3f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009820 tx=0x7f7e6402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 -- 172.21.15.16:0/841150179 shutdown_connections 2024-09-15T22:28:32.564 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 --2- 172.21.15.16:0/841150179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6810a3f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 -- 172.21.15.16:0/841150179 >> 172.21.15.16:0/841150179 conn(0x7f7e68101bc0 msgr2=0x7f7e68104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 -- 172.21.15.16:0/841150179 shutdown_connections 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 -- 172.21.15.16:0/841150179 wait complete. 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 Processor -- start 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.941+0000 7f7e7051a640 1 -- start start 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e7051a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e7051a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6810b970 con 0x7f7e68108000 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e6e2b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e6e2b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46428/0 (socket says 172.21.15.16:46428) 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e6e2b6640 1 -- 172.21.15.16:0/3138873028 learned_addr learned my addr 172.21.15.16:0/3138873028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e6e2b6640 1 -- 172.21.15.16:0/3138873028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6819bb30 con 0x7f7e68108000 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e6e2b6640 1 --2- 172.21.15.16:0/3138873028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6819b5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7e640097f0 tx=0x7f7e64030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e5f7fe640 1 -- 172.21.15.16:0/3138873028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64042070 con 0x7f7e68108000 2024-09-15T22:28:32.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.942+0000 7f7e5f7fe640 1 -- 172.21.15.16:0/3138873028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e64032ca0 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.943+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6819bdc0 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.943+0000 7f7e5f7fe640 1 -- 172.21.15.16:0/3138873028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64037040 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.943+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6819eab0 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.943+0000 7f7e5f7fe640 1 -- 172.21.15.16:0/3138873028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7e64045070 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.943+0000 7f7e5f7fe640 1 -- 172.21.15.16:0/3138873028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7e64047780 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.944+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e68108460 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:31.946+0000 7f7e5f7fe640 1 -- 172.21.15.16:0/3138873028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7e64046d40 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.008+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7e6819ec80 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.009+0000 7f7e5f7fe640 1 -- 172.21.15.16:0/3138873028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f7e6403db10 con 0x7f7e68108000 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.010+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 msgr2=0x7f7e6819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.010+0000 7f7e7051a640 1 --2- 172.21.15.16:0/3138873028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6819b5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7e640097f0 tx=0x7f7e64030ec0 comp rx=0 tx=0).stop 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.010+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 shutdown_connections 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.010+0000 7f7e7051a640 1 --2- 172.21.15.16:0/3138873028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e68108000 0x7f7e6819b5f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:32.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.010+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 >> 172.21.15.16:0/3138873028 conn(0x7f7e68101bc0 msgr2=0x7f7e681027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:32.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.010+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 shutdown_connections 2024-09-15T22:28:32.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:32.010+0000 7f7e7051a640 1 -- 172.21.15.16:0/3138873028 wait complete. 2024-09-15T22:28:32.567 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-09-15T22:28:32.711 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:32 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:32.452+0000 7fc56546d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:28:32.711 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:32 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:32.515+0000 7fc56546d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:28:33.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:32 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:32.710+0000 7fc56546d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:28:34.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:33 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:33.631+0000 7fc56546d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:28:34.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:33 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:33.701+0000 7fc56546d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:28:34.728 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:34 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:34.453+0000 7fc56546d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:28:34.728 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:34 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:34.575+0000 7fc56546d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:28:34.728 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:34 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:34.726+0000 7fc56546d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:28:34.982 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:34 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:34.792+0000 7fc56546d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:28:34.983 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:34 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:34.950+0000 7fc56546d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:28:35.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2694827780' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:28:35.266 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.024+0000 7fc56546d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:28:35.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.090+0000 7fc56546d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:28:35.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.265+0000 7fc56546d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:28:35.556 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.334+0000 7fc56546d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:28:35.556 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.403+0000 7fc56546d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:28:35.556 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.491+0000 7fc56546d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:28:35.556 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.554+0000 7fc56546d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:28:35.725 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:28:35.726 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:28:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:28:35.728 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:28:23:528693+0000", 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:28:35.729 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:28:23.529340+0000", 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.985+0000 7f3d6aa0a640 1 Processor -- start 2024-09-15T22:28:35.730 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.986+0000 7f3d6aa0a640 1 -- start start 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.986+0000 7f3d6aa0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d6410a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.986+0000 7f3d6aa0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6407be10 con 0x7f3d64107fe0 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.986+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d6410a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.986+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d6410a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46432/0 (socket says 172.21.15.16:46432) 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.986+0000 7f3d63fff640 1 -- 172.21.15.16:0/459962245 learned_addr learned my addr 172.21.15.16:0/459962245 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.986+0000 7f3d63fff640 1 -- 172.21.15.16:0/459962245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6410a910 con 0x7f3d64107fe0 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.987+0000 7f3d63fff640 1 --2- 172.21.15.16:0/459962245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d6410a3d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009820 tx=0x7f3d5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe2cde286ed7ea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.987+0000 7f3d637fe640 1 -- 172.21.15.16:0/459962245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d54037070 con 0x7f3d64107fe0 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.987+0000 7f3d637fe640 1 -- 172.21.15.16:0/459962245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3d54032900 con 0x7f3d64107fe0 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.987+0000 7f3d637fe640 1 -- 172.21.15.16:0/459962245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d54030850 con 0x7f3d64107fe0 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.988+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/459962245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 msgr2=0x7f3d6410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.988+0000 7f3d6aa0a640 1 --2- 172.21.15.16:0/459962245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d6410a3d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009820 tx=0x7f3d5402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:35.731 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.988+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/459962245 shutdown_connections 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.988+0000 7f3d6aa0a640 1 --2- 172.21.15.16:0/459962245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d6410a3d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.988+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/459962245 >> 172.21.15.16:0/459962245 conn(0x7f3d64101bc0 msgr2=0x7f3d64104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.988+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/459962245 shutdown_connections 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.988+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/459962245 wait complete. 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d6aa0a640 1 Processor -- start 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d6aa0a640 1 -- start start 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d6aa0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d64107860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d6aa0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6410b950 con 0x7f3d64107fe0 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d64107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d64107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46436/0 (socket says 172.21.15.16:46436) 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d63fff640 1 -- 172.21.15.16:0/2694827780 learned_addr learned my addr 172.21.15.16:0/2694827780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d63fff640 1 -- 172.21.15.16:0/2694827780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d64107da0 con 0x7f3d64107fe0 2024-09-15T22:28:35.732 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.989+0000 7f3d63fff640 1 --2- 172.21.15.16:0/2694827780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d64107860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d54031000 tx=0x7f3d540326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.990+0000 7f3d61ffb640 1 -- 172.21.15.16:0/2694827780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d54042070 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.990+0000 7f3d61ffb640 1 -- 172.21.15.16:0/2694827780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3d54032be0 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.990+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d64105f90 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.990+0000 7f3d61ffb640 1 -- 172.21.15.16:0/2694827780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d54037040 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.990+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d641064d0 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.990+0000 7f3d61ffb640 1 -- 172.21.15.16:0/2694827780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d54045070 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.990+0000 7f3d61ffb640 1 -- 172.21.15.16:0/2694827780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d54047ad0 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.991+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d64108440 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:34.993+0000 7f3d61ffb640 1 -- 172.21.15.16:0/2694827780 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3d5404c030 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.055+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3d64106c00 con 0x7f3d64107fe0 2024-09-15T22:28:35.733 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.055+0000 7f3d61ffb640 1 -- 172.21.15.16:0/2694827780 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f3d540328a0 con 0x7f3d64107fe0 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.056+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 msgr2=0x7f3d64107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.056+0000 7f3d6aa0a640 1 --2- 172.21.15.16:0/2694827780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d64107860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3d54031000 tx=0x7f3d540326e0 comp rx=0 tx=0).stop 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.056+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 shutdown_connections 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.056+0000 7f3d6aa0a640 1 --2- 172.21.15.16:0/2694827780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d64107fe0 0x7f3d64107860 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.056+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 >> 172.21.15.16:0/2694827780 conn(0x7f3d64101bc0 msgr2=0x7f3d64102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.056+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 shutdown_connections 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:35.056+0000 7f3d6aa0a640 1 -- 172.21.15.16:0/2694827780 wait complete. 2024-09-15T22:28:35.734 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-09-15T22:28:35.806 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:35.617+0000 7fc56546d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:28:36.807 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:36.444+0000 7fc56546d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:28:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: Activating manager daemon x 2024-09-15T22:28:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: mgrmap e2: x(active, starting, since 0.00341997s) 2024-09-15T22:28:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:28:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: Manager daemon x is now available 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' 2024-09-15T22:28:36.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:36 smithi016 ceph-mon[31492]: from='mgr.14100 172.21.15.16:0/970856460' entity='mgr.x' 2024-09-15T22:28:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:38 smithi016 ceph-mon[31492]: mgrmap e3: x(active, since 1.00634s) 2024-09-15T22:28:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:38 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3094830737' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:28:38.921 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:38.932 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:28:38.933 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:28:38.934 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:28:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:28:23:528693+0000", 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:28:38.936 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:28:38.937 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:28:23.529340+0000", 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.126+0000 7f7d9a2a1640 1 Processor -- start 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.126+0000 7f7d9a2a1640 1 -- start start 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.126+0000 7f7d9a2a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d94106870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.126+0000 7f7d9a2a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d94106e40 con 0x7f7d94106470 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.126+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d94106870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:38.938 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.126+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d94106870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46500/0 (socket says 172.21.15.16:46500) 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.127+0000 7f7d93fff640 1 -- 172.21.15.16:0/89093540 learned_addr learned my addr 172.21.15.16:0/89093540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.127+0000 7f7d93fff640 1 -- 172.21.15.16:0/89093540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d94107670 con 0x7f7d94106470 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.127+0000 7f7d93fff640 1 --2- 172.21.15.16:0/89093540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d94106870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009820 tx=0x7f7d8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd988b6f1d75d086 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d92ffd640 1 -- 172.21.15.16:0/89093540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d84037070 con 0x7f7d94106470 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d92ffd640 1 -- 172.21.15.16:0/89093540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d84032900 con 0x7f7d94106470 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d92ffd640 1 -- 172.21.15.16:0/89093540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d84030850 con 0x7f7d94106470 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/89093540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 msgr2=0x7f7d94106870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d9a2a1640 1 --2- 172.21.15.16:0/89093540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d94106870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009820 tx=0x7f7d8402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/89093540 shutdown_connections 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d9a2a1640 1 --2- 172.21.15.16:0/89093540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d94106870 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/89093540 >> 172.21.15.16:0/89093540 conn(0x7f7d94101c00 msgr2=0x7f7d94104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/89093540 shutdown_connections 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.128+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/89093540 wait complete. 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.129+0000 7f7d9a2a1640 1 Processor -- start 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.129+0000 7f7d9a2a1640 1 -- start start 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.129+0000 7f7d9a2a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d9419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.129+0000 7f7d9a2a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410aa20 con 0x7f7d94106470 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d9419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d9419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46512/0 (socket says 172.21.15.16:46512) 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d93fff640 1 -- 172.21.15.16:0/3094830737 learned_addr learned my addr 172.21.15.16:0/3094830737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d93fff640 1 -- 172.21.15.16:0/3094830737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9419a9a0 con 0x7f7d94106470 2024-09-15T22:28:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d93fff640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d9419a460 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009940 tx=0x7f7d84032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d917fa640 1 -- 172.21.15.16:0/3094830737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d84042070 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d917fa640 1 -- 172.21.15.16:0/3094830737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d840308c0 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9419ac30 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d917fa640 1 -- 172.21.15.16:0/3094830737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d84037040 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.130+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9419d920 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.131+0000 7f7d917fa640 1 -- 172.21.15.16:0/3094830737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f7d84045070 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.131+0000 7f7d917fa640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f7d7c039050 0x7f7d7c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.131+0000 7f7d917fa640 1 -- 172.21.15.16:0/3094830737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d8406d0b0 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.131+0000 7f7d937fe640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f7d7c039050 0x7f7d7c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.131+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9410aba0 con 0x7f7d94106470 2024-09-15T22:28:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.132+0000 7f7d937fe640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f7d7c039050 0x7f7d7c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d880098c0 tx=0x7f7d88006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.137+0000 7f7d917fa640 1 -- 172.21.15.16:0/3094830737 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d8403cd90 con 0x7f7d94106470 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.378+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7d9419de70 con 0x7f7d94106470 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.378+0000 7f7d917fa640 1 -- 172.21.15.16:0/3094830737 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f7d8403c870 con 0x7f7d94106470 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f7d7c039050 msgr2=0x7f7d7c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f7d7c039050 0x7f7d7c03b510 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7d880098c0 tx=0x7f7d88006db0 comp rx=0 tx=0).stop 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 msgr2=0x7f7d9419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d9419a460 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009940 tx=0x7f7d84032ce0 comp rx=0 tx=0).stop 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 shutdown_connections 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f7d7c039050 0x7f7d7c03b510 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 --2- 172.21.15.16:0/3094830737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d94106470 0x7f7d9419a460 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:38.942 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 >> 172.21.15.16:0/3094830737 conn(0x7f7d94101c00 msgr2=0x7f7d94102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:38.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 shutdown_connections 2024-09-15T22:28:38.943 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:38.381+0000 7f7d9a2a1640 1 -- 172.21.15.16:0/3094830737 wait complete. 2024-09-15T22:28:38.943 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-09-15T22:28:39.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:39 smithi016 ceph-mon[31492]: mgrmap e4: x(active, since 2s) 2024-09-15T22:28:40.009 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:40.009 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T22:28:40.009 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:40.009 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T22:28:40.010 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.309+0000 7fb4cc321640 1 Processor -- start 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.309+0000 7fb4cc321640 1 -- start start 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.309+0000 7fb4cc321640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c4107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.309+0000 7fb4cc321640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4107f00 con 0x7fb4c4107530 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.309+0000 7fb4ca0bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c4107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.309+0000 7fb4ca0bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c4107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46514/0 (socket says 172.21.15.16:46514) 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.309+0000 7fb4ca0bd640 1 -- 172.21.15.16:0/3413128741 learned_addr learned my addr 172.21.15.16:0/3413128741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.310+0000 7fb4ca0bd640 1 -- 172.21.15.16:0/3413128741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c4108730 con 0x7fb4c4107530 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.310+0000 7fb4ca0bd640 1 --2- 172.21.15.16:0/3413128741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c4107930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009820 tx=0x7fb4c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e33f9f5d40c3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.310+0000 7fb4c90bb640 1 -- 172.21.15.16:0/3413128741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0037070 con 0x7fb4c4107530 2024-09-15T22:28:40.011 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.310+0000 7fb4c90bb640 1 -- 172.21.15.16:0/3413128741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4c0032900 con 0x7fb4c4107530 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.310+0000 7fb4c90bb640 1 -- 172.21.15.16:0/3413128741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0030850 con 0x7fb4c4107530 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.311+0000 7fb4cc321640 1 -- 172.21.15.16:0/3413128741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 msgr2=0x7fb4c4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.311+0000 7fb4cc321640 1 --2- 172.21.15.16:0/3413128741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c4107930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009820 tx=0x7fb4c002a270 comp rx=0 tx=0).stop 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.311+0000 7fb4cc321640 1 -- 172.21.15.16:0/3413128741 shutdown_connections 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.311+0000 7fb4cc321640 1 --2- 172.21.15.16:0/3413128741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c4107930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.311+0000 7fb4cc321640 1 -- 172.21.15.16:0/3413128741 >> 172.21.15.16:0/3413128741 conn(0x7fb4c4102cc0 msgr2=0x7fb4c4105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.311+0000 7fb4cc321640 1 -- 172.21.15.16:0/3413128741 shutdown_connections 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.311+0000 7fb4cc321640 1 -- 172.21.15.16:0/3413128741 wait complete. 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4cc321640 1 Processor -- start 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4cc321640 1 -- start start 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4cc321640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c419d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4cc321640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c410bae0 con 0x7fb4c4107530 2024-09-15T22:28:40.012 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4ca0bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c419d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4ca0bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c419d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46530/0 (socket says 172.21.15.16:46530) 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4ca0bd640 1 -- 172.21.15.16:0/3371212968 learned_addr learned my addr 172.21.15.16:0/3371212968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4ca0bd640 1 -- 172.21.15.16:0/3371212968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c419dcc0 con 0x7fb4c4107530 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.312+0000 7fb4ca0bd640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c419d780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009940 tx=0x7fb4c0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.313+0000 7fb4bb7fe640 1 -- 172.21.15.16:0/3371212968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0042070 con 0x7fb4c4107530 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.313+0000 7fb4bb7fe640 1 -- 172.21.15.16:0/3371212968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4c00308c0 con 0x7fb4c4107530 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.313+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c419df50 con 0x7fb4c4107530 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.313+0000 7fb4bb7fe640 1 -- 172.21.15.16:0/3371212968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0037040 con 0x7fb4c4107530 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.313+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c41a0c40 con 0x7fb4c4107530 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.314+0000 7fb4bb7fe640 1 -- 172.21.15.16:0/3371212968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fb4c0045070 con 0x7fb4c4107530 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.314+0000 7fb4bb7fe640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fb4b00391b0 0x7fb4b003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.314+0000 7fb4bb7fe640 1 -- 172.21.15.16:0/3371212968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4c006c1e0 con 0x7fb4c4107530 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.314+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c410bc60 con 0x7fb4c4107530 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.314+0000 7fb4c98bc640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fb4b00391b0 0x7fb4b003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.314+0000 7fb4c98bc640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fb4b00391b0 0x7fb4b003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40098c0 tx=0x7fb4b4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.319+0000 7fb4bb7fe640 1 -- 172.21.15.16:0/3371212968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4c004b170 con 0x7fb4c4107530 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.493+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb4c41079b0 con 0x7fb4c4107530 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.494+0000 7fb4bb7fe640 1 -- 172.21.15.16:0/3371212968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fb4c002fd90 con 0x7fb4c4107530 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fb4b00391b0 msgr2=0x7fb4b003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fb4b00391b0 0x7fb4b003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40098c0 tx=0x7fb4b4006db0 comp rx=0 tx=0).stop 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 msgr2=0x7fb4c419d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c419d780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009940 tx=0x7fb4c0032ce0 comp rx=0 tx=0).stop 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 shutdown_connections 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fb4b00391b0 0x7fb4b003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:40.014 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 --2- 172.21.15.16:0/3371212968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c4107530 0x7fb4c419d780 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:40.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 >> 172.21.15.16:0/3371212968 conn(0x7fb4c4102cc0 msgr2=0x7fb4c4103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:40.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 shutdown_connections 2024-09-15T22:28:40.015 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:39.496+0000 7fb4cc321640 1 -- 172.21.15.16:0/3371212968 wait complete. 2024-09-15T22:28:40.015 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-09-15T22:28:40.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:40 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3371212968' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T22:28:41.789 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:41 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/905901443' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-15T22:28:41.790 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ignoring --setuser ceph since I am not root 2024-09-15T22:28:41.790 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ignoring --setgroup ceph since I am not root 2024-09-15T22:28:41.790 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:41.607+0000 7f1d60ed5040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:28:41.790 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:41.666+0000 7f1d60ed5040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:28:42.022 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.396+0000 7fcbaf87d640 1 Processor -- start 2024-09-15T22:28:42.022 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.396+0000 7fcbaf87d640 1 -- start start 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.396+0000 7fcbaf87d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.396+0000 7fcbaf87d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcba807c870 con 0x7fcba807de40 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.397+0000 7fcbad619640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.397+0000 7fcbad619640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46544/0 (socket says 172.21.15.16:46544) 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.397+0000 7fcbad619640 1 -- 172.21.15.16:0/3787376937 learned_addr learned my addr 172.21.15.16:0/3787376937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.397+0000 7fcbad619640 1 -- 172.21.15.16:0/3787376937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba807c9f0 con 0x7fcba807de40 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcbad619640 1 --2- 172.21.15.16:0/3787376937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba807c2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009820 tx=0x7fcba402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe82392e34eb2a24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcb9ffff640 1 -- 172.21.15.16:0/3787376937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba4037070 con 0x7fcba807de40 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcb9ffff640 1 -- 172.21.15.16:0/3787376937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcba4032900 con 0x7fcba807de40 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcb9ffff640 1 -- 172.21.15.16:0/3787376937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba4030850 con 0x7fcba807de40 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcbaf87d640 1 -- 172.21.15.16:0/3787376937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 msgr2=0x7fcba807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:42.023 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcbaf87d640 1 --2- 172.21.15.16:0/3787376937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba807c2a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009820 tx=0x7fcba402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcbaf87d640 1 -- 172.21.15.16:0/3787376937 shutdown_connections 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.398+0000 7fcbaf87d640 1 --2- 172.21.15.16:0/3787376937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba807c2a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.399+0000 7fcbaf87d640 1 -- 172.21.15.16:0/3787376937 >> 172.21.15.16:0/3787376937 conn(0x7fcba8102cc0 msgr2=0x7fcba8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.399+0000 7fcbaf87d640 1 -- 172.21.15.16:0/3787376937 shutdown_connections 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.399+0000 7fcbaf87d640 1 -- 172.21.15.16:0/3787376937 wait complete. 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.399+0000 7fcbaf87d640 1 Processor -- start 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.399+0000 7fcbaf87d640 1 -- start start 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.399+0000 7fcbaf87d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.399+0000 7fcbaf87d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcba810dba0 con 0x7fcba807de40 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcbad619640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcbad619640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46546/0 (socket says 172.21.15.16:46546) 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcbad619640 1 -- 172.21.15.16:0/905901443 learned_addr learned my addr 172.21.15.16:0/905901443 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:42.024 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcbad619640 1 -- 172.21.15.16:0/905901443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba819dc80 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcbad619640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba819d740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcba40097f0 tx=0x7fcba4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba4037040 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcba4040070 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba819df10 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba403a390 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.400+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a0c00 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.401+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fcba403a5e0 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.402+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba807c320 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.402+0000 7fcb9e7fc640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fcb940391b0 0x7fcb9403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.402+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcba406c380 con 0x7fcba807de40 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.402+0000 7fcbace18640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fcb940391b0 0x7fcb9403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:42.025 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.402+0000 7fcbace18640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fcb940391b0 0x7fcb9403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb980098c0 tx=0x7fcb98006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.407+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcba4042310 con 0x7fcba807de40 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:40.628+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fcba819e390 con 0x7fcba807de40 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.456+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fcba40371e0 con 0x7fcba807de40 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.457+0000 7fcb9e7fc640 1 -- 172.21.15.16:0/905901443 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fcba404acf0 con 0x7fcba807de40 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fcb940391b0 msgr2=0x7fcb9403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fcb940391b0 0x7fcb9403b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb980098c0 tx=0x7fcb98006db0 comp rx=0 tx=0).stop 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 msgr2=0x7fcba819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba819d740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcba40097f0 tx=0x7fcba4030d80 comp rx=0 tx=0).stop 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 shutdown_connections 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fcb940391b0 0x7fcb9403b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 --2- 172.21.15.16:0/905901443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba807de40 0x7fcba819d740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 >> 172.21.15.16:0/905901443 conn(0x7fcba8102cc0 msgr2=0x7fcba8103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:42.026 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.459+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 shutdown_connections 2024-09-15T22:28:42.027 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:41.460+0000 7fcbaf87d640 1 -- 172.21.15.16:0/905901443 wait complete. 2024-09-15T22:28:42.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:41.821+0000 7f1d60ed5040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:28:42.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:41.891+0000 7f1d60ed5040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:28:42.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:42.055+0000 7f1d60ed5040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:28:42.350 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:42.202+0000 7f1d60ed5040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:28:42.605 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:42.540+0000 7f1d60ed5040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:28:42.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:42 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/905901443' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-15T22:28:42.605 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:42 smithi016 ceph-mon[31492]: mgrmap e5: x(active, since 5s) 2024-09-15T22:28:43.012 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:42.603+0000 7f1d60ed5040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:28:43.012 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:42.799+0000 7f1d60ed5040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:28:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:28:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-15T22:28:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:28:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T22:28:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T22:28:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:28:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.428+0000 7fe9e0c58640 1 Processor -- start 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.428+0000 7fe9e0c58640 1 -- start start 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.428+0000 7fe9e0c58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc10a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.428+0000 7fe9e0c58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc07bdf0 con 0x7fe9dc108000 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.428+0000 7fe9da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc10a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.429+0000 7fe9da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc10a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51322/0 (socket says 172.21.15.16:51322) 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.429+0000 7fe9da59c640 1 -- 172.21.15.16:0/95322741 learned_addr learned my addr 172.21.15.16:0/95322741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.429+0000 7fe9da59c640 1 -- 172.21.15.16:0/95322741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc10a930 con 0x7fe9dc108000 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9da59c640 1 --2- 172.21.15.16:0/95322741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc10a3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009820 tx=0x7fe9cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e1547dc3624f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9d9d9b640 1 -- 172.21.15.16:0/95322741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc037070 con 0x7fe9dc108000 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9d9d9b640 1 -- 172.21.15.16:0/95322741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9cc032900 con 0x7fe9dc108000 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9d9d9b640 1 -- 172.21.15.16:0/95322741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc030850 con 0x7fe9dc108000 2024-09-15T22:28:43.272 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9e0c58640 1 -- 172.21.15.16:0/95322741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 msgr2=0x7fe9dc10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9e0c58640 1 --2- 172.21.15.16:0/95322741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc10a3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009820 tx=0x7fe9cc02a270 comp rx=0 tx=0).stop 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9e0c58640 1 -- 172.21.15.16:0/95322741 shutdown_connections 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9e0c58640 1 --2- 172.21.15.16:0/95322741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc10a3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.430+0000 7fe9e0c58640 1 -- 172.21.15.16:0/95322741 >> 172.21.15.16:0/95322741 conn(0x7fe9dc101bc0 msgr2=0x7fe9dc104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.431+0000 7fe9e0c58640 1 -- 172.21.15.16:0/95322741 shutdown_connections 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.431+0000 7fe9e0c58640 1 -- 172.21.15.16:0/95322741 wait complete. 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.431+0000 7fe9e0c58640 1 Processor -- start 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.431+0000 7fe9e0c58640 1 -- start start 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.431+0000 7fe9e0c58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc107880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.431+0000 7fe9e0c58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10b970 con 0x7fe9dc108000 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc107880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc107880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51330/0 (socket says 172.21.15.16:51330) 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9da59c640 1 -- 172.21.15.16:0/3538606588 learned_addr learned my addr 172.21.15.16:0/3538606588 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:43.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9da59c640 1 -- 172.21.15.16:0/3538606588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc107dc0 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9da59c640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc107880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0097f0 tx=0x7fe9cc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9d37fe640 1 -- 172.21.15.16:0/3538606588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc042070 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc105fb0 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9d37fe640 1 -- 172.21.15.16:0/3538606588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9cc032ca0 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9d37fe640 1 -- 172.21.15.16:0/3538606588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc037040 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.432+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc1064f0 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.433+0000 7fe9d37fe640 1 -- 172.21.15.16:0/3538606588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fe9cc045070 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.433+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc108460 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.433+0000 7fe9d37fe640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 0x7fe9c803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.433+0000 7fe9d3fff640 1 -- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 msgr2=0x7fe9c803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3949807891 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.433+0000 7fe9d37fe640 1 -- 172.21.15.16:0/3538606588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe9cc06d1c0 con 0x7fe9dc108000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.433+0000 7fe9d3fff640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 0x7fe9c803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:28:43.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.443+0000 7fe9d37fe640 1 -- 172.21.15.16:0/3538606588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9cc045520 con 0x7fe9dc108000 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.634+0000 7fe9d3fff640 1 -- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 msgr2=0x7fe9c803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3949807891 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.634+0000 7fe9d3fff640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 0x7fe9c803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.666+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe9dc106770 con 0x7fe9dc108000 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.667+0000 7fe9d37fe640 1 -- 172.21.15.16:0/3538606588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fe9cc03add0 con 0x7fe9dc108000 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 msgr2=0x7fe9c803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 0x7fe9c803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 msgr2=0x7fe9dc107880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc107880 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0097f0 tx=0x7fe9cc030ec0 comp rx=0 tx=0).stop 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 shutdown_connections 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7fe9c8039200 0x7fe9c803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 --2- 172.21.15.16:0/3538606588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9dc108000 0x7fe9dc107880 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:43.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 >> 172.21.15.16:0/3538606588 conn(0x7fe9dc101bc0 msgr2=0x7fe9dc1027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:43.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 shutdown_connections 2024-09-15T22:28:43.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:42.669+0000 7fe9e0c58640 1 -- 172.21.15.16:0/3538606588 wait complete. 2024-09-15T22:28:43.276 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-15T22:28:43.276 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-09-15T22:28:43.540 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:43 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3538606588' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T22:28:43.804 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:43.732+0000 7f1d60ed5040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:28:44.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:43.802+0000 7f1d60ed5040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:28:44.806 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:44.545+0000 7f1d60ed5040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:28:44.807 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:44.665+0000 7f1d60ed5040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:28:45.107 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:44.814+0000 7f1d60ed5040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:28:45.107 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:44.878+0000 7f1d60ed5040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:28:45.107 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.034+0000 7f1d60ed5040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:28:45.107 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.105+0000 7f1d60ed5040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:28:45.402 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.168+0000 7f1d60ed5040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:28:45.402 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.333+0000 7f1d60ed5040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:28:45.402 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.401+0000 7f1d60ed5040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:28:45.681 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.470+0000 7f1d60ed5040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:28:45.681 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.556+0000 7f1d60ed5040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:28:45.681 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.618+0000 7f1d60ed5040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:28:45.681 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:45.679+0000 7f1d60ed5040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:46.505+0000 7f1d60ed5040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: Active manager daemon x restarted 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: Activating manager daemon x 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: osdmap e2: 0 total, 0 up, 0 in 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: mgrmap e6: x(active, starting, since 0.00362329s) 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:28:46.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: Manager daemon x is now available 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:28:46.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:46 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:28:48.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:28:48.069 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32b251e640 1 Processor -- start 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32b251e640 1 -- start start 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32b251e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac10a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32b251e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac075750 con 0x7f32ac1080a0 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac10a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac10a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51332/0 (socket says 172.21.15.16:51332) 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32abfff640 1 -- 172.21.15.16:0/1174080682 learned_addr learned my addr 172.21.15.16:0/1174080682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.622+0000 7f32abfff640 1 -- 172.21.15.16:0/1174080682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac10a9d0 con 0x7f32ac1080a0 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.623+0000 7f32abfff640 1 --2- 172.21.15.16:0/1174080682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac10a490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f329c009820 tx=0x7f329c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b80bd67344e9c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.623+0000 7f32aaffd640 1 -- 172.21.15.16:0/1174080682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c037070 con 0x7f32ac1080a0 2024-09-15T22:28:48.070 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.623+0000 7f32aaffd640 1 -- 172.21.15.16:0/1174080682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f329c032900 con 0x7f32ac1080a0 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.623+0000 7f32aaffd640 1 -- 172.21.15.16:0/1174080682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c030850 con 0x7f32ac1080a0 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 -- 172.21.15.16:0/1174080682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 msgr2=0x7f32ac10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 --2- 172.21.15.16:0/1174080682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac10a490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f329c009820 tx=0x7f329c02a270 comp rx=0 tx=0).stop 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 -- 172.21.15.16:0/1174080682 shutdown_connections 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 --2- 172.21.15.16:0/1174080682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac10a490 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 -- 172.21.15.16:0/1174080682 >> 172.21.15.16:0/1174080682 conn(0x7f32ac101c80 msgr2=0x7f32ac1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 -- 172.21.15.16:0/1174080682 shutdown_connections 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 -- 172.21.15.16:0/1174080682 wait complete. 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.624+0000 7f32b251e640 1 Processor -- start 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32b251e640 1 -- start start 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32b251e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac107920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:48.071 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32b251e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac10b9b0 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac107920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac107920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51334/0 (socket says 172.21.15.16:51334) 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32abfff640 1 -- 172.21.15.16:0/1442023502 learned_addr learned my addr 172.21.15.16:0/1442023502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32abfff640 1 -- 172.21.15.16:0/1442023502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac107e60 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.625+0000 7f32abfff640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac107920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f329c031000 tx=0x7f329c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.626+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c042070 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.626+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f329c032be0 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.626+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac106050 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.626+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f329c037040 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.626+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac106590 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.627+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f329c045070 con 0x7f32ac1080a0 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.627+0000 7f32a97fa640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 0x7f329403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.627+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 --> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f329403bd80 con 0x7f32940391b0 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.627+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f329c06c540 con 0x7f32ac1080a0 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.627+0000 7f32ab7fe640 1 -- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 msgr2=0x7f329403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3949807891 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.627+0000 7f32ab7fe640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 0x7f329403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.828+0000 7f32ab7fe640 1 -- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 msgr2=0x7f329403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3949807891 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:43.828+0000 7f32ab7fe640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 0x7f329403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:44.229+0000 7f32ab7fe640 1 -- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 msgr2=0x7f329403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3949807891 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:44.229+0000 7f32ab7fe640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 0x7f329403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:45.030+0000 7f32ab7fe640 1 -- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 msgr2=0x7f329403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/3949807891 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:45.030+0000 7f32ab7fe640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 0x7f329403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:46.508+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f329c06b000 con 0x7f32ac1080a0 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:46.509+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 msgr2=0x7f329403b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:46.509+0000 7f32a97fa640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 0x7f329403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.509+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f329c03cb30 con 0x7f32ac1080a0 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.509+0000 7f32a97fa640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f329403cae0 0x7f329403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.509+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f329c03ee50 con 0x7f329403cae0 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.510+0000 7f32ab7fe640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f329403cae0 0x7f329403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.510+0000 7f32ab7fe640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f329403cae0 0x7f329403eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32a0003980 tx=0x7f32a0006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.511+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f329c03ee50 con 0x7f329403cae0 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.518+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f32ac108500 con 0x7f329403cae0 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.518+0000 7f32a97fa640 1 -- 172.21.15.16:0/1442023502 <== mgr.14118 v2:172.21.15.16:6800/2652317700 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f32ac108500 con 0x7f329403cae0 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f329403cae0 msgr2=0x7f329403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f329403cae0 0x7f329403eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f32a0003980 tx=0x7f32a0006e10 comp rx=0 tx=0).stop 2024-09-15T22:28:48.074 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 msgr2=0x7f32ac107920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac107920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f329c031000 tx=0x7f329c0326e0 comp rx=0 tx=0).stop 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 shutdown_connections 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f329403cae0 0x7f329403eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:6800/3949807891,v1:172.21.15.16:6801/3949807891] conn(0x7f32940391b0 0x7f329403b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 --2- 172.21.15.16:0/1442023502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac1080a0 0x7f32ac107920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 >> 172.21.15.16:0/1442023502 conn(0x7f32ac101c80 msgr2=0x7f32ac1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 shutdown_connections 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:47.519+0000 7f32b251e640 1 -- 172.21.15.16:0/1442023502 wait complete. 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-09-15T22:28:48.075 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-09-15T22:28:48.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:48 smithi016 ceph-mon[31492]: Found migration_current of "None". Setting to last migration. 2024-09-15T22:28:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:48 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:48 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:48 smithi016 ceph-mon[31492]: mgrmap e7: x(active, since 1.00535s) 2024-09-15T22:28:49.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:49.029+0000 7f1ccd9fa640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:28:49] ENGINE Error in HTTPServer.serve 2024-09-15T22:28:49.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: Traceback (most recent call last): 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._connections.run(self.expiration_interval) 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._run(expiration_interval) 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: s = self.context.wrap_socket( 2024-09-15T22:28:49.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:28:49.269 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: return self.sslsocket_class._create( 2024-09-15T22:28:49.269 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:28:49.269 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self.do_handshake() 2024-09-15T22:28:49.269 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:28:49.269 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._sslobj.do_handshake() 2024-09-15T22:28:49.269 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:28:49.269 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:28:49.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.482+0000 7f3d08fad640 1 Processor -- start 2024-09-15T22:28:49.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.482+0000 7f3d08fad640 1 -- start start 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.483+0000 7f3d08fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0410a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.483+0000 7f3d08fad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d04075790 con 0x7f3d04108060 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.483+0000 7f3d0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0410a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.483+0000 7f3d0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0410a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51400/0 (socket says 172.21.15.16:51400) 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.483+0000 7f3d0259c640 1 -- 172.21.15.16:0/1950578116 learned_addr learned my addr 172.21.15.16:0/1950578116 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.483+0000 7f3d0259c640 1 -- 172.21.15.16:0/1950578116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0410a990 con 0x7f3d04108060 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.484+0000 7f3d0259c640 1 --2- 172.21.15.16:0/1950578116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0410a450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009820 tx=0x7f3cf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9076bf5888ae9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.484+0000 7f3d0159a640 1 -- 172.21.15.16:0/1950578116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cf4037070 con 0x7f3d04108060 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.484+0000 7f3d0159a640 1 -- 172.21.15.16:0/1950578116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3cf4032900 con 0x7f3d04108060 2024-09-15T22:28:49.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.484+0000 7f3d0159a640 1 -- 172.21.15.16:0/1950578116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cf4030850 con 0x7f3d04108060 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.485+0000 7f3d08fad640 1 -- 172.21.15.16:0/1950578116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 msgr2=0x7f3d0410a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.485+0000 7f3d08fad640 1 --2- 172.21.15.16:0/1950578116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0410a450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009820 tx=0x7f3cf402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.485+0000 7f3d08fad640 1 -- 172.21.15.16:0/1950578116 shutdown_connections 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.485+0000 7f3d08fad640 1 --2- 172.21.15.16:0/1950578116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0410a450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.485+0000 7f3d08fad640 1 -- 172.21.15.16:0/1950578116 >> 172.21.15.16:0/1950578116 conn(0x7f3d04101c00 msgr2=0x7f3d04104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.485+0000 7f3d08fad640 1 -- 172.21.15.16:0/1950578116 shutdown_connections 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.485+0000 7f3d08fad640 1 -- 172.21.15.16:0/1950578116 wait complete. 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d08fad640 1 Processor -- start 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d08fad640 1 -- start start 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d08fad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0419b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:49.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d08fad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0410b970 con 0x7f3d04108060 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0419b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0419b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51414/0 (socket says 172.21.15.16:51414) 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d0259c640 1 -- 172.21.15.16:0/490241282 learned_addr learned my addr 172.21.15.16:0/490241282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d0259c640 1 -- 172.21.15.16:0/490241282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0419bab0 con 0x7f3d04108060 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.486+0000 7f3d0259c640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0419b570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40097f0 tx=0x7f3cf4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.487+0000 7f3cf37fe640 1 -- 172.21.15.16:0/490241282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cf4037040 con 0x7f3d04108060 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.487+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0419bd40 con 0x7f3d04108060 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.487+0000 7f3cf37fe640 1 -- 172.21.15.16:0/490241282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3cf4040070 con 0x7f3d04108060 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.487+0000 7f3cf37fe640 1 -- 172.21.15.16:0/490241282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cf403a390 con 0x7f3d04108060 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.487+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0419ea30 con 0x7f3d04108060 2024-09-15T22:28:49.278 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.488+0000 7f3cf37fe640 1 -- 172.21.15.16:0/490241282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f3cf403a630 con 0x7f3d04108060 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.488+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d041084c0 con 0x7f3d04108060 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.488+0000 7f3cf37fe640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3cec0390a0 0x7f3cec03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.488+0000 7f3cf37fe640 1 -- 172.21.15.16:0/490241282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3cf406c5f0 con 0x7f3d04108060 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.488+0000 7f3d01d9b640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3cec0390a0 0x7f3cec03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.488+0000 7f3d01d9b640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3cec0390a0 0x7f3cec03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800ad30 tx=0x7f3cf80093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.493+0000 7f3cf37fe640 1 -- 172.21.15.16:0/490241282 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3cf404a200 con 0x7f3d04108060 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.703+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3d04102ed0 con 0x7f3cec0390a0 2024-09-15T22:28:49.279 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.707+0000 7f3cf37fe640 1 -- 172.21.15.16:0/490241282 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d04102ed0 con 0x7f3cec0390a0 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.709+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3cec0390a0 msgr2=0x7f3cec03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.709+0000 7f3d08fad640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3cec0390a0 0x7f3cec03b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800ad30 tx=0x7f3cf80093f0 comp rx=0 tx=0).stop 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.709+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 msgr2=0x7f3d0419b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.709+0000 7f3d08fad640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0419b570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40097f0 tx=0x7f3cf4030d80 comp rx=0 tx=0).stop 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.710+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 shutdown_connections 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.710+0000 7f3d08fad640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3cec0390a0 0x7f3cec03b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.710+0000 7f3d08fad640 1 --2- 172.21.15.16:0/490241282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d04108060 0x7f3d0419b570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.710+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 >> 172.21.15.16:0/490241282 conn(0x7f3d04101c00 msgr2=0x7f3d041027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.710+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 shutdown_connections 2024-09-15T22:28:49.280 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:48.710+0000 7f3d08fad640 1 -- 172.21.15.16:0/490241282 wait complete. 2024-09-15T22:28:49.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-mon[31492]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T22:28:49.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-mon[31492]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T22:28:49.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:49.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:49.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:49 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:50.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.725+0000 7f3a77fea640 1 Processor -- start 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.725+0000 7f3a77fea640 1 -- start start 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.726+0000 7f3a77fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a70107950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.726+0000 7f3a77fea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70107f20 con 0x7f3a70107550 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.726+0000 7f3a75d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a70107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.726+0000 7f3a75d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a70107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51422/0 (socket says 172.21.15.16:51422) 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.726+0000 7f3a75d86640 1 -- 172.21.15.16:0/2262857399 learned_addr learned my addr 172.21.15.16:0/2262857399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.726+0000 7f3a75d86640 1 -- 172.21.15.16:0/2262857399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a70108750 con 0x7f3a70107550 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.727+0000 7f3a75d86640 1 --2- 172.21.15.16:0/2262857399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a70107950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009820 tx=0x7f3a6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6e8617fc947c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.727+0000 7f3a74d84640 1 -- 172.21.15.16:0/2262857399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a68037070 con 0x7f3a70107550 2024-09-15T22:28:50.484 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.727+0000 7f3a74d84640 1 -- 172.21.15.16:0/2262857399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a68032900 con 0x7f3a70107550 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.727+0000 7f3a74d84640 1 -- 172.21.15.16:0/2262857399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a68030850 con 0x7f3a70107550 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 -- 172.21.15.16:0/2262857399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 msgr2=0x7f3a70107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 --2- 172.21.15.16:0/2262857399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a70107950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009820 tx=0x7f3a6802a270 comp rx=0 tx=0).stop 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 -- 172.21.15.16:0/2262857399 shutdown_connections 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 --2- 172.21.15.16:0/2262857399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a70107950 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 -- 172.21.15.16:0/2262857399 >> 172.21.15.16:0/2262857399 conn(0x7f3a70102d00 msgr2=0x7f3a70105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 -- 172.21.15.16:0/2262857399 shutdown_connections 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 -- 172.21.15.16:0/2262857399 wait complete. 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 Processor -- start 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.728+0000 7f3a77fea640 1 -- start start 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a77fea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a7019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:50.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a77fea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7010bb00 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a75d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a7019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a75d86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a7019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51432/0 (socket says 172.21.15.16:51432) 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a75d86640 1 -- 172.21.15.16:0/3310559119 learned_addr learned my addr 172.21.15.16:0/3310559119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a75d86640 1 -- 172.21.15.16:0/3310559119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7019dd10 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a75d86640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a7019d7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009940 tx=0x7f3a68032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a6effd640 1 -- 172.21.15.16:0/3310559119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a68042070 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a6effd640 1 -- 172.21.15.16:0/3310559119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a680308c0 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7019dfa0 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a6effd640 1 -- 172.21.15.16:0/3310559119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a68037040 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.729+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a701a0c90 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.730+0000 7f3a6effd640 1 -- 172.21.15.16:0/3310559119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f3a68045070 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.730+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7010bc80 con 0x7f3a70107550 2024-09-15T22:28:50.486 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.730+0000 7f3a6effd640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3a58039200 0x7f3a5803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.731+0000 7f3a6effd640 1 -- 172.21.15.16:0/3310559119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3a6806c320 con 0x7f3a70107550 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.731+0000 7f3a75585640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3a58039200 0x7f3a5803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.731+0000 7f3a75585640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3a58039200 0x7f3a5803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400ad30 tx=0x7f3a640093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.735+0000 7f3a6effd640 1 -- 172.21.15.16:0/3310559119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a6804b170 con 0x7f3a70107550 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.914+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f3a70103f60 con 0x7f3a58039200 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.914+0000 7f3a6effd640 1 -- 172.21.15.16:0/3310559119 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f3a70103f60 con 0x7f3a58039200 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3a58039200 msgr2=0x7f3a5803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3a58039200 0x7f3a5803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400ad30 tx=0x7f3a640093f0 comp rx=0 tx=0).stop 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 msgr2=0x7f3a7019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a7019d7d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3a68009940 tx=0x7f3a68032ce0 comp rx=0 tx=0).stop 2024-09-15T22:28:50.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 shutdown_connections 2024-09-15T22:28:50.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f3a58039200 0x7f3a5803b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:50.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 --2- 172.21.15.16:0/3310559119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a70107550 0x7f3a7019d7d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:50.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 >> 172.21.15.16:0/3310559119 conn(0x7f3a70102d00 msgr2=0x7f3a70103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:50.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 shutdown_connections 2024-09-15T22:28:50.488 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:49.916+0000 7f3a77fea640 1 -- 172.21.15.16:0/3310559119 wait complete. 2024-09-15T22:28:50.488 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:50 smithi016 ceph-mon[31492]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:50 smithi016 ceph-mon[31492]: [15/Sep/2024:22:28:48] ENGINE Bus STARTING 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:50 smithi016 ceph-mon[31492]: [15/Sep/2024:22:28:49] ENGINE Serving on https://172.21.15.16:7150 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:50 smithi016 ceph-mon[31492]: [15/Sep/2024:22:28:49] ENGINE Error in HTTPServer.serve 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-15T22:28:50.793 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:50 smithi016 ceph-mon[31492]: [15/Sep/2024:22:28:49] ENGINE Serving on http://172.21.15.16:8765 2024-09-15T22:28:50.794 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:50 smithi016 ceph-mon[31492]: [15/Sep/2024:22:28:49] ENGINE Bus STARTED 2024-09-15T22:28:50.795 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:50 smithi016 ceph-mon[31492]: mgrmap e8: x(active, since 3s) 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: Generating public/private rsa key pair. 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: Your identification has been saved in /tmp/tmpbb3gcjut/key 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: Your public key has been saved in /tmp/tmpbb3gcjut/key.pub 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: The key fingerprint is: 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: SHA256:kuxm9w7dQciCcubB5WCkR+8jW8v/TGAPaKnloOvkA2g ceph-af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: The key's randomart image is: 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: +---[RSA 3072]----+ 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: | .= . | 2024-09-15T22:28:51.508 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: | = * . . | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: | o B + o . | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: | B + + . | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: | . B S + . | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: |.E. o % = = . | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: |. .o * * . + | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: | o.+ . + o | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: | .+. .+.o | 2024-09-15T22:28:51.509 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: +----[SHA256]-----+ 2024-09-15T22:28:51.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-mon[31492]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:28:51.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:51.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:51 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.875+0000 7f579f7c6640 1 Processor -- start 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.875+0000 7f579f7c6640 1 -- start start 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.875+0000 7f579f7c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.875+0000 7f579f7c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f579807c870 con 0x7f579807de40 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.875+0000 7f579d562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.875+0000 7f579d562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579807c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51448/0 (socket says 172.21.15.16:51448) 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.875+0000 7f579d562640 1 -- 172.21.15.16:0/3023823343 learned_addr learned my addr 172.21.15.16:0/3023823343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.876+0000 7f579d562640 1 -- 172.21.15.16:0/3023823343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579807c9f0 con 0x7f579807de40 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.876+0000 7f579d562640 1 --2- 172.21.15.16:0/3023823343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579807c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5794009820 tx=0x7f579402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568bc9cf8caee3da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:51.830 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f578ffff640 1 -- 172.21.15.16:0/3023823343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5794037070 con 0x7f579807de40 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f578ffff640 1 -- 172.21.15.16:0/3023823343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5794032900 con 0x7f579807de40 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f578ffff640 1 -- 172.21.15.16:0/3023823343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5794030850 con 0x7f579807de40 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f579f7c6640 1 -- 172.21.15.16:0/3023823343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 msgr2=0x7f579807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f579f7c6640 1 --2- 172.21.15.16:0/3023823343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579807c2a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5794009820 tx=0x7f579402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f579f7c6640 1 -- 172.21.15.16:0/3023823343 shutdown_connections 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f579f7c6640 1 --2- 172.21.15.16:0/3023823343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579807c2a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f579f7c6640 1 -- 172.21.15.16:0/3023823343 >> 172.21.15.16:0/3023823343 conn(0x7f5798102cc0 msgr2=0x7f5798105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:51.831 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f579f7c6640 1 -- 172.21.15.16:0/3023823343 shutdown_connections 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.877+0000 7f579f7c6640 1 -- 172.21.15.16:0/3023823343 wait complete. 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.878+0000 7f579f7c6640 1 Processor -- start 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.878+0000 7f579f7c6640 1 -- start start 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.878+0000 7f579f7c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.878+0000 7f579f7c6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f579810dba0 con 0x7f579807de40 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.878+0000 7f579d562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.878+0000 7f579d562640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579819d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51450/0 (socket says 172.21.15.16:51450) 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.878+0000 7f579d562640 1 -- 172.21.15.16:0/30703953 learned_addr learned my addr 172.21.15.16:0/30703953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.879+0000 7f579d562640 1 -- 172.21.15.16:0/30703953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579819dc80 con 0x7f579807de40 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.879+0000 7f579d562640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579819d740 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57940097f0 tx=0x7f5794030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.879+0000 7f578e7fc640 1 -- 172.21.15.16:0/30703953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5794037040 con 0x7f579807de40 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.879+0000 7f578e7fc640 1 -- 172.21.15.16:0/30703953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5794040070 con 0x7f579807de40 2024-09-15T22:28:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.879+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579819df10 con 0x7f579807de40 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.879+0000 7f578e7fc640 1 -- 172.21.15.16:0/30703953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f579403a390 con 0x7f579807de40 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.879+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57981a0c00 con 0x7f579807de40 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.880+0000 7f578e7fc640 1 -- 172.21.15.16:0/30703953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f5794032900 con 0x7f579807de40 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.880+0000 7f578e7fc640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f5784039200 0x7f578403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.880+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579807c320 con 0x7f579807de40 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.880+0000 7f578e7fc640 1 -- 172.21.15.16:0/30703953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f579406c710 con 0x7f579807de40 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.880+0000 7f579cd61640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f5784039200 0x7f578403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.881+0000 7f579cd61640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f5784039200 0x7f578403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57880098c0 tx=0x7f5788006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:50.885+0000 7f578e7fc640 1 -- 172.21.15.16:0/30703953 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5794040cc0 con 0x7f579807de40 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.060+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f5798103f80 con 0x7f5784039200 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.252+0000 7f578e7fc640 1 -- 172.21.15.16:0/30703953 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5798103f80 con 0x7f5784039200 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f5784039200 msgr2=0x7f578403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:51.833 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f5784039200 0x7f578403b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57880098c0 tx=0x7f5788006db0 comp rx=0 tx=0).stop 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 msgr2=0x7f579819d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579819d740 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f57940097f0 tx=0x7f5794030d80 comp rx=0 tx=0).stop 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 shutdown_connections 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f5784039200 0x7f578403b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 --2- 172.21.15.16:0/30703953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579807de40 0x7f579819d740 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 >> 172.21.15.16:0/30703953 conn(0x7f5798102cc0 msgr2=0x7f5798103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 shutdown_connections 2024-09-15T22:28:51.834 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:51.254+0000 7f579f7c6640 1 -- 172.21.15.16:0/30703953 wait complete. 2024-09-15T22:28:52.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:52 smithi016 ceph-mon[31492]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:28:52.670 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:52 smithi016 ceph-mon[31492]: Generating ssh key... 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5Y+b/spprJuGE0Zka3RAdjSAxMu8rL8Ao3Zsr+NNIElFU2MTG/jXmqCgtKFKZjsMhDtXhRY7kRofSCi/+CEi8MavsJs8qpYX/dPAybC5hirGXrqe+bTVIVKhdxd6WjF0JNdeyCKFm0hte/myrLU1eHXXEGUrhp53RNvLKLPrFlfaPIl09DM0LJ7ktyliziWo4kGjwWBLRMXxs/YIYh5voAsEIDldHwQ/myIXdl4eu1IqdwhPzP/Pms9GFy4IBQDokb4fpXasDydWy5Ijv4W2cCp/+lYGTZSW+kuyV6qjehZJ8nqVJnv8jn5mG58XJU/SA+WIwynVU8y8vy1zOI2r0xszHo4wv7+vYjA7p3PzJmBf5Txv8ulV8BHS2ugD6pTgV38wcovAAf/lw01JftCg7qRYHGOEKl/W6BTI7ILFBQKQPsILxzHRZHyYYpG6bBr6xcS+J7FlrbgYuf+WpQJ/gDl10YFril5UfWfwiNv+07o8331gcXbfqq7ImX0Hs9Q0= ceph-af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.180+0000 7f65f296a640 1 Processor -- start 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.180+0000 7f65f296a640 1 -- start start 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.180+0000 7f65f296a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec10a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.180+0000 7f65f296a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec07be50 con 0x7f65ec107fa0 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.181+0000 7f65ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec10a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.181+0000 7f65ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec10a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53240/0 (socket says 172.21.15.16:53240) 2024-09-15T22:28:52.961 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.181+0000 7f65ebfff640 1 -- 172.21.15.16:0/3505810939 learned_addr learned my addr 172.21.15.16:0/3505810939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.181+0000 7f65ebfff640 1 -- 172.21.15.16:0/3505810939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec10a8d0 con 0x7f65ec107fa0 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65ebfff640 1 --2- 172.21.15.16:0/3505810939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec10a390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009820 tx=0x7f65dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387aedc4bb35472e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65eaffd640 1 -- 172.21.15.16:0/3505810939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc037070 con 0x7f65ec107fa0 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65eaffd640 1 -- 172.21.15.16:0/3505810939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65dc032900 con 0x7f65ec107fa0 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65eaffd640 1 -- 172.21.15.16:0/3505810939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc030850 con 0x7f65ec107fa0 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65f296a640 1 -- 172.21.15.16:0/3505810939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 msgr2=0x7f65ec10a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65f296a640 1 --2- 172.21.15.16:0/3505810939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec10a390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009820 tx=0x7f65dc02a270 comp rx=0 tx=0).stop 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65f296a640 1 -- 172.21.15.16:0/3505810939 shutdown_connections 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65f296a640 1 --2- 172.21.15.16:0/3505810939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec10a390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.182+0000 7f65f296a640 1 -- 172.21.15.16:0/3505810939 >> 172.21.15.16:0/3505810939 conn(0x7f65ec101bc0 msgr2=0x7f65ec104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.183+0000 7f65f296a640 1 -- 172.21.15.16:0/3505810939 shutdown_connections 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.183+0000 7f65f296a640 1 -- 172.21.15.16:0/3505810939 wait complete. 2024-09-15T22:28:52.962 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.183+0000 7f65f296a640 1 Processor -- start 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.183+0000 7f65f296a640 1 -- start start 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.183+0000 7f65f296a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.183+0000 7f65f296a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec10b970 con 0x7f65ec107fa0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53254/0 (socket says 172.21.15.16:53254) 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65ebfff640 1 -- 172.21.15.16:0/281368127 learned_addr learned my addr 172.21.15.16:0/281368127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65ebfff640 1 -- 172.21.15.16:0/281368127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec19dd10 con 0x7f65ec107fa0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65ebfff640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec19d7d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0097f0 tx=0x7f65dc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65e97fa640 1 -- 172.21.15.16:0/281368127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc042070 con 0x7f65ec107fa0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65e97fa640 1 -- 172.21.15.16:0/281368127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65dc032ca0 con 0x7f65ec107fa0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec19dfa0 con 0x7f65ec107fa0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65e97fa640 1 -- 172.21.15.16:0/281368127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65dc037040 con 0x7f65ec107fa0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.184+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a0c90 con 0x7f65ec107fa0 2024-09-15T22:28:52.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.185+0000 7f65e97fa640 1 -- 172.21.15.16:0/281368127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f65dc045070 con 0x7f65ec107fa0 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.185+0000 7f65e97fa640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f65d4039200 0x7f65d403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.185+0000 7f65e97fa640 1 -- 172.21.15.16:0/281368127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f65dc06c2b0 con 0x7f65ec107fa0 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.185+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec108400 con 0x7f65ec107fa0 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.186+0000 7f65eb7fe640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f65d4039200 0x7f65d403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.186+0000 7f65eb7fe640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f65d4039200 0x7f65d403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65e00098c0 tx=0x7f65e0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.190+0000 7f65e97fa640 1 -- 172.21.15.16:0/281368127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65dc045520 con 0x7f65ec107fa0 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.366+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f65ec102ea0 con 0x7f65d4039200 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.367+0000 7f65e97fa640 1 -- 172.21.15.16:0/281368127 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f65ec102ea0 con 0x7f65d4039200 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.370+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f65d4039200 msgr2=0x7f65d403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.370+0000 7f65f296a640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f65d4039200 0x7f65d403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65e00098c0 tx=0x7f65e0006db0 comp rx=0 tx=0).stop 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.370+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 msgr2=0x7f65ec19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:52.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.370+0000 7f65f296a640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec19d7d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0097f0 tx=0x7f65dc030ec0 comp rx=0 tx=0).stop 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.370+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 shutdown_connections 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.370+0000 7f65f296a640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f65d4039200 0x7f65d403b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.370+0000 7f65f296a640 1 --2- 172.21.15.16:0/281368127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65ec107fa0 0x7f65ec19d7d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.371+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 >> 172.21.15.16:0/281368127 conn(0x7f65ec101bc0 msgr2=0x7f65ec102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.371+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 shutdown_connections 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:52.371+0000 7f65f296a640 1 -- 172.21.15.16:0/281368127 wait complete. 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:28:52.965 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-09-15T22:28:52.968 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-09-15T22:28:53.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:53 smithi016 ceph-mon[31492]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:28:54.902 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:54 smithi016 ceph-mon[31492]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:28:55.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:55 smithi016 ceph-mon[31492]: Deploying cephadm binary to smithi016 2024-09-15T22:28:57.988 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.391+0000 7fc626713640 1 Processor -- start 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.391+0000 7fc626713640 1 -- start start 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.391+0000 7fc626713640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62010a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.391+0000 7fc626713640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6200757b0 con 0x7fc620108040 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.391+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62010a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.391+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62010a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53260/0 (socket says 172.21.15.16:53260) 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.391+0000 7fc61ffff640 1 -- 172.21.15.16:0/1387052965 learned_addr learned my addr 172.21.15.16:0/1387052965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.392+0000 7fc61ffff640 1 -- 172.21.15.16:0/1387052965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62010a970 con 0x7fc620108040 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.392+0000 7fc61ffff640 1 --2- 172.21.15.16:0/1387052965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62010a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc610009820 tx=0x7fc61002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b1af2a5400bd35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc61effd640 1 -- 172.21.15.16:0/1387052965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc610037070 con 0x7fc620108040 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc61effd640 1 -- 172.21.15.16:0/1387052965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc610032900 con 0x7fc620108040 2024-09-15T22:28:57.989 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc61effd640 1 -- 172.21.15.16:0/1387052965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc610030850 con 0x7fc620108040 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc626713640 1 -- 172.21.15.16:0/1387052965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 msgr2=0x7fc62010a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc626713640 1 --2- 172.21.15.16:0/1387052965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62010a430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc610009820 tx=0x7fc61002a270 comp rx=0 tx=0).stop 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc626713640 1 -- 172.21.15.16:0/1387052965 shutdown_connections 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc626713640 1 --2- 172.21.15.16:0/1387052965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62010a430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc626713640 1 -- 172.21.15.16:0/1387052965 >> 172.21.15.16:0/1387052965 conn(0x7fc620101c00 msgr2=0x7fc620104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc626713640 1 -- 172.21.15.16:0/1387052965 shutdown_connections 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.393+0000 7fc626713640 1 -- 172.21.15.16:0/1387052965 wait complete. 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.394+0000 7fc626713640 1 Processor -- start 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.394+0000 7fc626713640 1 -- start start 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc626713640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc626713640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc62010b9b0 con 0x7fc620108040 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:57.990 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53276/0 (socket says 172.21.15.16:53276) 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61ffff640 1 -- 172.21.15.16:0/2070921411 learned_addr learned my addr 172.21.15.16:0/2070921411 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61ffff640 1 -- 172.21.15.16:0/2070921411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62019baa0 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61ffff640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62019b560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6100097f0 tx=0x7fc610030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61d7fa640 1 -- 172.21.15.16:0/2070921411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc610037040 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61d7fa640 1 -- 172.21.15.16:0/2070921411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc610040070 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62019bd30 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc61d7fa640 1 -- 172.21.15.16:0/2070921411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc61003a390 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.395+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62019ea20 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.396+0000 7fc61d7fa640 1 -- 172.21.15.16:0/2070921411 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fc61003a5e0 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.397+0000 7fc61d7fa640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fc608039200 0x7fc60803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.397+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6201084a0 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.397+0000 7fc61d7fa640 1 -- 172.21.15.16:0/2070921411 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc61006c8b0 con 0x7fc620108040 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.397+0000 7fc61f7fe640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fc608039200 0x7fc60803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:57.991 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.397+0000 7fc61f7fe640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fc608039200 0x7fc60803b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6140098c0 tx=0x7fc614006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.402+0000 7fc61d7fa640 1 -- 172.21.15.16:0/2070921411 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc610043360 con 0x7fc620108040 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:53.581+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}) -- 0x7fc620102ee0 con 0x7fc608039200 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.445+0000 7fc61d7fa640 1 -- 172.21.15.16:0/2070921411 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc620102ee0 con 0x7fc608039200 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fc608039200 msgr2=0x7fc60803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fc608039200 0x7fc60803b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc6140098c0 tx=0x7fc614006db0 comp rx=0 tx=0).stop 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 msgr2=0x7fc62019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62019b560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc6100097f0 tx=0x7fc610030d80 comp rx=0 tx=0).stop 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 shutdown_connections 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fc608039200 0x7fc60803b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 --2- 172.21.15.16:0/2070921411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc620108040 0x7fc62019b560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 >> 172.21.15.16:0/2070921411 conn(0x7fc620101c00 msgr2=0x7fc6201027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 shutdown_connections 2024-09-15T22:28:57.992 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:57.448+0000 7fc626713640 1 -- 172.21.15.16:0/2070921411 wait complete. 2024-09-15T22:28:57.993 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-09-15T22:28:58.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:58 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:28:58.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:58 smithi016 ceph-mon[31492]: Added host smithi016 2024-09-15T22:28:58.626 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:58 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.382+0000 7fddf01a9640 1 Processor -- start 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.382+0000 7fddf01a9640 1 -- start start 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.383+0000 7fddf01a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde810a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.383+0000 7fddf01a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8075730 con 0x7fdde81080c0 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.383+0000 7fddedf45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde810a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.383+0000 7fddedf45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde810a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53284/0 (socket says 172.21.15.16:53284) 2024-09-15T22:28:59.194 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.383+0000 7fddedf45640 1 -- 172.21.15.16:0/3058679843 learned_addr learned my addr 172.21.15.16:0/3058679843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.383+0000 7fddedf45640 1 -- 172.21.15.16:0/3058679843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde810a9f0 con 0x7fdde81080c0 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.384+0000 7fddedf45640 1 --2- 172.21.15.16:0/3058679843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde810a4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009820 tx=0x7fdde402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f036c7a27801a502 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.384+0000 7fddecf43640 1 -- 172.21.15.16:0/3058679843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdde4037070 con 0x7fdde81080c0 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.384+0000 7fddecf43640 1 -- 172.21.15.16:0/3058679843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdde4032900 con 0x7fdde81080c0 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.384+0000 7fddecf43640 1 -- 172.21.15.16:0/3058679843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdde4030850 con 0x7fdde81080c0 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.384+0000 7fddf01a9640 1 -- 172.21.15.16:0/3058679843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 msgr2=0x7fdde810a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.384+0000 7fddf01a9640 1 --2- 172.21.15.16:0/3058679843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde810a4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdde4009820 tx=0x7fdde402a270 comp rx=0 tx=0).stop 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 -- 172.21.15.16:0/3058679843 shutdown_connections 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 --2- 172.21.15.16:0/3058679843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde810a4b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 -- 172.21.15.16:0/3058679843 >> 172.21.15.16:0/3058679843 conn(0x7fdde8101c00 msgr2=0x7fdde8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 -- 172.21.15.16:0/3058679843 shutdown_connections 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 -- 172.21.15.16:0/3058679843 wait complete. 2024-09-15T22:28:59.195 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 Processor -- start 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 -- start start 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.385+0000 7fddf01a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdde810b970 con 0x7fdde81080c0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fddedf45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fddedf45640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde819d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53300/0 (socket says 172.21.15.16:53300) 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fddedf45640 1 -- 172.21.15.16:0/4187185325 learned_addr learned my addr 172.21.15.16:0/4187185325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fddedf45640 1 -- 172.21.15.16:0/4187185325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde819dcb0 con 0x7fdde81080c0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fddedf45640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde819d770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdde4032ce0 tx=0x7fdde4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fdddeffd640 1 -- 172.21.15.16:0/4187185325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdde4042070 con 0x7fdde81080c0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde819df40 con 0x7fdde81080c0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fdddeffd640 1 -- 172.21.15.16:0/4187185325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdde40308c0 con 0x7fdde81080c0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fdddeffd640 1 -- 172.21.15.16:0/4187185325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdde4037040 con 0x7fdde81080c0 2024-09-15T22:28:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.386+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde81a0c30 con 0x7fdde81080c0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.387+0000 7fdddeffd640 1 -- 172.21.15.16:0/4187185325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fdde4045070 con 0x7fdde81080c0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.387+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde8108520 con 0x7fdde81080c0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.387+0000 7fdddeffd640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fddd40391b0 0x7fddd403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.387+0000 7fdddeffd640 1 -- 172.21.15.16:0/4187185325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdde406c410 con 0x7fdde81080c0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.387+0000 7fdded744640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fddd40391b0 0x7fddd403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.388+0000 7fdded744640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fddd40391b0 0x7fddd403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fddd80098c0 tx=0x7fddd8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.392+0000 7fdddeffd640 1 -- 172.21.15.16:0/4187185325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdde404b170 con 0x7fdde81080c0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.620+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdde8102e50 con 0x7fddd40391b0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.624+0000 7fdddeffd640 1 -- 172.21.15.16:0/4187185325 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdde8102e50 con 0x7fddd40391b0 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.626+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fddd40391b0 msgr2=0x7fddd403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.626+0000 7fddf01a9640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fddd40391b0 0x7fddd403b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fddd80098c0 tx=0x7fddd8006db0 comp rx=0 tx=0).stop 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.626+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 msgr2=0x7fdde819d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:28:59.197 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.626+0000 7fddf01a9640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde819d770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdde4032ce0 tx=0x7fdde4032d10 comp rx=0 tx=0).stop 2024-09-15T22:28:59.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.627+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 shutdown_connections 2024-09-15T22:28:59.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.627+0000 7fddf01a9640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fddd40391b0 0x7fddd403b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:59.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.627+0000 7fddf01a9640 1 --2- 172.21.15.16:0/4187185325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdde81080c0 0x7fdde819d770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:28:59.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.627+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 >> 172.21.15.16:0/4187185325 conn(0x7fdde8101c00 msgr2=0x7fdde8102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:28:59.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.627+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 shutdown_connections 2024-09-15T22:28:59.198 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:58.627+0000 7fddf01a9640 1 -- 172.21.15.16:0/4187185325 wait complete. 2024-09-15T22:28:59.198 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-09-15T22:28:59.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:59 smithi016 ceph-mon[31492]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:28:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:59 smithi016 ceph-mon[31492]: Saving service mon spec with placement count:5 2024-09-15T22:28:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:28:59 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:29:00.352 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.610+0000 7fbb8fa1b640 1 Processor -- start 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.611+0000 7fbb8fa1b640 1 -- start start 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.611+0000 7fbb8fa1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.611+0000 7fbb8fa1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8807c890 con 0x7fbb8807de60 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.611+0000 7fbb8d7b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.611+0000 7fbb8d7b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53314/0 (socket says 172.21.15.16:53314) 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.611+0000 7fbb8d7b7640 1 -- 172.21.15.16:0/3937050092 learned_addr learned my addr 172.21.15.16:0/3937050092 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.611+0000 7fbb8d7b7640 1 -- 172.21.15.16:0/3937050092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8807ca10 con 0x7fbb8807de60 2024-09-15T22:29:00.353 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.612+0000 7fbb8d7b7640 1 --2- 172.21.15.16:0/3937050092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8807c2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009820 tx=0x7fbb8402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a225969e9af358d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.612+0000 7fbb7ffff640 1 -- 172.21.15.16:0/3937050092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb84037070 con 0x7fbb8807de60 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.612+0000 7fbb7ffff640 1 -- 172.21.15.16:0/3937050092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb84032900 con 0x7fbb8807de60 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.612+0000 7fbb7ffff640 1 -- 172.21.15.16:0/3937050092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb84030850 con 0x7fbb8807de60 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.613+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/3937050092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 msgr2=0x7fbb8807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.613+0000 7fbb8fa1b640 1 --2- 172.21.15.16:0/3937050092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8807c2c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009820 tx=0x7fbb8402a270 comp rx=0 tx=0).stop 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.613+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/3937050092 shutdown_connections 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.613+0000 7fbb8fa1b640 1 --2- 172.21.15.16:0/3937050092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8807c2c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.613+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/3937050092 >> 172.21.15.16:0/3937050092 conn(0x7fbb88102cc0 msgr2=0x7fbb88105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.613+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/3937050092 shutdown_connections 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.613+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/3937050092 wait complete. 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.614+0000 7fbb8fa1b640 1 Processor -- start 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.614+0000 7fbb8fa1b640 1 -- start start 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.614+0000 7fbb8fa1b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.614+0000 7fbb8fa1b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8810db80 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.614+0000 7fbb8d7b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.614+0000 7fbb8d7b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8819d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53324/0 (socket says 172.21.15.16:53324) 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.614+0000 7fbb8d7b7640 1 -- 172.21.15.16:0/4240008800 learned_addr learned my addr 172.21.15.16:0/4240008800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.615+0000 7fbb8d7b7640 1 -- 172.21.15.16:0/4240008800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8819dcb0 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.615+0000 7fbb8d7b7640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8819d770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb840097f0 tx=0x7fbb84030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.615+0000 7fbb7e7fc640 1 -- 172.21.15.16:0/4240008800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb84037040 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.615+0000 7fbb7e7fc640 1 -- 172.21.15.16:0/4240008800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb84040070 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.615+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8819df40 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.615+0000 7fbb7e7fc640 1 -- 172.21.15.16:0/4240008800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8403a390 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.615+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb881a0c30 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.616+0000 7fbb7e7fc640 1 -- 172.21.15.16:0/4240008800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fbb8403a5e0 con 0x7fbb8807de60 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.616+0000 7fbb7e7fc640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fbb74039200 0x7fbb7403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.616+0000 7fbb7e7fc640 1 -- 172.21.15.16:0/4240008800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb8406c760 con 0x7fbb8807de60 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.616+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8807c340 con 0x7fbb8807de60 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.616+0000 7fbb8cfb6640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fbb74039200 0x7fbb7403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.617+0000 7fbb8cfb6640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fbb74039200 0x7fbb7403b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb780098c0 tx=0x7fbb78006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.621+0000 7fbb7e7fc640 1 -- 172.21.15.16:0/4240008800 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb8404a200 con 0x7fbb8807de60 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.811+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbb88103fb0 con 0x7fbb74039200 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.815+0000 7fbb7e7fc640 1 -- 172.21.15.16:0/4240008800 <== mgr.14118 v2:172.21.15.16:6800/2652317700 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbb88103fb0 con 0x7fbb74039200 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fbb74039200 msgr2=0x7fbb7403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fbb74039200 0x7fbb7403b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbb780098c0 tx=0x7fbb78006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 msgr2=0x7fbb8819d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8819d770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb840097f0 tx=0x7fbb84030d80 comp rx=0 tx=0).stop 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 shutdown_connections 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fbb74039200 0x7fbb7403b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.356 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 --2- 172.21.15.16:0/4240008800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb8807de60 0x7fbb8819d770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:00.357 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 >> 172.21.15.16:0/4240008800 conn(0x7fbb88102cc0 msgr2=0x7fbb881038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:00.357 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 shutdown_connections 2024-09-15T22:29:00.357 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:28:59.817+0000 7fbb8fa1b640 1 -- 172.21.15.16:0/4240008800 wait complete. 2024-09-15T22:29:01.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:00 smithi016 ceph-mon[31492]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:29:01.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:00 smithi016 ceph-mon[31492]: Saving service mgr spec with placement count:2 2024-09-15T22:29:01.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:00 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:29:02.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:01 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1329129148' entity='client.admin' 2024-09-15T22:29:02.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:01 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:29:02.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea8e934640 1 Processor -- start 2024-09-15T22:29:02.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea8e934640 1 -- start start 2024-09-15T22:29:02.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea8e934640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea88107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:02.273 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea8e934640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea88107f00 con 0x7fea88107530 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea88107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea88107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53340/0 (socket says 172.21.15.16:53340) 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea87fff640 1 -- 172.21.15.16:0/3062313733 learned_addr learned my addr 172.21.15.16:0/3062313733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.782+0000 7fea87fff640 1 -- 172.21.15.16:0/3062313733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea88108730 con 0x7fea88107530 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.783+0000 7fea87fff640 1 --2- 172.21.15.16:0/3062313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea88107930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea78009820 tx=0x7fea7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b1b0a38ac42158 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.783+0000 7fea86ffd640 1 -- 172.21.15.16:0/3062313733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78037070 con 0x7fea88107530 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.783+0000 7fea86ffd640 1 -- 172.21.15.16:0/3062313733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea78032900 con 0x7fea88107530 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.783+0000 7fea86ffd640 1 -- 172.21.15.16:0/3062313733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78030850 con 0x7fea88107530 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.784+0000 7fea8e934640 1 -- 172.21.15.16:0/3062313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 msgr2=0x7fea88107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.784+0000 7fea8e934640 1 --2- 172.21.15.16:0/3062313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea88107930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fea78009820 tx=0x7fea7802a270 comp rx=0 tx=0).stop 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.784+0000 7fea8e934640 1 -- 172.21.15.16:0/3062313733 shutdown_connections 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.784+0000 7fea8e934640 1 --2- 172.21.15.16:0/3062313733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea88107930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:02.274 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.784+0000 7fea8e934640 1 -- 172.21.15.16:0/3062313733 >> 172.21.15.16:0/3062313733 conn(0x7fea88102cc0 msgr2=0x7fea88105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.784+0000 7fea8e934640 1 -- 172.21.15.16:0/3062313733 shutdown_connections 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.784+0000 7fea8e934640 1 -- 172.21.15.16:0/3062313733 wait complete. 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea8e934640 1 Processor -- start 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea8e934640 1 -- start start 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea8e934640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea8819b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea8e934640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810bae0 con 0x7fea88107530 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea8819b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea8819b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53352/0 (socket says 172.21.15.16:53352) 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea87fff640 1 -- 172.21.15.16:0/1329129148 learned_addr learned my addr 172.21.15.16:0/1329129148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.785+0000 7fea87fff640 1 -- 172.21.15.16:0/1329129148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819bac0 con 0x7fea88107530 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.786+0000 7fea87fff640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea8819b580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea780097f0 tx=0x7fea78030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.786+0000 7fea857fa640 1 -- 172.21.15.16:0/1329129148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea78037040 con 0x7fea88107530 2024-09-15T22:29:02.275 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.786+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8819bd50 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.786+0000 7fea857fa640 1 -- 172.21.15.16:0/1329129148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea78040070 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.786+0000 7fea857fa640 1 -- 172.21.15.16:0/1329129148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fea7803a390 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.786+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8819ea40 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.787+0000 7fea857fa640 1 -- 172.21.15.16:0/1329129148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fea7803a650 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.787+0000 7fea857fa640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fea700391b0 0x7fea7003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.787+0000 7fea857fa640 1 -- 172.21.15.16:0/1329129148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fea7806c770 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.787+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8810bc60 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.788+0000 7fea877fe640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fea700391b0 0x7fea7003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.788+0000 7fea877fe640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fea700391b0 0x7fea7003b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098c0 tx=0x7fea7c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.795+0000 7fea857fa640 1 -- 172.21.15.16:0/1329129148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea78042e30 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.972+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fea881079b0 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.975+0000 7fea857fa640 1 -- 172.21.15.16:0/1329129148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fea78043760 con 0x7fea88107530 2024-09-15T22:29:02.276 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fea700391b0 msgr2=0x7fea7003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fea700391b0 0x7fea7003b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098c0 tx=0x7fea7c006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 msgr2=0x7fea8819b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea8819b580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea780097f0 tx=0x7fea78030d80 comp rx=0 tx=0).stop 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 shutdown_connections 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7fea700391b0 0x7fea7003b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 --2- 172.21.15.16:0/1329129148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea88107530 0x7fea8819b580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 >> 172.21.15.16:0/1329129148 conn(0x7fea88102cc0 msgr2=0x7fea881038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 shutdown_connections 2024-09-15T22:29:02.277 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:00.977+0000 7fea8e934640 1 -- 172.21.15.16:0/1329129148 wait complete. 2024-09-15T22:29:03.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:03 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:29:03.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:03 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/911702029' entity='client.admin' 2024-09-15T22:29:03.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.685+0000 7f8567ed4640 1 Processor -- start 2024-09-15T22:29:03.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.686+0000 7f8567ed4640 1 -- start start 2024-09-15T22:29:03.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.686+0000 7f8567ed4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856010a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.686+0000 7f8567ed4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8560075790 con 0x7f8560108060 2024-09-15T22:29:03.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.686+0000 7f8565c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856010a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.686+0000 7f8565c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856010a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42712/0 (socket says 172.21.15.16:42712) 2024-09-15T22:29:03.565 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.686+0000 7f8565c70640 1 -- 172.21.15.16:0/3697030425 learned_addr learned my addr 172.21.15.16:0/3697030425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.686+0000 7f8565c70640 1 -- 172.21.15.16:0/3697030425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856010a990 con 0x7f8560108060 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.687+0000 7f8565c70640 1 --2- 172.21.15.16:0/3697030425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856010a450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8550009e80 tx=0x7f855002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c1893a1d2bb85a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.687+0000 7f8564c6e640 1 -- 172.21.15.16:0/3697030425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8550036070 con 0x7f8560108060 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.687+0000 7f8564c6e640 1 -- 172.21.15.16:0/3697030425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85500318e0 con 0x7f8560108060 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.687+0000 7f8564c6e640 1 -- 172.21.15.16:0/3697030425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8550007a00 con 0x7f8560108060 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.688+0000 7f8567ed4640 1 -- 172.21.15.16:0/3697030425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 msgr2=0x7f856010a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.688+0000 7f8567ed4640 1 --2- 172.21.15.16:0/3697030425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856010a450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8550009e80 tx=0x7f855002a7a0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.688+0000 7f8567ed4640 1 -- 172.21.15.16:0/3697030425 shutdown_connections 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.688+0000 7f8567ed4640 1 --2- 172.21.15.16:0/3697030425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856010a450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.688+0000 7f8567ed4640 1 -- 172.21.15.16:0/3697030425 >> 172.21.15.16:0/3697030425 conn(0x7f8560101c00 msgr2=0x7f8560104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.688+0000 7f8567ed4640 1 -- 172.21.15.16:0/3697030425 shutdown_connections 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.688+0000 7f8567ed4640 1 -- 172.21.15.16:0/3697030425 wait complete. 2024-09-15T22:29:03.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.689+0000 7f8567ed4640 1 Processor -- start 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.689+0000 7f8567ed4640 1 -- start start 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.689+0000 7f8567ed4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.689+0000 7f8567ed4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f856010b9b0 con 0x7f8560108060 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.689+0000 7f8565c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.689+0000 7f8565c70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42718/0 (socket says 172.21.15.16:42718) 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.689+0000 7f8565c70640 1 -- 172.21.15.16:0/911702029 learned_addr learned my addr 172.21.15.16:0/911702029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.690+0000 7f8565c70640 1 -- 172.21.15.16:0/911702029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856019dca0 con 0x7f8560108060 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.690+0000 7f8565c70640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856019d760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8550031cf0 tx=0x7f8550031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.690+0000 7f855effd640 1 -- 172.21.15.16:0/911702029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8550036050 con 0x7f8560108060 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.690+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856019df30 con 0x7f8560108060 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.690+0000 7f855effd640 1 -- 172.21.15.16:0/911702029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8550042070 con 0x7f8560108060 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.690+0000 7f855effd640 1 -- 172.21.15.16:0/911702029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f855003d9b0 con 0x7f8560108060 2024-09-15T22:29:03.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.690+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85601a0c20 con 0x7f8560108060 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.691+0000 7f855effd640 1 -- 172.21.15.16:0/911702029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f855003dc60 con 0x7f8560108060 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.691+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85601084c0 con 0x7f8560108060 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.691+0000 7f855effd640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f85480391b0 0x7f854803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.691+0000 7f855effd640 1 -- 172.21.15.16:0/911702029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f855006d7f0 con 0x7f8560108060 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.691+0000 7f856546f640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f85480391b0 0x7f854803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.692+0000 7f856546f640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f85480391b0 0x7f854803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85540098c0 tx=0x7f8554006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.697+0000 7f855effd640 1 -- 172.21.15.16:0/911702029 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f855004c200 con 0x7f8560108060 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.884+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f85601a0df0 con 0x7f8560108060 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.887+0000 7f855effd640 1 -- 172.21.15.16:0/911702029 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f855003bcf0 con 0x7f8560108060 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f85480391b0 msgr2=0x7f854803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f85480391b0 0x7f854803b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f85540098c0 tx=0x7f8554006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 msgr2=0x7f856019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856019d760 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8550031cf0 tx=0x7f8550031d20 comp rx=0 tx=0).stop 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 shutdown_connections 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f85480391b0 0x7f854803b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 --2- 172.21.15.16:0/911702029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8560108060 0x7f856019d760 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 >> 172.21.15.16:0/911702029 conn(0x7f8560101c00 msgr2=0x7f8560102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 shutdown_connections 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:02.889+0000 7f8567ed4640 1 -- 172.21.15.16:0/911702029 wait complete. 2024-09-15T22:29:03.569 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-09-15T22:29:05.615 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:29:05.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1280466153' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-15T22:29:05.616 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-mon[31492]: from='mgr.14118 172.21.15.16:0/3772399005' entity='mgr.x' 2024-09-15T22:29:05.616 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ignoring --setuser ceph since I am not root 2024-09-15T22:29:05.616 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ignoring --setgroup ceph since I am not root 2024-09-15T22:29:05.616 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:05.512+0000 7f8257b5b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:29:05.617 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:05.571+0000 7f8257b5b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:29:05.920 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:05.731+0000 7f8257b5b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:29:05.920 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:05.800+0000 7f8257b5b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.284+0000 7f986587a640 1 Processor -- start 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.284+0000 7f986587a640 1 -- start start 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.285+0000 7f986587a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986010a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.285+0000 7f986587a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f986007be10 con 0x7f9860107fe0 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.285+0000 7f985effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986010a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.285+0000 7f985effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986010a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42730/0 (socket says 172.21.15.16:42730) 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.285+0000 7f985effd640 1 -- 172.21.15.16:0/3114178909 learned_addr learned my addr 172.21.15.16:0/3114178909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:06.175 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.285+0000 7f985effd640 1 -- 172.21.15.16:0/3114178909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986010a910 con 0x7f9860107fe0 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.286+0000 7f985effd640 1 --2- 172.21.15.16:0/3114178909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986010a3d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9850009e80 tx=0x7f985002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9291a4a0e7f7ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.286+0000 7f985dffb640 1 -- 172.21.15.16:0/3114178909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9850036070 con 0x7f9860107fe0 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.286+0000 7f985dffb640 1 -- 172.21.15.16:0/3114178909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98500318e0 con 0x7f9860107fe0 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.286+0000 7f985dffb640 1 -- 172.21.15.16:0/3114178909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9850007a00 con 0x7f9860107fe0 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.287+0000 7f986587a640 1 -- 172.21.15.16:0/3114178909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 msgr2=0x7f986010a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.287+0000 7f986587a640 1 --2- 172.21.15.16:0/3114178909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986010a3d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9850009e80 tx=0x7f985002a7a0 comp rx=0 tx=0).stop 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.287+0000 7f986587a640 1 -- 172.21.15.16:0/3114178909 shutdown_connections 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.287+0000 7f986587a640 1 --2- 172.21.15.16:0/3114178909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986010a3d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.287+0000 7f986587a640 1 -- 172.21.15.16:0/3114178909 >> 172.21.15.16:0/3114178909 conn(0x7f9860101bc0 msgr2=0x7f9860104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.287+0000 7f986587a640 1 -- 172.21.15.16:0/3114178909 shutdown_connections 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.287+0000 7f986587a640 1 -- 172.21.15.16:0/3114178909 wait complete. 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f986587a640 1 Processor -- start 2024-09-15T22:29:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f986587a640 1 -- start start 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f986587a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f986587a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f986010b990 con 0x7f9860107fe0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f985effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f985effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986019b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42746/0 (socket says 172.21.15.16:42746) 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f985effd640 1 -- 172.21.15.16:0/1280466153 learned_addr learned my addr 172.21.15.16:0/1280466153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.288+0000 7f985effd640 1 -- 172.21.15.16:0/1280466153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986019bb10 con 0x7f9860107fe0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.289+0000 7f985effd640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986019b5d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f985002acc0 tx=0x7f9850031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.289+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9850036050 con 0x7f9860107fe0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.289+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9850037040 con 0x7f9860107fe0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.289+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986019bda0 con 0x7f9860107fe0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.289+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f985003d970 con 0x7f9860107fe0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.289+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986019ea90 con 0x7f9860107fe0 2024-09-15T22:29:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.290+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9850044020 con 0x7f9860107fe0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.290+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9860108440 con 0x7f9860107fe0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.290+0000 7f9864878640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f984c039200 0x7f984c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.290+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9850042020 con 0x7f9860107fe0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.291+0000 7f985e7fc640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f984c039200 0x7f984c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.291+0000 7f985e7fc640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f984c039200 0x7f984c03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98480098c0 tx=0x7f9848006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.295+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f985004b020 con 0x7f9860107fe0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:04.526+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f986019f080 con 0x7f9860107fe0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.359+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f985003bc60 con 0x7f9860107fe0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.360+0000 7f9864878640 1 -- 172.21.15.16:0/1280466153 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f985003dcd0 con 0x7f9860107fe0 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f984c039200 msgr2=0x7f984c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f984c039200 0x7f984c03b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98480098c0 tx=0x7f9848006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 msgr2=0x7f986019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986019b5d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f985002acc0 tx=0x7f9850031cc0 comp rx=0 tx=0).stop 2024-09-15T22:29:06.178 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 shutdown_connections 2024-09-15T22:29:06.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f984c039200 0x7f984c03b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:06.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 --2- 172.21.15.16:0/1280466153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9860107fe0 0x7f986019b5d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:06.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 >> 172.21.15.16:0/1280466153 conn(0x7f9860101bc0 msgr2=0x7f9860102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:06.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 shutdown_connections 2024-09-15T22:29:06.179 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:05.362+0000 7f986587a640 1 -- 172.21.15.16:0/1280466153 wait complete. 2024-09-15T22:29:06.295 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:05.957+0000 7f8257b5b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:29:06.295 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:06.104+0000 7f8257b5b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:29:06.547 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:06 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1280466153' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-15T22:29:06.548 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:06 smithi016 ceph-mon[31492]: mgrmap e9: x(active, since 18s) 2024-09-15T22:29:06.548 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:06.444+0000 7f8257b5b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:29:06.548 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:06.509+0000 7f8257b5b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:29:06.798 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:06.714+0000 7f8257b5b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:29:07.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:07 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2069818666' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T22:29:07.512 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.553+0000 7f960a915640 1 Processor -- start 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.554+0000 7f960a915640 1 -- start start 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.554+0000 7f960a915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f9604106250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.554+0000 7f960a915640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9604075790 con 0x7f9604105e50 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.554+0000 7f9603fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f9604106250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.554+0000 7f9603fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f9604106250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42758/0 (socket says 172.21.15.16:42758) 2024-09-15T22:29:07.513 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.554+0000 7f9603fff640 1 -- 172.21.15.16:0/4127640294 learned_addr learned my addr 172.21.15.16:0/4127640294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.554+0000 7f9603fff640 1 -- 172.21.15.16:0/4127640294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9604106790 con 0x7f9604105e50 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.555+0000 7f9603fff640 1 --2- 172.21.15.16:0/4127640294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f9604106250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009820 tx=0x7f95f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e9febf11011412 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.555+0000 7f9602ffd640 1 -- 172.21.15.16:0/4127640294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f4037070 con 0x7f9604105e50 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.555+0000 7f9602ffd640 1 -- 172.21.15.16:0/4127640294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f4032920 con 0x7f9604105e50 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.555+0000 7f9602ffd640 1 -- 172.21.15.16:0/4127640294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f4030810 con 0x7f9604105e50 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.556+0000 7f960a915640 1 -- 172.21.15.16:0/4127640294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 msgr2=0x7f9604106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.556+0000 7f960a915640 1 --2- 172.21.15.16:0/4127640294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f9604106250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009820 tx=0x7f95f402a270 comp rx=0 tx=0).stop 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.556+0000 7f960a915640 1 -- 172.21.15.16:0/4127640294 shutdown_connections 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.556+0000 7f960a915640 1 --2- 172.21.15.16:0/4127640294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f9604106250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.556+0000 7f960a915640 1 -- 172.21.15.16:0/4127640294 >> 172.21.15.16:0/4127640294 conn(0x7f9604101c00 msgr2=0x7f9604104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.556+0000 7f960a915640 1 -- 172.21.15.16:0/4127640294 shutdown_connections 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.556+0000 7f960a915640 1 -- 172.21.15.16:0/4127640294 wait complete. 2024-09-15T22:29:07.514 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f960a915640 1 Processor -- start 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f960a915640 1 -- start start 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f960a915640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f960419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f960a915640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96041078d0 con 0x7f9604105e50 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f9603fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f960419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f9603fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f960419d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42768/0 (socket says 172.21.15.16:42768) 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f9603fff640 1 -- 172.21.15.16:0/2069818666 learned_addr learned my addr 172.21.15.16:0/2069818666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.557+0000 7f9603fff640 1 -- 172.21.15.16:0/2069818666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960419dc90 con 0x7f9604105e50 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.558+0000 7f9603fff640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f960419d750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95f4030380 tx=0x7f95f4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.558+0000 7f96017fa640 1 -- 172.21.15.16:0/2069818666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f4037040 con 0x7f9604105e50 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.558+0000 7f96017fa640 1 -- 172.21.15.16:0/2069818666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f4040070 con 0x7f9604105e50 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.558+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960419df20 con 0x7f9604105e50 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.558+0000 7f96017fa640 1 -- 172.21.15.16:0/2069818666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f403a7b0 con 0x7f9604105e50 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.558+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96041a0c10 con 0x7f9604105e50 2024-09-15T22:29:07.515 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.559+0000 7f96017fa640 1 -- 172.21.15.16:0/2069818666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f95f403aa30 con 0x7f9604105e50 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.559+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9604106250 con 0x7f9604105e50 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.559+0000 7f96017fa640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.559+0000 7f96017fa640 1 -- 172.21.15.16:0/2069818666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f95f406c930 con 0x7f9604105e50 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.559+0000 7f96037fe640 1 -- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2652317700 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.559+0000 7f96037fe640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.568+0000 7f96017fa640 1 -- 172.21.15.16:0/2069818666 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95f4041270 con 0x7f9604105e50 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.760+0000 7f96037fe640 1 -- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2652317700 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.760+0000 7f96037fe640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.795+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f960402d9a0 con 0x7f9604105e50 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.796+0000 7f96017fa640 1 -- 172.21.15.16:0/2069818666 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f95f403ae70 con 0x7f9604105e50 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 msgr2=0x7f95ec03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:29:07.516 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 msgr2=0x7f960419d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f960419d750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95f4030380 tx=0x7f95f4032670 comp rx=0 tx=0).stop 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 shutdown_connections 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f95ec039200 0x7f95ec03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 --2- 172.21.15.16:0/2069818666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9604105e50 0x7f960419d750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 >> 172.21.15.16:0/2069818666 conn(0x7f9604101c00 msgr2=0x7f96041943f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 shutdown_connections 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:06.798+0000 7f960a915640 1 -- 172.21.15.16:0/2069818666 wait complete. 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-15T22:29:07.517 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-09-15T22:29:07.763 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:07.660+0000 7f8257b5b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:29:07.763 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:07.731+0000 7f8257b5b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:29:08.629 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:08.506+0000 7f8257b5b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:29:08.629 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:08.627+0000 7f8257b5b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:29:09.003 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:08.779+0000 7f8257b5b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:29:09.003 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:08.843+0000 7f8257b5b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:29:09.003 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.001+0000 7f8257b5b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:29:09.305 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.074+0000 7f8257b5b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:29:09.305 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.137+0000 7f8257b5b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:29:09.305 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.303+0000 7f8257b5b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:29:09.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.371+0000 7f8257b5b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:29:09.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.441+0000 7f8257b5b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:29:09.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.527+0000 7f8257b5b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:29:10.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.590+0000 7f8257b5b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:29:10.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:09.651+0000 7f8257b5b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:29:10.734 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:10.481+0000 7f8257b5b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: Active manager daemon x restarted 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: Activating manager daemon x 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: osdmap e3: 0 total, 0 up, 0 in 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: mgrmap e10: x(active, starting, since 0.00343397s) 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:29:10.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:29:10.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: Manager daemon x is now available 2024-09-15T22:29:10.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:10.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:29:10.736 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.937+0000 7f8adee46640 1 Processor -- start 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.937+0000 7f8adee46640 1 -- start start 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.937+0000 7f8adee46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad810a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:12.376 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.937+0000 7f8adee46640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad80757d0 con 0x7f8ad8108020 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.937+0000 7f8adcbe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad810a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.937+0000 7f8adcbe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad810a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42770/0 (socket says 172.21.15.16:42770) 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.937+0000 7f8adcbe2640 1 -- 172.21.15.16:0/179508068 learned_addr learned my addr 172.21.15.16:0/179508068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.938+0000 7f8adcbe2640 1 -- 172.21.15.16:0/179508068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad810a950 con 0x7f8ad8108020 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.938+0000 7f8adcbe2640 1 --2- 172.21.15.16:0/179508068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad810a410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009820 tx=0x7f8acc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b393a6ac76fb38c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.939+0000 7f8ad77fe640 1 -- 172.21.15.16:0/179508068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8acc037070 con 0x7f8ad8108020 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.939+0000 7f8ad77fe640 1 -- 172.21.15.16:0/179508068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8acc032920 con 0x7f8ad8108020 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.939+0000 7f8ad77fe640 1 -- 172.21.15.16:0/179508068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8acc030810 con 0x7f8ad8108020 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.939+0000 7f8adee46640 1 -- 172.21.15.16:0/179508068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 msgr2=0x7f8ad810a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.939+0000 7f8adee46640 1 --2- 172.21.15.16:0/179508068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad810a410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009820 tx=0x7f8acc02a270 comp rx=0 tx=0).stop 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.940+0000 7f8adee46640 1 -- 172.21.15.16:0/179508068 shutdown_connections 2024-09-15T22:29:12.377 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.940+0000 7f8adee46640 1 --2- 172.21.15.16:0/179508068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad810a410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.940+0000 7f8adee46640 1 -- 172.21.15.16:0/179508068 >> 172.21.15.16:0/179508068 conn(0x7f8ad8101c80 msgr2=0x7f8ad81040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.940+0000 7f8adee46640 1 -- 172.21.15.16:0/179508068 shutdown_connections 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.940+0000 7f8adee46640 1 -- 172.21.15.16:0/179508068 wait complete. 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.940+0000 7f8adee46640 1 Processor -- start 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.940+0000 7f8adee46640 1 -- start start 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8adee46640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8adee46640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad810b9d0 con 0x7f8ad8108020 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8adcbe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8adcbe2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42786/0 (socket says 172.21.15.16:42786) 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8adcbe2640 1 -- 172.21.15.16:0/1722588148 learned_addr learned my addr 172.21.15.16:0/1722588148 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8adcbe2640 1 -- 172.21.15.16:0/1722588148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad819bb00 con 0x7f8ad8108020 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8adcbe2640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad819b5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8acc030420 tx=0x7f8acc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:12.378 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.941+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8acc037040 con 0x7f8ad8108020 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.942+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8acc040070 con 0x7f8ad8108020 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.942+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad819bd90 con 0x7f8ad8108020 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.942+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8acc03a6e0 con 0x7f8ad8108020 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.942+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad819ea80 con 0x7f8ad8108020 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.943+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f8acc03a9e0 con 0x7f8ad8108020 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.943+0000 7f8ad5ffb640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 0x7f8ac803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.943+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 --> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8ac803bdd0 con 0x7f8ac8039200 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.943+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8acc06c940 con 0x7f8ad8108020 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.943+0000 7f8ad7fff640 1 -- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 msgr2=0x7f8ac803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2652317700 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:07.943+0000 7f8ad7fff640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 0x7f8ac803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:08.144+0000 7f8ad7fff640 1 -- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 msgr2=0x7f8ac803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2652317700 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:08.144+0000 7f8ad7fff640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 0x7f8ac803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:29:12.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:08.545+0000 7f8ad7fff640 1 -- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 msgr2=0x7f8ac803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2652317700 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:08.545+0000 7f8ad7fff640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 0x7f8ac803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:09.346+0000 7f8ad7fff640 1 -- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 msgr2=0x7f8ac803b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/2652317700 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:09.346+0000 7f8ad7fff640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 0x7f8ac803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:10.484+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f8acc06b3f0 con 0x7f8ad8108020 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:10.485+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 msgr2=0x7f8ac803b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:10.485+0000 7f8ad5ffb640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 0x7f8ac803b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.486+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f8acc045460 con 0x7f8ad8108020 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.486+0000 7f8ad5ffb640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8ac803ccc0 0x7f8ac803f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.487+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8acc006810 con 0x7f8ac803ccc0 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.487+0000 7f8ad7fff640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8ac803ccc0 0x7f8ac803f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.488+0000 7f8ad7fff640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8ac803ccc0 0x7f8ac803f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4003980 tx=0x7f8ac4006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:12.380 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.488+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8acc006810 con 0x7f8ac803ccc0 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.495+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8ad8108480 con 0x7f8ac803ccc0 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8ad5ffb640 1 -- 172.21.15.16:0/1722588148 <== mgr.14150 v2:172.21.15.16:6800/81475521 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f8ad8108480 con 0x7f8ac803ccc0 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8ac803ccc0 msgr2=0x7f8ac803f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8ac803ccc0 0x7f8ac803f0b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4003980 tx=0x7f8ac4006e10 comp rx=0 tx=0).stop 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 msgr2=0x7f8ad819b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad819b5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8acc030420 tx=0x7f8acc032670 comp rx=0 tx=0).stop 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 shutdown_connections 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8ac803ccc0 0x7f8ac803f0b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:6800/2652317700,v1:172.21.15.16:6801/2652317700] conn(0x7f8ac8039200 0x7f8ac803b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 --2- 172.21.15.16:0/1722588148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ad8108020 0x7f8ad819b5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 >> 172.21.15.16:0/1722588148 conn(0x7f8ad8101c80 msgr2=0x7f8ad8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 shutdown_connections 2024-09-15T22:29:12.381 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:11.496+0000 7f8adee46640 1 -- 172.21.15.16:0/1722588148 wait complete. 2024-09-15T22:29:12.382 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-09-15T22:29:12.382 INFO:teuthology.orchestra.run.smithi016.stdout:Generating a dashboard self-signed certificate... 2024-09-15T22:29:12.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:12 smithi016 ceph-mon[31492]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T22:29:12.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:12 smithi016 ceph-mon[31492]: mgrmap e11: x(active, since 1.00614s) 2024-09-15T22:29:12.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:12 smithi016 ceph-mon[31492]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T22:29:12.740 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:13.690 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:13.415+0000 7f81c2caf640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:29:13] ENGINE Error in HTTPServer.serve 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: Traceback (most recent call last): 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._connections.run(self.expiration_interval) 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._run(expiration_interval) 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: s = self.context.wrap_socket( 2024-09-15T22:29:13.691 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:29:13.692 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: return self.sslsocket_class._create( 2024-09-15T22:29:13.692 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:29:13.692 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self.do_handshake() 2024-09-15T22:29:13.692 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:29:13.692 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._sslobj.do_handshake() 2024-09-15T22:29:13.692 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:29:13.692 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:29:13.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-15T22:29:13.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.822+0000 7fb90116d640 1 Processor -- start 2024-09-15T22:29:13.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.822+0000 7fb90116d640 1 -- start start 2024-09-15T22:29:13.699 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.822+0000 7fb90116d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc10a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.822+0000 7fb90116d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc075810 con 0x7fb8fc107fe0 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.823+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc10a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.823+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc10a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60204/0 (socket says 172.21.15.16:60204) 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.823+0000 7fb8fad9d640 1 -- 172.21.15.16:0/2647953300 learned_addr learned my addr 172.21.15.16:0/2647953300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.823+0000 7fb8fad9d640 1 -- 172.21.15.16:0/2647953300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc10a910 con 0x7fb8fc107fe0 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.823+0000 7fb8fad9d640 1 --2- 172.21.15.16:0/2647953300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc10a3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009e80 tx=0x7fb8f402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9fb314d4d146e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb8f9d9b640 1 -- 172.21.15.16:0/2647953300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8f4036070 con 0x7fb8fc107fe0 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb8f9d9b640 1 -- 172.21.15.16:0/2647953300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f40318e0 con 0x7fb8fc107fe0 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb8f9d9b640 1 -- 172.21.15.16:0/2647953300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8f4007a00 con 0x7fb8fc107fe0 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb90116d640 1 -- 172.21.15.16:0/2647953300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 msgr2=0x7fb8fc10a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb90116d640 1 --2- 172.21.15.16:0/2647953300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc10a3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4009e80 tx=0x7fb8f402a7a0 comp rx=0 tx=0).stop 2024-09-15T22:29:13.700 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb90116d640 1 -- 172.21.15.16:0/2647953300 shutdown_connections 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb90116d640 1 --2- 172.21.15.16:0/2647953300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc10a3d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb90116d640 1 -- 172.21.15.16:0/2647953300 >> 172.21.15.16:0/2647953300 conn(0x7fb8fc101c00 msgr2=0x7fb8fc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb90116d640 1 -- 172.21.15.16:0/2647953300 shutdown_connections 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.824+0000 7fb90116d640 1 -- 172.21.15.16:0/2647953300 wait complete. 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.825+0000 7fb90116d640 1 Processor -- start 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.825+0000 7fb90116d640 1 -- start start 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.825+0000 7fb90116d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.825+0000 7fb90116d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc10b9f0 con 0x7fb8fc107fe0 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.825+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb8fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc19d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60210/0 (socket says 172.21.15.16:60210) 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb8fad9d640 1 -- 172.21.15.16:0/957623928 learned_addr learned my addr 172.21.15.16:0/957623928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:13.701 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb8fad9d640 1 -- 172.21.15.16:0/957623928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc19dcd0 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb8fad9d640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc19d790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4031cf0 tx=0x7fb8f4031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8f4036050 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc19df60 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8f4042070 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8f403d9b0 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.826+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc1a0c50 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.827+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7fb8f403dc60 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.827+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc108440 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.827+0000 7fb8f3fff640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8e80390f0 0x7fb8e803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.827+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb8f406d5d0 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.827+0000 7fb8fa59c640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8e80390f0 0x7fb8e803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.828+0000 7fb8fa59c640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8e80390f0 0x7fb8e803b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098c0 tx=0x7fb8e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.832+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8f4044310 con 0x7fb8fc107fe0 2024-09-15T22:29:13.702 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:12.982+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb8f4039300 con 0x7fb8fc107fe0 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.026+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb8fc102e40 con 0x7fb8e80390f0 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.083+0000 7fb8f3fff640 1 -- 172.21.15.16:0/957623928 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb8fc102e40 con 0x7fb8e80390f0 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8e80390f0 msgr2=0x7fb8e803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8e80390f0 0x7fb8e803b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40098c0 tx=0x7fb8e4006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 msgr2=0x7fb8fc19d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc19d790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4031cf0 tx=0x7fb8f4031d20 comp rx=0 tx=0).stop 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 shutdown_connections 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8e80390f0 0x7fb8e803b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 --2- 172.21.15.16:0/957623928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8fc107fe0 0x7fb8fc19d790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 >> 172.21.15.16:0/957623928 conn(0x7fb8fc101c00 msgr2=0x7fb8fc102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.085+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 shutdown_connections 2024-09-15T22:29:13.703 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:13.086+0000 7fb90116d640 1 -- 172.21.15.16:0/957623928 wait complete. 2024-09-15T22:29:13.704 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-mon[31492]: mgrmap e12: x(active, since 2s) 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-mon[31492]: [15/Sep/2024:22:29:13] ENGINE Bus STARTING 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-mon[31492]: [15/Sep/2024:22:29:13] ENGINE Serving on https://172.21.15.16:7150 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:13 smithi016 ceph-mon[31492]: [15/Sep/2024:22:29:13] ENGINE Error in HTTPServer.serve 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:29:14.018 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-15T22:29:14.019 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:29:14.020 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-15T22:29:14.020 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:29:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:14 smithi016 ceph-mon[31492]: [15/Sep/2024:22:29:13] ENGINE Serving on http://172.21.15.16:8765 2024-09-15T22:29:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:14 smithi016 ceph-mon[31492]: [15/Sep/2024:22:29:13] ENGINE Bus STARTED 2024-09-15T22:29:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:15.685 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$a6EHVRWwsCXhWtP9F6G8x.Sm7DBj/5nR/LblaiwhC7WEyiV4SxXVe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726439354, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.294+0000 7f5eeaf2a640 1 Processor -- start 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.294+0000 7f5eeaf2a640 1 -- start start 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.294+0000 7f5eeaf2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee410a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.294+0000 7f5eeaf2a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee40757d0 con 0x7f5ee4108020 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.294+0000 7f5ee8cc6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee410a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.294+0000 7f5ee8cc6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee410a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60222/0 (socket says 172.21.15.16:60222) 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.294+0000 7f5ee8cc6640 1 -- 172.21.15.16:0/1545550402 learned_addr learned my addr 172.21.15.16:0/1545550402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.295+0000 7f5ee8cc6640 1 -- 172.21.15.16:0/1545550402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee410a950 con 0x7f5ee4108020 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.295+0000 7f5ee8cc6640 1 --2- 172.21.15.16:0/1545550402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee410a410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009820 tx=0x7f5ed802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a9d83ea2c136e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5ee37fe640 1 -- 172.21.15.16:0/1545550402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed8037070 con 0x7f5ee4108020 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5ee37fe640 1 -- 172.21.15.16:0/1545550402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ed8032920 con 0x7f5ee4108020 2024-09-15T22:29:15.686 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5ee37fe640 1 -- 172.21.15.16:0/1545550402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed8030810 con 0x7f5ee4108020 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/1545550402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 msgr2=0x7f5ee410a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5eeaf2a640 1 --2- 172.21.15.16:0/1545550402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee410a410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009820 tx=0x7f5ed802a270 comp rx=0 tx=0).stop 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/1545550402 shutdown_connections 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5eeaf2a640 1 --2- 172.21.15.16:0/1545550402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee410a410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/1545550402 >> 172.21.15.16:0/1545550402 conn(0x7f5ee4101c00 msgr2=0x7f5ee4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/1545550402 shutdown_connections 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.296+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/1545550402 wait complete. 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5eeaf2a640 1 Processor -- start 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5eeaf2a640 1 -- start start 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5eeaf2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee41078a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5eeaf2a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee410b9d0 con 0x7f5ee4108020 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5ee8cc6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee41078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5ee8cc6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee41078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60228/0 (socket says 172.21.15.16:60228) 2024-09-15T22:29:15.687 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5ee8cc6640 1 -- 172.21.15.16:0/392207554 learned_addr learned my addr 172.21.15.16:0/392207554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.297+0000 7f5ee8cc6640 1 -- 172.21.15.16:0/392207554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee4107de0 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.298+0000 7f5ee8cc6640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee41078a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80097f0 tx=0x7f5ed80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.298+0000 7f5ee1ffb640 1 -- 172.21.15.16:0/392207554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed8037040 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.298+0000 7f5ee1ffb640 1 -- 172.21.15.16:0/392207554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ed8030cc0 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.298+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee4105fd0 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.298+0000 7f5ee1ffb640 1 -- 172.21.15.16:0/392207554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ed803a6b0 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.298+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee4106510 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.299+0000 7f5ee1ffb640 1 -- 172.21.15.16:0/392207554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f5ed803a8a0 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.299+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee4108480 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.299+0000 7f5ee1ffb640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ed4039200 0x7f5ed403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.299+0000 7f5ee1ffb640 1 -- 172.21.15.16:0/392207554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5ed806c900 con 0x7f5ee4108020 2024-09-15T22:29:15.688 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.299+0000 7f5ee3fff640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ed4039200 0x7f5ed403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.300+0000 7f5ee3fff640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ed4039200 0x7f5ed403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00098c0 tx=0x7f5ed0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.304+0000 7f5ee1ffb640 1 -- 172.21.15.16:0/392207554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ed803ace0 con 0x7f5ee4108020 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.498+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f5ee4106790 con 0x7f5ed4039200 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.747+0000 7f5ee1ffb640 1 -- 172.21.15.16:0/392207554 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f5ee4106790 con 0x7f5ed4039200 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ed4039200 msgr2=0x7f5ed403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ed4039200 0x7f5ed403b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00098c0 tx=0x7f5ed0006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 msgr2=0x7f5ee41078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee41078a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80097f0 tx=0x7f5ed80327b0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 shutdown_connections 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ed4039200 0x7f5ed403b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 --2- 172.21.15.16:0/392207554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ee4108020 0x7f5ee41078a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.749+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 >> 172.21.15.16:0/392207554 conn(0x7f5ee4101c00 msgr2=0x7f5ee4102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.750+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 shutdown_connections 2024-09-15T22:29:15.689 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:14.750+0000 7f5eeaf2a640 1 -- 172.21.15.16:0/392207554 wait complete. 2024-09-15T22:29:15.690 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-09-15T22:29:16.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:15 smithi016 ceph-mon[31492]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:29:16.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-09-15T22:29:16.854 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6ea1039640 1 Processor -- start 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6ea1039640 1 -- start start 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6ea1039640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6ea1039640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c107f00 con 0x7f6e9c107530 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6e9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6e9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60234/0 (socket says 172.21.15.16:60234) 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6e9ad9d640 1 -- 172.21.15.16:0/1565743134 learned_addr learned my addr 172.21.15.16:0/1565743134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.094+0000 7f6e9ad9d640 1 -- 172.21.15.16:0/1565743134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c108730 con 0x7f6e9c107530 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.095+0000 7f6e9ad9d640 1 --2- 172.21.15.16:0/1565743134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c107930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009820 tx=0x7f6e8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58107229df58f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.095+0000 7f6e9a59c640 1 -- 172.21.15.16:0/1565743134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e8c037070 con 0x7f6e9c107530 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.095+0000 7f6e9a59c640 1 -- 172.21.15.16:0/1565743134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8c032920 con 0x7f6e9c107530 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.095+0000 7f6e9a59c640 1 -- 172.21.15.16:0/1565743134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e8c030810 con 0x7f6e9c107530 2024-09-15T22:29:16.855 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.096+0000 7f6ea1039640 1 -- 172.21.15.16:0/1565743134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 msgr2=0x7f6e9c107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.096+0000 7f6ea1039640 1 --2- 172.21.15.16:0/1565743134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c107930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009820 tx=0x7f6e8c02a270 comp rx=0 tx=0).stop 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.096+0000 7f6ea1039640 1 -- 172.21.15.16:0/1565743134 shutdown_connections 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.096+0000 7f6ea1039640 1 --2- 172.21.15.16:0/1565743134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c107930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.096+0000 7f6ea1039640 1 -- 172.21.15.16:0/1565743134 >> 172.21.15.16:0/1565743134 conn(0x7f6e9c102cc0 msgr2=0x7f6e9c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.096+0000 7f6ea1039640 1 -- 172.21.15.16:0/1565743134 shutdown_connections 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.096+0000 7f6ea1039640 1 -- 172.21.15.16:0/1565743134 wait complete. 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6ea1039640 1 Processor -- start 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6ea1039640 1 -- start start 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6ea1039640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6ea1039640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9c10bae0 con 0x7f6e9c107530 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6e9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c19b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:16.856 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6e9ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c19b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60246/0 (socket says 172.21.15.16:60246) 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6e9ad9d640 1 -- 172.21.15.16:0/3714116870 learned_addr learned my addr 172.21.15.16:0/3714116870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.097+0000 7f6e9ad9d640 1 -- 172.21.15.16:0/3714116870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c19bae0 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.098+0000 7f6e9ad9d640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c19b5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c030560 tx=0x7f6e8c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.098+0000 7f6e98d99640 1 -- 172.21.15.16:0/3714116870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e8c037040 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.098+0000 7f6e98d99640 1 -- 172.21.15.16:0/3714116870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8c040070 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.098+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9c19bd70 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.098+0000 7f6e98d99640 1 -- 172.21.15.16:0/3714116870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e8c03a740 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.098+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e9c19ea60 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.099+0000 7f6e98d99640 1 -- 172.21.15.16:0/3714116870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f6e8c03a8e0 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.099+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9c10bc60 con 0x7f6e9c107530 2024-09-15T22:29:16.857 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.099+0000 7f6e98d99640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6e88039260 0x7f6e8803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.099+0000 7f6e98d99640 1 -- 172.21.15.16:0/3714116870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6e8c06c930 con 0x7f6e9c107530 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.099+0000 7f6e93fff640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6e88039260 0x7f6e8803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.099+0000 7f6e93fff640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6e88039260 0x7f6e8803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e840098c0 tx=0x7f6e84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.104+0000 7f6e98d99640 1 -- 172.21.15.16:0/3714116870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e8c043360 con 0x7f6e9c107530 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.278+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f6e9c19ec30 con 0x7f6e9c107530 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.279+0000 7f6e98d99640 1 -- 172.21.15.16:0/3714116870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6e8c040220 con 0x7f6e9c107530 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6e88039260 msgr2=0x7f6e8803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6e88039260 0x7f6e8803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e840098c0 tx=0x7f6e84006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 msgr2=0x7f6e9c19b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c19b5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c030560 tx=0x7f6e8c032670 comp rx=0 tx=0).stop 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 shutdown_connections 2024-09-15T22:29:16.858 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6e88039260 0x7f6e8803b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:16.859 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 --2- 172.21.15.16:0/3714116870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e9c107530 0x7f6e9c19b5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:16.859 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 >> 172.21.15.16:0/3714116870 conn(0x7f6e9c102cc0 msgr2=0x7f6e9c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:16.859 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 shutdown_connections 2024-09-15T22:29:16.859 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:16.281+0000 7f6ea1039640 1 -- 172.21.15.16:0/3714116870 wait complete. 2024-09-15T22:29:16.859 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:29:16.859 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-15T22:29:16.866 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:29:16.866 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-15T22:29:16.866 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-15T22:29:16.866 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-15T22:29:16.868 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-09-15T22:29:16.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:16.868 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-09-15T22:29:16.868 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-09-15T22:29:16.868 INFO:teuthology.orchestra.run.smithi016.stdout: Password: sxb7ixq5gn 2024-09-15T22:29:16.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:16.868 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config directory 2024-09-15T22:29:17.196 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:16 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3714116870' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-15T22:29:18.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:17 smithi016 ceph-mon[31492]: mgrmap e13: x(active, since 6s) 2024-09-15T22:29:18.116 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:17 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2839326012' entity='client.admin' 2024-09-15T22:29:18.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.278+0000 7f7622b47640 1 Processor -- start 2024-09-15T22:29:18.124 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.278+0000 7f7622b47640 1 -- start start 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.278+0000 7f7622b47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.278+0000 7f7622b47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f761c107f00 con 0x7f761c107530 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.278+0000 7f76208e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.279+0000 7f76208e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60262/0 (socket says 172.21.15.16:60262) 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.279+0000 7f76208e3640 1 -- 172.21.15.16:0/918531651 learned_addr learned my addr 172.21.15.16:0/918531651 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.279+0000 7f76208e3640 1 -- 172.21.15.16:0/918531651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c108730 con 0x7f761c107530 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.279+0000 7f76208e3640 1 --2- 172.21.15.16:0/918531651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c107930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7610009820 tx=0x7f761002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb23035a82d914ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f761b7fe640 1 -- 172.21.15.16:0/918531651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610037070 con 0x7f761c107530 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f761b7fe640 1 -- 172.21.15.16:0/918531651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610032920 con 0x7f761c107530 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f761b7fe640 1 -- 172.21.15.16:0/918531651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610030810 con 0x7f761c107530 2024-09-15T22:29:18.125 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f7622b47640 1 -- 172.21.15.16:0/918531651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 msgr2=0x7f761c107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f7622b47640 1 --2- 172.21.15.16:0/918531651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c107930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7610009820 tx=0x7f761002a270 comp rx=0 tx=0).stop 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f7622b47640 1 -- 172.21.15.16:0/918531651 shutdown_connections 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f7622b47640 1 --2- 172.21.15.16:0/918531651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c107930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f7622b47640 1 -- 172.21.15.16:0/918531651 >> 172.21.15.16:0/918531651 conn(0x7f761c102cc0 msgr2=0x7f761c105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f7622b47640 1 -- 172.21.15.16:0/918531651 shutdown_connections 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.280+0000 7f7622b47640 1 -- 172.21.15.16:0/918531651 wait complete. 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.281+0000 7f7622b47640 1 Processor -- start 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.281+0000 7f7622b47640 1 -- start start 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.281+0000 7f7622b47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c07db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.281+0000 7f7622b47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f761c10bae0 con 0x7f761c107530 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.281+0000 7f76208e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c07db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.281+0000 7f76208e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c07db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60276/0 (socket says 172.21.15.16:60276) 2024-09-15T22:29:18.126 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.281+0000 7f76208e3640 1 -- 172.21.15.16:0/2839326012 learned_addr learned my addr 172.21.15.16:0/2839326012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.282+0000 7f76208e3640 1 -- 172.21.15.16:0/2839326012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c07e050 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.282+0000 7f76208e3640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c07db10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76100097f0 tx=0x7f76100327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.282+0000 7f7619ffb640 1 -- 172.21.15.16:0/2839326012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7610037040 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.282+0000 7f7619ffb640 1 -- 172.21.15.16:0/2839326012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7610030cc0 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.282+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c07c240 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.282+0000 7f7619ffb640 1 -- 172.21.15.16:0/2839326012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f761003a640 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.282+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c07c780 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.283+0000 7f7619ffb640 1 -- 172.21.15.16:0/2839326012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f761003a8f0 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.283+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c1079b0 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.283+0000 7f7619ffb640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f760c039300 0x7f760c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.283+0000 7f7619ffb640 1 -- 172.21.15.16:0/2839326012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f761006c960 con 0x7f761c107530 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.283+0000 7f761bfff640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f760c039300 0x7f760c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:18.127 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.284+0000 7f761bfff640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f760c039300 0x7f760c03b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76080098c0 tx=0x7f7608006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.288+0000 7f7619ffb640 1 -- 172.21.15.16:0/2839326012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76100371e0 con 0x7f761c107530 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.556+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f761c07ca00 con 0x7f761c107530 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.559+0000 7f7619ffb640 1 -- 172.21.15.16:0/2839326012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7610031c30 con 0x7f761c107530 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f760c039300 msgr2=0x7f760c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f760c039300 0x7f760c03b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f76080098c0 tx=0x7f7608006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 msgr2=0x7f761c07db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c07db10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76100097f0 tx=0x7f76100327b0 comp rx=0 tx=0).stop 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 shutdown_connections 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f760c039300 0x7f760c03b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 --2- 172.21.15.16:0/2839326012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f761c107530 0x7f761c07db10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 >> 172.21.15.16:0/2839326012 conn(0x7f761c102cc0 msgr2=0x7f761c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:18.128 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 shutdown_connections 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr 2024-09-15T22:29:17.561+0000 7f7622b47640 1 -- 172.21.15.16:0/2839326012 wait complete. 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-09-15T22:29:18.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.130 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-15T22:29:18.130 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:18.130 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-09-15T22:29:18.168 INFO:tasks.cephadm:Fetching config... 2024-09-15T22:29:18.168 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:29:18.168 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-15T22:29:18.185 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-15T22:29:18.186 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:29:18.186 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-15T22:29:18.244 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-15T22:29:18.244 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:29:18.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-15T22:29:18.311 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-15T22:29:18.311 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:29:18.311 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-15T22:29:18.368 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-15T22:29:18.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5Y+b/spprJuGE0Zka3RAdjSAxMu8rL8Ao3Zsr+NNIElFU2MTG/jXmqCgtKFKZjsMhDtXhRY7kRofSCi/+CEi8MavsJs8qpYX/dPAybC5hirGXrqe+bTVIVKhdxd6WjF0JNdeyCKFm0hte/myrLU1eHXXEGUrhp53RNvLKLPrFlfaPIl09DM0LJ7ktyliziWo4kGjwWBLRMXxs/YIYh5voAsEIDldHwQ/myIXdl4eu1IqdwhPzP/Pms9GFy4IBQDokb4fpXasDydWy5Ijv4W2cCp/+lYGTZSW+kuyV6qjehZJ8nqVJnv8jn5mG58XJU/SA+WIwynVU8y8vy1zOI2r0xszHo4wv7+vYjA7p3PzJmBf5Txv8ulV8BHS2ugD6pTgV38wcovAAf/lw01JftCg7qRYHGOEKl/W6BTI7ILFBQKQPsILxzHRZHyYYpG6bBr6xcS+J7FlrbgYuf+WpQJ/gDl10YFril5UfWfwiNv+07o8331gcXbfqq7ImX0Hs9Q0= ceph-af43fc90-73b1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:29:18.447 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5Y+b/spprJuGE0Zka3RAdjSAxMu8rL8Ao3Zsr+NNIElFU2MTG/jXmqCgtKFKZjsMhDtXhRY7kRofSCi/+CEi8MavsJs8qpYX/dPAybC5hirGXrqe+bTVIVKhdxd6WjF0JNdeyCKFm0hte/myrLU1eHXXEGUrhp53RNvLKLPrFlfaPIl09DM0LJ7ktyliziWo4kGjwWBLRMXxs/YIYh5voAsEIDldHwQ/myIXdl4eu1IqdwhPzP/Pms9GFy4IBQDokb4fpXasDydWy5Ijv4W2cCp/+lYGTZSW+kuyV6qjehZJ8nqVJnv8jn5mG58XJU/SA+WIwynVU8y8vy1zOI2r0xszHo4wv7+vYjA7p3PzJmBf5Txv8ulV8BHS2ugD6pTgV38wcovAAf/lw01JftCg7qRYHGOEKl/W6BTI7ILFBQKQPsILxzHRZHyYYpG6bBr6xcS+J7FlrbgYuf+WpQJ/gDl10YFril5UfWfwiNv+07o8331gcXbfqq7ImX0Hs9Q0= ceph-af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:18.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5Y+b/spprJuGE0Zka3RAdjSAxMu8rL8Ao3Zsr+NNIElFU2MTG/jXmqCgtKFKZjsMhDtXhRY7kRofSCi/+CEi8MavsJs8qpYX/dPAybC5hirGXrqe+bTVIVKhdxd6WjF0JNdeyCKFm0hte/myrLU1eHXXEGUrhp53RNvLKLPrFlfaPIl09DM0LJ7ktyliziWo4kGjwWBLRMXxs/YIYh5voAsEIDldHwQ/myIXdl4eu1IqdwhPzP/Pms9GFy4IBQDokb4fpXasDydWy5Ijv4W2cCp/+lYGTZSW+kuyV6qjehZJ8nqVJnv8jn5mG58XJU/SA+WIwynVU8y8vy1zOI2r0xszHo4wv7+vYjA7p3PzJmBf5Txv8ulV8BHS2ugD6pTgV38wcovAAf/lw01JftCg7qRYHGOEKl/W6BTI7ILFBQKQPsILxzHRZHyYYpG6bBr6xcS+J7FlrbgYuf+WpQJ/gDl10YFril5UfWfwiNv+07o8331gcXbfqq7ImX0Hs9Q0= ceph-af43fc90-73b1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:29:18.501 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5Y+b/spprJuGE0Zka3RAdjSAxMu8rL8Ao3Zsr+NNIElFU2MTG/jXmqCgtKFKZjsMhDtXhRY7kRofSCi/+CEi8MavsJs8qpYX/dPAybC5hirGXrqe+bTVIVKhdxd6WjF0JNdeyCKFm0hte/myrLU1eHXXEGUrhp53RNvLKLPrFlfaPIl09DM0LJ7ktyliziWo4kGjwWBLRMXxs/YIYh5voAsEIDldHwQ/myIXdl4eu1IqdwhPzP/Pms9GFy4IBQDokb4fpXasDydWy5Ijv4W2cCp/+lYGTZSW+kuyV6qjehZJ8nqVJnv8jn5mG58XJU/SA+WIwynVU8y8vy1zOI2r0xszHo4wv7+vYjA7p3PzJmBf5Txv8ulV8BHS2ugD6pTgV38wcovAAf/lw01JftCg7qRYHGOEKl/W6BTI7ILFBQKQPsILxzHRZHyYYpG6bBr6xcS+J7FlrbgYuf+WpQJ/gDl10YFril5UfWfwiNv+07o8331gcXbfqq7ImX0Hs9Q0= ceph-af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:18.514 DEBUG:teuthology.orchestra.run.smithi130:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5Y+b/spprJuGE0Zka3RAdjSAxMu8rL8Ao3Zsr+NNIElFU2MTG/jXmqCgtKFKZjsMhDtXhRY7kRofSCi/+CEi8MavsJs8qpYX/dPAybC5hirGXrqe+bTVIVKhdxd6WjF0JNdeyCKFm0hte/myrLU1eHXXEGUrhp53RNvLKLPrFlfaPIl09DM0LJ7ktyliziWo4kGjwWBLRMXxs/YIYh5voAsEIDldHwQ/myIXdl4eu1IqdwhPzP/Pms9GFy4IBQDokb4fpXasDydWy5Ijv4W2cCp/+lYGTZSW+kuyV6qjehZJ8nqVJnv8jn5mG58XJU/SA+WIwynVU8y8vy1zOI2r0xszHo4wv7+vYjA7p3PzJmBf5Txv8ulV8BHS2ugD6pTgV38wcovAAf/lw01JftCg7qRYHGOEKl/W6BTI7ILFBQKQPsILxzHRZHyYYpG6bBr6xcS+J7FlrbgYuf+WpQJ/gDl10YFril5UfWfwiNv+07o8331gcXbfqq7ImX0Hs9Q0= ceph-af43fc90-73b1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:29:18.555 INFO:teuthology.orchestra.run.smithi130.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5Y+b/spprJuGE0Zka3RAdjSAxMu8rL8Ao3Zsr+NNIElFU2MTG/jXmqCgtKFKZjsMhDtXhRY7kRofSCi/+CEi8MavsJs8qpYX/dPAybC5hirGXrqe+bTVIVKhdxd6WjF0JNdeyCKFm0hte/myrLU1eHXXEGUrhp53RNvLKLPrFlfaPIl09DM0LJ7ktyliziWo4kGjwWBLRMXxs/YIYh5voAsEIDldHwQ/myIXdl4eu1IqdwhPzP/Pms9GFy4IBQDokb4fpXasDydWy5Ijv4W2cCp/+lYGTZSW+kuyV6qjehZJ8nqVJnv8jn5mG58XJU/SA+WIwynVU8y8vy1zOI2r0xszHo4wv7+vYjA7p3PzJmBf5Txv8ulV8BHS2ugD6pTgV38wcovAAf/lw01JftCg7qRYHGOEKl/W6BTI7ILFBQKQPsILxzHRZHyYYpG6bBr6xcS+J7FlrbgYuf+WpQJ/gDl10YFril5UfWfwiNv+07o8331gcXbfqq7ImX0Hs9Q0= ceph-af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:18.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-15T22:29:18.828 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.397+0000 7f96a4d9c640 1 -- 172.21.15.16:0/3005194806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 msgr2=0x7f96a01019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.397+0000 7f96a4d9c640 1 --2- 172.21.15.16:0/3005194806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a01019e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9690009880 tx=0x7f969002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.397+0000 7f96a4d9c640 1 -- 172.21.15.16:0/3005194806 shutdown_connections 2024-09-15T22:29:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.397+0000 7f96a4d9c640 1 --2- 172.21.15.16:0/3005194806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a01019e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.397+0000 7f96a4d9c640 1 -- 172.21.15.16:0/3005194806 >> 172.21.15.16:0/3005194806 conn(0x7f96a00fcd70 msgr2=0x7f96a00ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.397+0000 7f96a4d9c640 1 -- 172.21.15.16:0/3005194806 shutdown_connections 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.397+0000 7f96a4d9c640 1 -- 172.21.15.16:0/3005194806 wait complete. 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.398+0000 7f96a4d9c640 1 Processor -- start 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.398+0000 7f96a4d9c640 1 -- start start 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.398+0000 7f96a4d9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a0196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f96a4d9c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01076f0 con 0x7f96a01015e0 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f969e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a0196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f969e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a0196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60288/0 (socket says 172.21.15.16:60288) 2024-09-15T22:29:19.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f969e59c640 1 -- 172.21.15.16:0/2866486578 learned_addr learned my addr 172.21.15.16:0/2866486578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f969e59c640 1 -- 172.21.15.16:0/2866486578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a01972e0 con 0x7f96a01015e0 2024-09-15T22:29:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f969e59c640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a0196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9690004f60 tx=0x7f96900048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f9697fff640 1 -- 172.21.15.16:0/2866486578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f969003f070 con 0x7f96a01015e0 2024-09-15T22:29:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a0197570 con 0x7f96a01015e0 2024-09-15T22:29:19.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f9697fff640 1 -- 172.21.15.16:0/2866486578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9690004d40 con 0x7f96a01015e0 2024-09-15T22:29:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.399+0000 7f9697fff640 1 -- 172.21.15.16:0/2866486578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9690035030 con 0x7f96a01015e0 2024-09-15T22:29:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.400+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a019a260 con 0x7f96a01015e0 2024-09-15T22:29:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.401+0000 7f9697fff640 1 -- 172.21.15.16:0/2866486578 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f9690042070 con 0x7f96a01015e0 2024-09-15T22:29:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.401+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a0107e70 con 0x7f96a01015e0 2024-09-15T22:29:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.401+0000 7f9697fff640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f968c039300 0x7f968c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.401+0000 7f9697fff640 1 -- 172.21.15.16:0/2866486578 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f969006cfe0 con 0x7f96a01015e0 2024-09-15T22:29:19.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.401+0000 7f9695d9b640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f968c039300 0x7f968c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:19.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.402+0000 7f9695d9b640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f968c039300 0x7f968c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f96880098c0 tx=0x7f9688006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:19.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.406+0000 7f9697fff640 1 -- 172.21.15.16:0/2866486578 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9690042360 con 0x7f96a01015e0 2024-09-15T22:29:19.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.585+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f96a002d830 con 0x7f96a01015e0 2024-09-15T22:29:19.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.588+0000 7f9697fff640 1 -- 172.21.15.16:0/2866486578 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f969002f300 con 0x7f96a01015e0 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f968c039300 msgr2=0x7f968c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f968c039300 0x7f968c03b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f96880098c0 tx=0x7f9688006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 msgr2=0x7f96a0196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a0196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9690004f60 tx=0x7f96900048f0 comp rx=0 tx=0).stop 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 shutdown_connections 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f968c039300 0x7f968c03b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 --2- 172.21.15.16:0/2866486578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96a01015e0 0x7f96a0196da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:19.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.591+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 >> 172.21.15.16:0/2866486578 conn(0x7f96a00fcd70 msgr2=0x7f96a00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.592+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 shutdown_connections 2024-09-15T22:29:19.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:19.592+0000 7f96a4d9c640 1 -- 172.21.15.16:0/2866486578 wait complete. 2024-09-15T22:29:20.643 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-15T22:29:20.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-15T22:29:20.844 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:20 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2866486578' entity='client.admin' 2024-09-15T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.507+0000 7fc442158640 1 -- 172.21.15.16:0/2505995138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 msgr2=0x7fc43c1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.507+0000 7fc442158640 1 --2- 172.21.15.16:0/2505995138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c1044c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009880 tx=0x7fc42c02a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.507+0000 7fc442158640 1 -- 172.21.15.16:0/2505995138 shutdown_connections 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.507+0000 7fc442158640 1 --2- 172.21.15.16:0/2505995138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c1044c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.507+0000 7fc442158640 1 -- 172.21.15.16:0/2505995138 >> 172.21.15.16:0/2505995138 conn(0x7fc43c0fbcd0 msgr2=0x7fc43c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.507+0000 7fc442158640 1 -- 172.21.15.16:0/2505995138 shutdown_connections 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.507+0000 7fc442158640 1 -- 172.21.15.16:0/2505995138 wait complete. 2024-09-15T22:29:21.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.508+0000 7fc442158640 1 Processor -- start 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.508+0000 7fc442158640 1 -- start start 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.508+0000 7fc442158640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.508+0000 7fc442158640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c107670 con 0x7fc43c1020d0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.508+0000 7fc43b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.508+0000 7fc43b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44916/0 (socket says 172.21.15.16:44916) 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.508+0000 7fc43b7fe640 1 -- 172.21.15.16:0/287859064 learned_addr learned my addr 172.21.15.16:0/287859064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.509+0000 7fc43b7fe640 1 -- 172.21.15.16:0/287859064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c197280 con 0x7fc43c1020d0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.509+0000 7fc43b7fe640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c196d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc42c004f60 tx=0x7fc42c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.509+0000 7fc438ff9640 1 -- 172.21.15.16:0/287859064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc42c03f070 con 0x7fc43c1020d0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.509+0000 7fc438ff9640 1 -- 172.21.15.16:0/287859064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc42c003ad0 con 0x7fc43c1020d0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.509+0000 7fc438ff9640 1 -- 172.21.15.16:0/287859064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc42c035030 con 0x7fc43c1020d0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.509+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c197570 con 0x7fc43c1020d0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.509+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c19a200 con 0x7fc43c1020d0 2024-09-15T22:29:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.510+0000 7fc438ff9640 1 -- 172.21.15.16:0/287859064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc42c042070 con 0x7fc43c1020d0 2024-09-15T22:29:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.510+0000 7fc438ff9640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc424039300 0x7fc42403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.510+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43c107df0 con 0x7fc43c1020d0 2024-09-15T22:29:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.510+0000 7fc438ff9640 1 -- 172.21.15.16:0/287859064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc42c06d360 con 0x7fc43c1020d0 2024-09-15T22:29:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.510+0000 7fc43affd640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc424039300 0x7fc42403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.511+0000 7fc43affd640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc424039300 0x7fc42403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4300098c0 tx=0x7fc430006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.515+0000 7fc438ff9640 1 -- 172.21.15.16:0/287859064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc42c02f300 con 0x7fc43c1020d0 2024-09-15T22:29:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.698+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc43c02d970 con 0x7fc424039300 2024-09-15T22:29:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.701+0000 7fc438ff9640 1 -- 172.21.15.16:0/287859064 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc43c02d970 con 0x7fc424039300 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc424039300 msgr2=0x7fc42403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc424039300 0x7fc42403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc4300098c0 tx=0x7fc430006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 msgr2=0x7fc43c196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c196d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc42c004f60 tx=0x7fc42c0048f0 comp rx=0 tx=0).stop 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 shutdown_connections 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc424039300 0x7fc42403b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 --2- 172.21.15.16:0/287859064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc43c1020d0 0x7fc43c196d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 >> 172.21.15.16:0/287859064 conn(0x7fc43c0fbcd0 msgr2=0x7fc43c0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.703+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 shutdown_connections 2024-09-15T22:29:21.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:21.704+0000 7fc442158640 1 -- 172.21.15.16:0/287859064 wait complete. 2024-09-15T22:29:22.406 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi059 2024-09-15T22:29:22.406 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:29:22.406 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/etc/ceph/ceph.conf 2024-09-15T22:29:22.425 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:29:22.425 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:22.483 INFO:tasks.cephadm:Adding host smithi059 to orchestrator... 2024-09-15T22:29:22.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch host add smithi059 2024-09-15T22:29:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:23.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:22 smithi016 ceph-mon[31492]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:29:23.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:22 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:23.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.363+0000 7f8f722ae640 1 -- 172.21.15.16:0/4208374581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 msgr2=0x7f8f6c1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.363+0000 7f8f722ae640 1 --2- 172.21.15.16:0/4208374581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c1044c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009880 tx=0x7f8f5c02a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.363+0000 7f8f722ae640 1 -- 172.21.15.16:0/4208374581 shutdown_connections 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.363+0000 7f8f722ae640 1 --2- 172.21.15.16:0/4208374581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c1044c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.363+0000 7f8f722ae640 1 -- 172.21.15.16:0/4208374581 >> 172.21.15.16:0/4208374581 conn(0x7f8f6c0fbcd0 msgr2=0x7f8f6c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.363+0000 7f8f722ae640 1 -- 172.21.15.16:0/4208374581 shutdown_connections 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.363+0000 7f8f722ae640 1 -- 172.21.15.16:0/4208374581 wait complete. 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f722ae640 1 Processor -- start 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f722ae640 1 -- start start 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f722ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f722ae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c107670 con 0x7f8f6c1020d0 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44938/0 (socket says 172.21.15.16:44938) 2024-09-15T22:29:23.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f6bfff640 1 -- 172.21.15.16:0/3993918280 learned_addr learned my addr 172.21.15.16:0/3993918280 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f6bfff640 1 -- 172.21.15.16:0/3993918280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c197250 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.364+0000 7f8f6bfff640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c196d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c004f60 tx=0x7f8f5c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.365+0000 7f8f697fa640 1 -- 172.21.15.16:0/3993918280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c03f070 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.365+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c197540 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.365+0000 7f8f697fa640 1 -- 172.21.15.16:0/3993918280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f5c003ad0 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.365+0000 7f8f697fa640 1 -- 172.21.15.16:0/3993918280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c035030 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.365+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c19a1c0 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.366+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c107df0 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.366+0000 7f8f697fa640 1 -- 172.21.15.16:0/3993918280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8f5c042070 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.366+0000 7f8f697fa640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f540392a0 0x7f8f5403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.366+0000 7f8f697fa640 1 -- 172.21.15.16:0/3993918280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f5c06d1e0 con 0x7f8f6c1020d0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.366+0000 7f8f6b7fe640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f540392a0 0x7f8f5403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:23.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.367+0000 7f8f6b7fe640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f540392a0 0x7f8f5403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098c0 tx=0x7f8f60006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:23.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.371+0000 7f8f697fa640 1 -- 172.21.15.16:0/3993918280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f5c039350 con 0x7f8f6c1020d0 2024-09-15T22:29:23.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:23.557+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "target": ["mon-mgr", ""]}) -- 0x7f8f6c02d8d0 con 0x7f8f540392a0 2024-09-15T22:29:24.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:24.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:24.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:29:24.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:24.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:25.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:24 smithi016 ceph-mon[31492]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:29:25.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:24 smithi016 ceph-mon[31492]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:29:26.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:25 smithi016 ceph-mon[31492]: Deploying cephadm binary to smithi059 2024-09-15T22:29:27.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.433+0000 7f8f697fa640 1 -- 172.21.15.16:0/3993918280 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8f6c02d8d0 con 0x7f8f540392a0 2024-09-15T22:29:27.435 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi059' with addr '172.21.15.59' 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f540392a0 msgr2=0x7f8f5403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f540392a0 0x7f8f5403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098c0 tx=0x7f8f60006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 msgr2=0x7f8f6c196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c196d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c004f60 tx=0x7f8f5c0048f0 comp rx=0 tx=0).stop 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 shutdown_connections 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f540392a0 0x7f8f5403b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 --2- 172.21.15.16:0/3993918280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c1020d0 0x7f8f6c196d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 >> 172.21.15.16:0/3993918280 conn(0x7f8f6c0fbcd0 msgr2=0x7f8f6c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 shutdown_connections 2024-09-15T22:29:27.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:27.436+0000 7f8f722ae640 1 -- 172.21.15.16:0/3993918280 wait complete. 2024-09-15T22:29:28.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:29:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:28.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:28.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:28 smithi016 ceph-mon[31492]: Added host smithi059 2024-09-15T22:29:28.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:28.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:28.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.739+0000 7fd15b5bd640 1 -- 172.21.15.16:0/1850465470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 msgr2=0x7fd154109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.739+0000 7fd15b5bd640 1 --2- 172.21.15.16:0/1850465470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd154109b50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd150009880 tx=0x7fd15002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.739+0000 7fd15b5bd640 1 -- 172.21.15.16:0/1850465470 shutdown_connections 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.739+0000 7fd15b5bd640 1 --2- 172.21.15.16:0/1850465470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd154109b50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.739+0000 7fd15b5bd640 1 -- 172.21.15.16:0/1850465470 >> 172.21.15.16:0/1850465470 conn(0x7fd1540ff450 msgr2=0x7fd154101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.739+0000 7fd15b5bd640 1 -- 172.21.15.16:0/1850465470 shutdown_connections 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.739+0000 7fd15b5bd640 1 -- 172.21.15.16:0/1850465470 wait complete. 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd15b5bd640 1 Processor -- start 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd15b5bd640 1 -- start start 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd15b5bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd1541993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd15b5bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd15410de30 con 0x7fd154109770 2024-09-15T22:29:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd159359640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd1541993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd159359640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd1541993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44952/0 (socket says 172.21.15.16:44952) 2024-09-15T22:29:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd159359640 1 -- 172.21.15.16:0/244985740 learned_addr learned my addr 172.21.15.16:0/244985740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:28.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.740+0000 7fd159359640 1 -- 172.21.15.16:0/244985740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1541998e0 con 0x7fd154109770 2024-09-15T22:29:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.741+0000 7fd159359640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd1541993a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd150009230 tx=0x7fd150003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.741+0000 7fd14a7fc640 1 -- 172.21.15.16:0/244985740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd15003f070 con 0x7fd154109770 2024-09-15T22:29:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.741+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd154199bd0 con 0x7fd154109770 2024-09-15T22:29:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.741+0000 7fd14a7fc640 1 -- 172.21.15.16:0/244985740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15003bd40 con 0x7fd154109770 2024-09-15T22:29:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.741+0000 7fd14a7fc640 1 -- 172.21.15.16:0/244985740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd150035030 con 0x7fd154109770 2024-09-15T22:29:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.741+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15419dc70 con 0x7fd154109770 2024-09-15T22:29:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.742+0000 7fd14a7fc640 1 -- 172.21.15.16:0/244985740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd150042070 con 0x7fd154109770 2024-09-15T22:29:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.742+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd154104eb0 con 0x7fd154109770 2024-09-15T22:29:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.742+0000 7fd14a7fc640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd1400392a0 0x7fd14003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.742+0000 7fd14a7fc640 1 -- 172.21.15.16:0/244985740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd15006cef0 con 0x7fd154109770 2024-09-15T22:29:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.742+0000 7fd158b58640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd1400392a0 0x7fd14003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.743+0000 7fd158b58640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd1400392a0 0x7fd14003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd1440098c0 tx=0x7fd144006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.747+0000 7fd14a7fc640 1 -- 172.21.15.16:0/244985740 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd150030d60 con 0x7fd154109770 2024-09-15T22:29:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.927+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd154109c30 con 0x7fd1400392a0 2024-09-15T22:29:28.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.928+0000 7fd14a7fc640 1 -- 172.21.15.16:0/244985740 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fd154109c30 con 0x7fd1400392a0 2024-09-15T22:29:28.930 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:28.930 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}] 2024-09-15T22:29:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd1400392a0 msgr2=0x7fd14003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd1400392a0 0x7fd14003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd1440098c0 tx=0x7fd144006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:28.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 msgr2=0x7fd1541993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd1541993a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd150009230 tx=0x7fd150003d80 comp rx=0 tx=0).stop 2024-09-15T22:29:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 shutdown_connections 2024-09-15T22:29:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd1400392a0 0x7fd14003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 --2- 172.21.15.16:0/244985740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd154109770 0x7fd1541993a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.930+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 >> 172.21.15.16:0/244985740 conn(0x7fd1540ff450 msgr2=0x7fd154107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.931+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 shutdown_connections 2024-09-15T22:29:28.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:28.931+0000 7fd15b5bd640 1 -- 172.21.15.16:0/244985740 wait complete. 2024-09-15T22:29:29.507 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi130 2024-09-15T22:29:29.508 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:29:29.508 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.conf 2024-09-15T22:29:29.527 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:29:29.527 DEBUG:teuthology.orchestra.run.smithi130:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:29.585 INFO:tasks.cephadm:Adding host smithi130 to orchestrator... 2024-09-15T22:29:29.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch host add smithi130 2024-09-15T22:29:29.840 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:30.053 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:30.054 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:29 smithi016 ceph-mon[31492]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 -- 172.21.15.16:0/3303508291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 msgr2=0x7f88f0102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 --2- 172.21.15.16:0/3303508291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f0102b30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009880 tx=0x7f88e002a670 comp rx=0 tx=0).stop 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 -- 172.21.15.16:0/3303508291 shutdown_connections 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 --2- 172.21.15.16:0/3303508291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f0102b30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 -- 172.21.15.16:0/3303508291 >> 172.21.15.16:0/3303508291 conn(0x7f88f00fc1b0 msgr2=0x7f88f00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 -- 172.21.15.16:0/3303508291 shutdown_connections 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 -- 172.21.15.16:0/3303508291 wait complete. 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 Processor -- start 2024-09-15T22:29:30.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.263+0000 7f88f690e640 1 -- start start 2024-09-15T22:29:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88f690e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88f690e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88f010bcc0 con 0x7f88f0102750 2024-09-15T22:29:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f01971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44962/0 (socket says 172.21.15.16:44962) 2024-09-15T22:29:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88effff640 1 -- 172.21.15.16:0/785903201 learned_addr learned my addr 172.21.15.16:0/785903201 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:30.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88effff640 1 -- 172.21.15.16:0/785903201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f0197700 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88effff640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f01971c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009230 tx=0x7f88e0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88ed7fa640 1 -- 172.21.15.16:0/785903201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e003f070 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88ed7fa640 1 -- 172.21.15.16:0/785903201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88e003bc80 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f0197990 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.264+0000 7f88ed7fa640 1 -- 172.21.15.16:0/785903201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e0035030 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.265+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f0197df0 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.265+0000 7f88ed7fa640 1 -- 172.21.15.16:0/785903201 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f88e0042070 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.265+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f006a7b0 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.266+0000 7f88ed7fa640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f88d80392a0 0x7f88d803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.266+0000 7f88ed7fa640 1 -- 172.21.15.16:0/785903201 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f88e006cda0 con 0x7f88f0102750 2024-09-15T22:29:30.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.266+0000 7f88ef7fe640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f88d80392a0 0x7f88d803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:30.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.266+0000 7f88ef7fe640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f88d80392a0 0x7f88d803b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098c0 tx=0x7f88e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:30.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.271+0000 7f88ed7fa640 1 -- 172.21.15.16:0/785903201 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88e00314c0 con 0x7f88f0102750 2024-09-15T22:29:30.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:30.449+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}) -- 0x7f88f0198520 con 0x7f88d80392a0 2024-09-15T22:29:31.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:31 smithi016 ceph-mon[31492]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi130", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:29:31.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:31 smithi016 ceph-mon[31492]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: Deploying cephadm binary to smithi130 2024-09-15T22:29:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:29:32.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:32.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:29:32.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:32.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:32.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.416+0000 7f88ed7fa640 1 -- 172.21.15.16:0/785903201 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f88f0198520 con 0x7f88d80392a0 2024-09-15T22:29:34.419 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi130' with addr '172.21.15.130' 2024-09-15T22:29:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f88d80392a0 msgr2=0x7f88d803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f88d80392a0 0x7f88d803b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098c0 tx=0x7f88e4006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 msgr2=0x7f88f01971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f01971c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009230 tx=0x7f88e0003d40 comp rx=0 tx=0).stop 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 shutdown_connections 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f88d80392a0 0x7f88d803b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 --2- 172.21.15.16:0/785903201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f0102750 0x7f88f01971c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 >> 172.21.15.16:0/785903201 conn(0x7f88f00fc1b0 msgr2=0x7f88f0103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 shutdown_connections 2024-09-15T22:29:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:34.419+0000 7f88f690e640 1 -- 172.21.15.16:0/785903201 wait complete. 2024-09-15T22:29:34.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:34 smithi016 ceph-mon[31492]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:35.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:29:35.256 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:35.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:35.645 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:35 smithi016 ceph-mon[31492]: Added host smithi130 2024-09-15T22:29:35.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:35.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:35.646 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 -- 172.21.15.16:0/2293278814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 msgr2=0x7fad78102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 --2- 172.21.15.16:0/2293278814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad78102b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fad68009810 tx=0x7fad6802a540 comp rx=0 tx=0).stop 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 -- 172.21.15.16:0/2293278814 shutdown_connections 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 --2- 172.21.15.16:0/2293278814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad78102b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 -- 172.21.15.16:0/2293278814 >> 172.21.15.16:0/2293278814 conn(0x7fad78077310 msgr2=0x7fad780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 -- 172.21.15.16:0/2293278814 shutdown_connections 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 -- 172.21.15.16:0/2293278814 wait complete. 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.728+0000 7fad7d7cd640 1 Processor -- start 2024-09-15T22:29:35.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad7d7cd640 1 -- start start 2024-09-15T22:29:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad7d7cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad781971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad7d7cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fad7810bc80 con 0x7fad78102730 2024-09-15T22:29:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad781971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad76ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad781971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56456/0 (socket says 172.21.15.16:56456) 2024-09-15T22:29:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad76ffd640 1 -- 172.21.15.16:0/3391865502 learned_addr learned my addr 172.21.15.16:0/3391865502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:35.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad76ffd640 1 -- 172.21.15.16:0/3391865502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad781976f0 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.729+0000 7fad76ffd640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad781971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad68009930 tx=0x7fad68004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.730+0000 7fad6ffff640 1 -- 172.21.15.16:0/3391865502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad68031400 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.730+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad78197980 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.730+0000 7fad6ffff640 1 -- 172.21.15.16:0/3391865502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad680035f0 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.730+0000 7fad6ffff640 1 -- 172.21.15.16:0/3391865502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad6803c440 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.730+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad78197de0 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.730+0000 7fad6ffff640 1 -- 172.21.15.16:0/3391865502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fad68039040 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.731+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad78069f90 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.731+0000 7fad6ffff640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fad64039300 0x7fad6403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.731+0000 7fad6ffff640 1 -- 172.21.15.16:0/3391865502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fad6806d500 con 0x7fad78102730 2024-09-15T22:29:35.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.731+0000 7fad767fc640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fad64039300 0x7fad6403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:35.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.731+0000 7fad767fc640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fad64039300 0x7fad6403b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad600098c0 tx=0x7fad60006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:35.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.736+0000 7fad6ffff640 1 -- 172.21.15.16:0/3391865502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad68040030 con 0x7fad78102730 2024-09-15T22:29:35.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.934+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad78198570 con 0x7fad64039300 2024-09-15T22:29:35.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.936+0000 7fad6ffff640 1 -- 172.21.15.16:0/3391865502 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fad78198570 con 0x7fad64039300 2024-09-15T22:29:35.937 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:29:35.937 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.130", "hostname": "smithi130", "labels": [], "status": ""}] 2024-09-15T22:29:35.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fad64039300 msgr2=0x7fad6403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fad64039300 0x7fad6403b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad600098c0 tx=0x7fad60006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 msgr2=0x7fad781971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad781971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad68009930 tx=0x7fad68004570 comp rx=0 tx=0).stop 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 shutdown_connections 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fad64039300 0x7fad6403b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 --2- 172.21.15.16:0/3391865502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad78102730 0x7fad781971b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 >> 172.21.15.16:0/3391865502 conn(0x7fad78077310 msgr2=0x7fad781034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 shutdown_connections 2024-09-15T22:29:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:35.938+0000 7fad7d7cd640 1 -- 172.21.15.16:0/3391865502 wait complete. 2024-09-15T22:29:36.541 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:36 smithi016 ceph-mon[31492]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:36.541 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:36.573 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-15T22:29:36.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-15T22:29:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:29:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.269+0000 7fd05d502640 1 -- 172.21.15.16:0/2539718746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 msgr2=0x7fd058107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.269+0000 7fd05d502640 1 --2- 172.21.15.16:0/2539718746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd058107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd048009880 tx=0x7fd04802a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:37.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.269+0000 7fd05d502640 1 -- 172.21.15.16:0/2539718746 shutdown_connections 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.269+0000 7fd05d502640 1 --2- 172.21.15.16:0/2539718746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd058107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.269+0000 7fd05d502640 1 -- 172.21.15.16:0/2539718746 >> 172.21.15.16:0/2539718746 conn(0x7fd0580fd290 msgr2=0x7fd0580ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.270+0000 7fd05d502640 1 -- 172.21.15.16:0/2539718746 shutdown_connections 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.270+0000 7fd05d502640 1 -- 172.21.15.16:0/2539718746 wait complete. 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.270+0000 7fd05d502640 1 Processor -- start 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.270+0000 7fd05d502640 1 -- start start 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd05d502640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd0581970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd05d502640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd05810bc90 con 0x7fd058107660 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd056ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd0581970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd056ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd0581970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56478/0 (socket says 172.21.15.16:56478) 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd056ffd640 1 -- 172.21.15.16:0/2965563336 learned_addr learned my addr 172.21.15.16:0/2965563336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd056ffd640 1 -- 172.21.15.16:0/2965563336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058197600 con 0x7fd058107660 2024-09-15T22:29:37.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd056ffd640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd0581970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd048009230 tx=0x7fd048003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd03ffff640 1 -- 172.21.15.16:0/2965563336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd04803f070 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd058197890 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd03ffff640 1 -- 172.21.15.16:0/2965563336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04803bc80 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.271+0000 7fd03ffff640 1 -- 172.21.15.16:0/2965563336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd048035030 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.272+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05819b990 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.272+0000 7fd03ffff640 1 -- 172.21.15.16:0/2965563336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd048042070 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.272+0000 7fd03ffff640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0400392a0 0x7fd04003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.273+0000 7fd03ffff640 1 -- 172.21.15.16:0/2965563336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd04806cfb0 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.273+0000 7fd0567fc640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0400392a0 0x7fd04003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.273+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058102cd0 con 0x7fd058107660 2024-09-15T22:29:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.273+0000 7fd0567fc640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0400392a0 0x7fd04003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0098c0 tx=0x7fd04c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:37.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.278+0000 7fd03ffff640 1 -- 172.21.15.16:0/2965563336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd04804c350 con 0x7fd058107660 2024-09-15T22:29:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:37.485+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd058107af0 con 0x7fd058107660 2024-09-15T22:29:37.487 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:37 smithi016 ceph-mon[31492]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:29:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.424+0000 7fd03ffff640 1 -- 172.21.15.16:0/2965563336 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fd058107af0 con 0x7fd058107660 2024-09-15T22:29:38.426 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-15T22:29:38.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0400392a0 msgr2=0x7fd04003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0400392a0 0x7fd04003b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0098c0 tx=0x7fd04c006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 msgr2=0x7fd0581970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd0581970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd048009230 tx=0x7fd048003d40 comp rx=0 tx=0).stop 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 shutdown_connections 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0400392a0 0x7fd04003b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 --2- 172.21.15.16:0/2965563336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd058107660 0x7fd0581970c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 >> 172.21.15.16:0/2965563336 conn(0x7fd0580fd290 msgr2=0x7fd0580febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 shutdown_connections 2024-09-15T22:29:38.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:29:38.427+0000 7fd05d502640 1 -- 172.21.15.16:0/2965563336 wait complete. 2024-09-15T22:29:38.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:38 smithi016 ceph-mon[31492]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:38.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:38 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2965563336' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:29:39.113 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-09-15T22:29:39.117 INFO:tasks.cephadm:Adding mon.b on smithi059 2024-09-15T22:29:39.118 INFO:tasks.cephadm:Adding mon.c on smithi130 2024-09-15T22:29:39.118 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi059:172.21.15.59=b;smithi130:172.21.15.130=c' 2024-09-15T22:29:39.361 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:29:39.768 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.766+0000 7faf09c3a640 1 -- 172.21.15.130:0/3267689626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 msgr2=0x7faf04106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:39.768 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.766+0000 7faf09c3a640 1 --2- 172.21.15.130:0/3267689626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf04106940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faef4009880 tx=0x7faef402a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.766+0000 7faf09c3a640 1 -- 172.21.15.130:0/3267689626 shutdown_connections 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.766+0000 7faf09c3a640 1 --2- 172.21.15.130:0/3267689626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf04106940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.766+0000 7faf09c3a640 1 -- 172.21.15.130:0/3267689626 >> 172.21.15.130:0/3267689626 conn(0x7faf040fc1b0 msgr2=0x7faf040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.766+0000 7faf09c3a640 1 -- 172.21.15.130:0/3267689626 shutdown_connections 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.766+0000 7faf09c3a640 1 -- 172.21.15.130:0/3267689626 wait complete. 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.767+0000 7faf09c3a640 1 Processor -- start 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.767+0000 7faf09c3a640 1 -- start start 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.767+0000 7faf09c3a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf040fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:39.769 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.767+0000 7faf09c3a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf0410ab90 con 0x7faf04106560 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf040fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf040fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.130:39970/0 (socket says 172.21.15.130:39970) 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf037fe640 1 -- 172.21.15.130:0/854398644 learned_addr learned my addr 172.21.15.130:0/854398644 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf037fe640 1 -- 172.21.15.130:0/854398644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf040fe2a0 con 0x7faf04106560 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf037fe640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf040fc660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef4009230 tx=0x7faef4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf017fa640 1 -- 172.21.15.130:0/854398644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef403f070 con 0x7faf04106560 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf040fe530 con 0x7faf04106560 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf017fa640 1 -- 172.21.15.130:0/854398644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef403bc80 con 0x7faf04106560 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.768+0000 7faf017fa640 1 -- 172.21.15.130:0/854398644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef4035030 con 0x7faf04106560 2024-09-15T22:29:39.772 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.769+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf040fcd70 con 0x7faf04106560 2024-09-15T22:29:39.773 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.769+0000 7faf017fa640 1 -- 172.21.15.130:0/854398644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7faef4042070 con 0x7faf04106560 2024-09-15T22:29:39.773 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.769+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf04101bd0 con 0x7faf04106560 2024-09-15T22:29:39.773 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.770+0000 7faf017fa640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faef00392a0 0x7faef003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:39.773 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.770+0000 7faf017fa640 1 -- 172.21.15.130:0/854398644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faef406ce80 con 0x7faf04106560 2024-09-15T22:29:39.773 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.770+0000 7faefaffd640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faef00392a0 0x7faef003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:39.773 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.770+0000 7faefaffd640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faef00392a0 0x7faef003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faeec0098c0 tx=0x7faeec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:39.776 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.775+0000 7faf017fa640 1 -- 172.21.15.130:0/854398644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faef40314c0 con 0x7faf04106560 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2965563336' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi130", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:29:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: Updating smithi130:/etc/ceph/ceph.conf 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:39.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:39.952 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.951+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi059:172.21.15.59=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}) -- 0x7faf040fdb90 con 0x7faef00392a0 2024-09-15T22:29:39.958 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.956+0000 7faf017fa640 1 -- 172.21.15.130:0/854398644 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faf040fdb90 con 0x7faef00392a0 2024-09-15T22:29:39.958 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mon update... 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faef00392a0 msgr2=0x7faef003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faef00392a0 0x7faef003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7faeec0098c0 tx=0x7faeec006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 msgr2=0x7faf040fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf040fc660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faef4009230 tx=0x7faef4003d40 comp rx=0 tx=0).stop 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 shutdown_connections 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faef00392a0 0x7faef003b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 --2- 172.21.15.130:0/854398644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04106560 0x7faf040fc660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 >> 172.21.15.130:0/854398644 conn(0x7faf040fc1b0 msgr2=0x7faf04069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.958+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 shutdown_connections 2024-09-15T22:29:39.960 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:39.959+0000 7faf09c3a640 1 -- 172.21.15.130:0/854398644 wait complete. 2024-09-15T22:29:40.816 DEBUG:teuthology.orchestra.run.smithi059:mon.b> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.b.service 2024-09-15T22:29:40.819 DEBUG:teuthology.orchestra.run.smithi130:mon.c> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.c.service 2024-09-15T22:29:40.821 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:29:40.821 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:29:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi059:172.21.15.59=b;smithi130:172.21.15.130=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:29:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi059:172.21.15.59=b;smithi130:172.21.15.130=c;count:3 2024-09-15T22:29:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:29:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:29:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:41.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:29:41.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:41.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:40 smithi016 ceph-mon[31492]: Deploying daemon mon.c on smithi130 2024-09-15T22:29:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:41 smithi016 ceph-mon[31492]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:42.412 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:29:42.853 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.848+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2396394658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 msgr2=0x7f6f40106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.848+0000 7f6f47c0d640 1 --2- 172.21.15.130:0/2396394658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40106940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009880 tx=0x7f6f3c02a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.848+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2396394658 shutdown_connections 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.848+0000 7f6f47c0d640 1 --2- 172.21.15.130:0/2396394658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40106940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.848+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2396394658 >> 172.21.15.130:0/2396394658 conn(0x7f6f400fc1b0 msgr2=0x7f6f400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.848+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2396394658 shutdown_connections 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.848+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2396394658 wait complete. 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.849+0000 7f6f47c0d640 1 Processor -- start 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.849+0000 7f6f47c0d640 1 -- start start 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.849+0000 7f6f47c0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.849+0000 7f6f47c0d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4010ab90 con 0x7f6f40106560 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.849+0000 7f6f459a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:42.854 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.849+0000 7f6f459a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40196050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.130:36168/0 (socket says 172.21.15.130:36168) 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.849+0000 7f6f459a9640 1 -- 172.21.15.130:0/2745311947 learned_addr learned my addr 172.21.15.130:0/2745311947 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.850+0000 7f6f459a9640 1 -- 172.21.15.130:0/2745311947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f40196590 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.850+0000 7f6f459a9640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40196050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009230 tx=0x7f6f3c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.850+0000 7f6f36ffd640 1 -- 172.21.15.130:0/2745311947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f3c03f070 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.850+0000 7f6f36ffd640 1 -- 172.21.15.130:0/2745311947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3c03bc80 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.850+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f40196820 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.850+0000 7f6f36ffd640 1 -- 172.21.15.130:0/2745311947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f3c035030 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.850+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f40196c80 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.851+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f40101bd0 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.851+0000 7f6f36ffd640 1 -- 172.21.15.130:0/2745311947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f6f3c042070 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.851+0000 7f6f36ffd640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f2c0392a0 0x7f6f2c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.851+0000 7f6f36ffd640 1 -- 172.21.15.130:0/2745311947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f3c06cff0 con 0x7f6f40106560 2024-09-15T22:29:42.855 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.851+0000 7f6f451a8640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f2c0392a0 0x7f6f2c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:42.856 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.852+0000 7f6f451a8640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f2c0392a0 0x7f6f2c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f300098c0 tx=0x7f6f30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:42.858 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:42.856+0000 7f6f36ffd640 1 -- 172.21.15.130:0/2745311947 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f3c031e20 con 0x7f6f40106560 2024-09-15T22:29:43.092 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.090+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6f401973b0 con 0x7f6f40106560 2024-09-15T22:29:43.093 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.091+0000 7f6f36ffd640 1 -- 172.21.15.130:0/2745311947 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6f3c0372d0 con 0x7f6f40106560 2024-09-15T22:29:43.093 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:29:43.093 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":1,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","modified":"2024-09-15T22:28:19.896069Z","created":"2024-09-15T22:28:19.896069Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-15T22:29:43.093 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 1 2024-09-15T22:29:43.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f2c0392a0 msgr2=0x7f6f2c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:43.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f2c0392a0 0x7f6f2c03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f300098c0 tx=0x7f6f30006db0 comp rx=0 tx=0).stop 2024-09-15T22:29:43.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 msgr2=0x7f6f40196050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:43.095 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40196050 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009230 tx=0x7f6f3c003d40 comp rx=0 tx=0).stop 2024-09-15T22:29:43.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 shutdown_connections 2024-09-15T22:29:43.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f2c0392a0 0x7f6f2c03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:43.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 --2- 172.21.15.130:0/2745311947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f40106560 0x7f6f40196050 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:43.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 >> 172.21.15.130:0/2745311947 conn(0x7f6f400fc1b0 msgr2=0x7f6f400fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:43.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 shutdown_connections 2024-09-15T22:29:43.096 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:43.093+0000 7f6f47c0d640 1 -- 172.21.15.130:0/2745311947 wait complete. 2024-09-15T22:29:44.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:43 smithi016 ceph-mon[31492]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:44.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:43 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/2745311947' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:29:44.999 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:29:45.000 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:29:45.024 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:44 smithi130 systemd[1]: Started Ceph mon.c for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:29:45.252 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:29:50.037 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.035+0000 7fa35ac66640 1 -- 172.21.15.130:0/3799742105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340003530 msgr2=0x7fa3400059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.038 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.035+0000 7fa35ac66640 1 --2- 172.21.15.130:0/3799742105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340003530 0x7fa3400059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa3541016c0 tx=0x7fa33c02bfe0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.038 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.035+0000 7fa35ac66640 1 -- 172.21.15.130:0/3799742105 shutdown_connections 2024-09-15T22:29:50.038 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.035+0000 7fa35ac66640 1 --2- 172.21.15.130:0/3799742105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa340003530 0x7fa3400059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.038 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.035+0000 7fa35ac66640 1 -- 172.21.15.130:0/3799742105 >> 172.21.15.130:0/3799742105 conn(0x7fa3540fc280 msgr2=0x7fa3540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:50.038 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.035+0000 7fa35ac66640 1 -- 172.21.15.130:0/3799742105 shutdown_connections 2024-09-15T22:29:50.038 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.035+0000 7fa35ac66640 1 -- 172.21.15.130:0/3799742105 wait complete. 2024-09-15T22:29:50.040 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.036+0000 7fa35ac66640 1 Processor -- start 2024-09-15T22:29:50.040 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.036+0000 7fa35ac66640 1 -- start start 2024-09-15T22:29:50.040 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.036+0000 7fa35ac66640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 0x7fa354199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.036+0000 7fa35ac66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3541999b0 0x7fa35419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.036+0000 7fa35ac66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa354108690 con 0x7fa3541999b0 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.036+0000 7fa35ac66640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa354108510 con 0x7fa340003530 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa358a02640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 0x7fa354199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa358a02640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 0x7fa354199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54436/0 (socket says 172.21.15.130:54436) 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa358a02640 1 -- 172.21.15.130:0/418545074 learned_addr learned my addr 172.21.15.130:0/418545074 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa358a02640 1 -- 172.21.15.130:0/418545074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 msgr2=0x7fa354199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa358a02640 1 -- 172.21.15.130:0/418545074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 msgr2=0x7fa354199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa358a02640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 0x7fa354199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:29:50.041 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa353fff640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3541999b0 0x7fa35419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa358a02640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 0x7fa354199470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa353fff640 1 -- 172.21.15.130:0/418545074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 msgr2=0x7fa354199470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa353fff640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 0x7fa354199470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa353fff640 1 -- 172.21.15.130:0/418545074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa354199fd0 con 0x7fa3541999b0 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.037+0000 7fa353fff640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3541999b0 0x7fa35419dd40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa34400b3d0 tx=0x7fa34400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.038+0000 7fa351ffb640 1 -- 172.21.15.130:0/418545074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa344011630 con 0x7fa3541999b0 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.038+0000 7fa351ffb640 1 -- 172.21.15.130:0/418545074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa344011cb0 con 0x7fa3541999b0 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.038+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35419e280 con 0x7fa3541999b0 2024-09-15T22:29:50.042 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.038+0000 7fa351ffb640 1 -- 172.21.15.130:0/418545074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa344010df0 con 0x7fa3541999b0 2024-09-15T22:29:50.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.038+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35419e7c0 con 0x7fa3541999b0 2024-09-15T22:29:50.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.039+0000 7fa351ffb640 1 -- 172.21.15.130:0/418545074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa344018020 con 0x7fa3541999b0 2024-09-15T22:29:50.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.039+0000 7fa351ffb640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa334039090 0x7fa33403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:50.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.039+0000 7fa351ffb640 1 -- 172.21.15.130:0/418545074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa34404be80 con 0x7fa3541999b0 2024-09-15T22:29:50.043 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.039+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354101cc0 con 0x7fa3541999b0 2024-09-15T22:29:50.044 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.039+0000 7fa358a02640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa334039090 0x7fa33403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:50.044 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.040+0000 7fa358a02640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa334039090 0x7fa33403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009530 tx=0x7fa33c011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:50.046 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.045+0000 7fa351ffb640 1 -- 172.21.15.130:0/418545074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa34401f020 con 0x7fa3541999b0 2024-09-15T22:29:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:29:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: mon.a calling monitor election 2024-09-15T22:29:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: mon.c calling monitor election 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: monmap epoch 2 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: last_changed 2024-09-15T22:29:44.811395+0000 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: created 2024-09-15T22:28:19.896069+0000 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: min_mon_release 19 (squid) 2024-09-15T22:29:50.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: election_strategy: 1 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: fsmap 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: mgrmap e13: x(active, since 39s) 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: overall HEALTH_OK 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:29:50.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:50.284 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.283+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa35419a7d0 con 0x7fa3541999b0 2024-09-15T22:29:50.285 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.284+0000 7fa351ffb640 1 -- 172.21.15.130:0/418545074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa344018310 con 0x7fa3541999b0 2024-09-15T22:29:50.286 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:29:50.286 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":2,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","modified":"2024-09-15T22:29:44.811395Z","created":"2024-09-15T22:28:19.896069Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:29:50.286 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 2 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa334039090 msgr2=0x7fa33403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa334039090 0x7fa33403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009530 tx=0x7fa33c011040 comp rx=0 tx=0).stop 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3541999b0 msgr2=0x7fa35419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3541999b0 0x7fa35419dd40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa34400b3d0 tx=0x7fa34400b890 comp rx=0 tx=0).stop 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 shutdown_connections 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa334039090 0x7fa33403b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3541999b0 0x7fa35419dd40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 --2- 172.21.15.130:0/418545074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa340003530 0x7fa354199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:50.288 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 >> 172.21.15.130:0/418545074 conn(0x7fa3540fc280 msgr2=0x7fa35406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:50.289 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 shutdown_connections 2024-09-15T22:29:50.289 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:50.286+0000 7fa35ac66640 1 -- 172.21.15.130:0/418545074 wait complete. 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: mon.a calling monitor election 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: mon.c calling monitor election 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: monmap epoch 2 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: last_changed 2024-09-15T22:29:44.811395+0000 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: created 2024-09-15T22:28:19.896069+0000 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: min_mon_release 19 (squid) 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: election_strategy: 1 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-15T22:29:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: fsmap 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: mgrmap e13: x(active, since 39s) 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: overall HEALTH_OK 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:29:50.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:29:51.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:50 smithi130 ceph-mon[35742]: Deploying daemon mon.b on smithi059 2024-09-15T22:29:51.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:50 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/418545074' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:29:51.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:50 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:51.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:50 smithi016 ceph-mon[31492]: Deploying daemon mon.b on smithi059 2024-09-15T22:29:51.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:50 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/418545074' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:29:51.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:50 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:51.774 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:29:51.775 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:29:52.027 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:29:52.147 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:51 smithi130 ceph-mon[35742]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:52.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:51 smithi016 ceph-mon[31492]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.451+0000 7f0173a43640 1 -- 172.21.15.130:0/3976254619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c107660 msgr2=0x7f016c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.451+0000 7f0173a43640 1 --2- 172.21.15.130:0/3976254619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c107660 0x7f016c107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f015c009880 tx=0x7f015c02a5d0 comp rx=0 tx=0).stop 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.451+0000 7f0173a43640 1 -- 172.21.15.130:0/3976254619 shutdown_connections 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.451+0000 7f0173a43640 1 --2- 172.21.15.130:0/3976254619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c107660 0x7f016c107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.452+0000 7f0173a43640 1 -- 172.21.15.130:0/3976254619 >> 172.21.15.130:0/3976254619 conn(0x7f016c0fd320 msgr2=0x7f016c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.452+0000 7f0173a43640 1 -- 172.21.15.130:0/3976254619 shutdown_connections 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.452+0000 7f0173a43640 1 -- 172.21.15.130:0/3976254619 wait complete. 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.452+0000 7f0173a43640 1 Processor -- start 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.452+0000 7f0173a43640 1 -- start start 2024-09-15T22:29:52.458 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0173a43640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 0x7f016c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0173a43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c197900 0x7f016c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0173a43640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f016c10bd20 con 0x7f016c197900 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0173a43640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f016c10bba0 con 0x7f016c107660 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f01717df640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 0x7f016c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f01717df640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 0x7f016c1973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54118/0 (socket says 172.21.15.130:54118) 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f01717df640 1 -- 172.21.15.130:0/445228218 learned_addr learned my addr 172.21.15.130:0/445228218 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0170fde640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c197900 0x7f016c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f01717df640 1 -- 172.21.15.130:0/445228218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 msgr2=0x7f016c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f01717df640 1 -- 172.21.15.130:0/445228218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 msgr2=0x7f016c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f01717df640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 0x7f016c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f01717df640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 0x7f016c1973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0170fde640 1 -- 172.21.15.130:0/445228218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 msgr2=0x7f016c1973c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:29:52.459 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0170fde640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 0x7f016c1973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0170fde640 1 -- 172.21.15.130:0/445228218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c197f20 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.453+0000 7f0170fde640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c197900 0x7f016c19bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f016800b3d0 tx=0x7f016800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.454+0000 7f01627fc640 1 -- 172.21.15.130:0/445228218 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0168011630 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.454+0000 7f01627fc640 1 -- 172.21.15.130:0/445228218 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0168011cb0 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.454+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c19c1d0 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.454+0000 7f01627fc640 1 -- 172.21.15.130:0/445228218 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0168010df0 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.454+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c19c710 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.455+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c102da0 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.455+0000 7f01627fc640 1 -- 172.21.15.130:0/445228218 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f0168018020 con 0x7f016c197900 2024-09-15T22:29:52.460 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.455+0000 7f01627fc640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0158039090 0x7f015803b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:52.461 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.455+0000 7f01627fc640 1 -- 172.21.15.130:0/445228218 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f016804bff0 con 0x7f016c197900 2024-09-15T22:29:52.461 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.456+0000 7f01717df640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0158039090 0x7f015803b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:52.461 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.456+0000 7f01717df640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0158039090 0x7f015803b550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f015c0096c0 tx=0x7f015c005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:52.463 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.461+0000 7f01627fc640 1 -- 172.21.15.130:0/445228218 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f016801f020 con 0x7f016c197900 2024-09-15T22:29:52.691 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.689+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f016c19ca20 con 0x7f016c197900 2024-09-15T22:29:52.691 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.690+0000 7f01627fc640 1 -- 172.21.15.130:0/445228218 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f01680172c0 con 0x7f016c197900 2024-09-15T22:29:52.691 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:29:52.691 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":2,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","modified":"2024-09-15T22:29:44.811395Z","created":"2024-09-15T22:28:19.896069Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:29:52.692 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 2 2024-09-15T22:29:52.693 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0158039090 msgr2=0x7f015803b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:52.693 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0158039090 0x7f015803b550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f015c0096c0 tx=0x7f015c005a40 comp rx=0 tx=0).stop 2024-09-15T22:29:52.693 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c197900 msgr2=0x7f016c19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c197900 0x7f016c19bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f016800b3d0 tx=0x7f016800b890 comp rx=0 tx=0).stop 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 shutdown_connections 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0158039090 0x7f015803b550 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f016c197900 0x7f016c19bc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 --2- 172.21.15.130:0/445228218 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f016c107660 0x7f016c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 >> 172.21.15.130:0/445228218 conn(0x7f016c0fd320 msgr2=0x7f016c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 shutdown_connections 2024-09-15T22:29:52.694 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:52.692+0000 7f0173a43640 1 -- 172.21.15.130:0/445228218 wait complete. 2024-09-15T22:29:53.157 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:52 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/445228218' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:29:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:52 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/445228218' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:29:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:53 smithi059 systemd[1]: Started Ceph mon.b for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:29:54.315 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:29:54.315 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:29:54.566 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:29:59.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:29:59.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: mon.a calling monitor election 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: mon.c calling monitor election 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: mon.b calling monitor election 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: monmap epoch 3 2024-09-15T22:29:59.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: last_changed 2024-09-15T22:29:53.883048+0000 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: created 2024-09-15T22:28:19.896069+0000 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: min_mon_release 19 (squid) 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: election_strategy: 1 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: fsmap 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: mgrmap e13: x(active, since 48s) 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: overall HEALTH_OK 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:29:59.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:59.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: mon.a calling monitor election 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: mon.c calling monitor election 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: mon.b calling monitor election 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: monmap epoch 3 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:59.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: last_changed 2024-09-15T22:29:53.883048+0000 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: created 2024-09-15T22:28:19.896069+0000 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: min_mon_release 19 (squid) 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: election_strategy: 1 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: fsmap 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: mgrmap e13: x(active, since 48s) 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: overall HEALTH_OK 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.406 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.402+0000 7fa958d19640 1 -- 172.21.15.130:0/1275257106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa93c003530 msgr2=0x7fa93c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.402+0000 7fa958d19640 1 --2- 172.21.15.130:0/1275257106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa93c003530 0x7fa93c0059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa9541016c0 tx=0x7fa94802c490 comp rx=0 tx=0).stop 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.403+0000 7fa958d19640 1 -- 172.21.15.130:0/1275257106 shutdown_connections 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.403+0000 7fa958d19640 1 --2- 172.21.15.130:0/1275257106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa93c003530 0x7fa93c0059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.403+0000 7fa958d19640 1 -- 172.21.15.130:0/1275257106 >> 172.21.15.130:0/1275257106 conn(0x7fa9540fc280 msgr2=0x7fa9540fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.403+0000 7fa958d19640 1 -- 172.21.15.130:0/1275257106 shutdown_connections 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.403+0000 7fa958d19640 1 -- 172.21.15.130:0/1275257106 wait complete. 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.403+0000 7fa958d19640 1 Processor -- start 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa958d19640 1 -- start start 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa958d19640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 0x7fa9541973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:59.407 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa958d19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541978f0 0x7fa95419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa958d19640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 0x7fa95419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa958d19640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa954108680 con 0x7fa9541978f0 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa958d19640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa954108500 con 0x7fa95419c1c0 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa958d19640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa954108800 con 0x7fa93c003530 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa95259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 0x7fa9541973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa95259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 0x7fa9541973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:54124/0 (socket says 172.21.15.130:54124) 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa95259c640 1 -- 172.21.15.130:0/4186358305 learned_addr learned my addr 172.21.15.130:0/4186358305 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa951d9b640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541978f0 0x7fa95419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa95259c640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 msgr2=0x7fa9541973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa95259c640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 msgr2=0x7fa9541973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa95259c640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 0x7fa9541973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:29:59.408 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa952d9d640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 0x7fa95419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa95259c640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 0x7fa9541973b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa952d9d640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 msgr2=0x7fa95419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa952d9d640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 msgr2=0x7fa95419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa952d9d640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 0x7fa95419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa952d9d640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 0x7fa95419c670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa951d9b640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 msgr2=0x7fa9541973b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa951d9b640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 0x7fa9541973b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa951d9b640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 msgr2=0x7fa95419c670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa951d9b640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 0x7fa95419c670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.404+0000 7fa951d9b640 1 -- 172.21.15.130:0/4186358305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541a3f30 con 0x7fa9541978f0 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.405+0000 7fa951d9b640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541978f0 0x7fa95419bc80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b450 tx=0x7fa94400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.405+0000 7fa9437fe640 1 -- 172.21.15.130:0/4186358305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944004230 con 0x7fa9541978f0 2024-09-15T22:29:59.409 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.405+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541a4160 con 0x7fa9541978f0 2024-09-15T22:29:59.410 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.405+0000 7fa9437fe640 1 -- 172.21.15.130:0/4186358305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9440043d0 con 0x7fa9541978f0 2024-09-15T22:29:59.410 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.405+0000 7fa9437fe640 1 -- 172.21.15.130:0/4186358305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944003cb0 con 0x7fa9541978f0 2024-09-15T22:29:59.410 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.405+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a4670 con 0x7fa9541978f0 2024-09-15T22:29:59.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.406+0000 7fa9437fe640 1 -- 172.21.15.130:0/4186358305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fa94400f050 con 0x7fa9541978f0 2024-09-15T22:29:59.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.406+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9541097f0 con 0x7fa9541978f0 2024-09-15T22:29:59.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.406+0000 7fa9437fe640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa934039730 0x7fa93403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:29:59.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.406+0000 7fa9437fe640 1 -- 172.21.15.130:0/4186358305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa94404c160 con 0x7fa9541978f0 2024-09-15T22:29:59.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.406+0000 7fa95259c640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa934039730 0x7fa93403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:29:59.411 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.407+0000 7fa95259c640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa934039730 0x7fa93403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa94802ca00 tx=0x7fa94802c990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:29:59.413 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.411+0000 7fa9437fe640 1 -- 172.21.15.130:0/4186358305 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa94400d040 con 0x7fa9541978f0 2024-09-15T22:29:59.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:29:59.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: mon.a calling monitor election 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: mon.c calling monitor election 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: mon.b calling monitor election 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: monmap epoch 3 2024-09-15T22:29:59.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: last_changed 2024-09-15T22:29:53.883048+0000 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: created 2024-09-15T22:28:19.896069+0000 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: min_mon_release 19 (squid) 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: election_strategy: 1 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: 1: [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] mon.c 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: 2: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: fsmap 2024-09-15T22:29:59.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:29:59.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: mgrmap e13: x(active, since 48s) 2024-09-15T22:29:59.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: overall HEALTH_OK 2024-09-15T22:29:59.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:29:59.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.649+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa9541a4a60 con 0x7fa9541978f0 2024-09-15T22:29:59.651 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.650+0000 7fa9437fe640 1 -- 172.21.15.130:0/4186358305 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fa944011d00 con 0x7fa9541978f0 2024-09-15T22:29:59.652 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:29:59.652 INFO:teuthology.orchestra.run.smithi130.stdout:{"epoch":3,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","modified":"2024-09-15T22:29:53.883048Z","created":"2024-09-15T22:28:19.896069Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:3300","nonce":0},{"type":"v1","addr":"172.21.15.130:6789","nonce":0}]},"addr":"172.21.15.130:6789/0","public_addr":"172.21.15.130:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-15T22:29:59.652 INFO:teuthology.orchestra.run.smithi130.stderr:dumped monmap epoch 3 2024-09-15T22:29:59.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa934039730 msgr2=0x7fa93403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:59.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa934039730 0x7fa93403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa94802ca00 tx=0x7fa94802c990 comp rx=0 tx=0).stop 2024-09-15T22:29:59.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541978f0 msgr2=0x7fa95419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:29:59.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541978f0 0x7fa95419bc80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b450 tx=0x7fa94400b910 comp rx=0 tx=0).stop 2024-09-15T22:29:59.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 shutdown_connections 2024-09-15T22:29:59.654 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa934039730 0x7fa93403bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:59.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa95419c1c0 0x7fa95419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:59.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9541978f0 0x7fa95419bc80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:59.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 --2- 172.21.15.130:0/4186358305 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa93c003530 0x7fa9541973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:29:59.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 >> 172.21.15.130:0/4186358305 conn(0x7fa9540fc280 msgr2=0x7fa954105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:29:59.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 shutdown_connections 2024-09-15T22:29:59.655 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:29:59.652+0000 7fa958d19640 1 -- 172.21.15.130:0/4186358305 wait complete. 2024-09-15T22:30:00.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:59 smithi130 ceph-mon[35742]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:00.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:00.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:59 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/4186358305' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:30:00.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:29:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:30:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:59 smithi016 ceph-mon[31492]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:59 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/4186358305' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:30:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:29:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:30:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:59 smithi059 ceph-mon[34872]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:59 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/4186358305' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:30:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:29:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:30:00.700 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-15T22:30:00.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-15T22:30:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:01.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: overall HEALTH_OK 2024-09-15T22:30:01.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: Updating smithi130:/etc/ceph/ceph.conf 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:01.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:01.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: overall HEALTH_OK 2024-09-15T22:30:01.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: Updating smithi130:/etc/ceph/ceph.conf 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:01.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:01.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: overall HEALTH_OK 2024-09-15T22:30:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: Updating smithi130:/etc/ceph/ceph.conf 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:01.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:01.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:01.446 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:02.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.197+0000 7f746ba3e640 1 -- 172.21.15.16:0/1959075898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 msgr2=0x7f7464107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.197+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1959075898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464107a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7460009c20 tx=0x7f746002aa90 comp rx=0 tx=0).stop 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.197+0000 7f746ba3e640 1 -- 172.21.15.16:0/1959075898 shutdown_connections 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.197+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1959075898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464107a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.197+0000 7f746ba3e640 1 -- 172.21.15.16:0/1959075898 >> 172.21.15.16:0/1959075898 conn(0x7f74640fd260 msgr2=0x7f74640ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.197+0000 7f746ba3e640 1 -- 172.21.15.16:0/1959075898 shutdown_connections 2024-09-15T22:30:02.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.197+0000 7f746ba3e640 1 -- 172.21.15.16:0/1959075898 wait complete. 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 Processor -- start 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 -- start start 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74641978a0 0x7f746419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f746419c170 0x7f746419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f746410bc80 con 0x7f7464107650 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f746410bb00 con 0x7f746419c170 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f746ba3e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f746410be00 con 0x7f74641978a0 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f74697da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f74697da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51466/0 (socket says 172.21.15.16:51466) 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f74697da640 1 -- 172.21.15.16:0/1195634789 learned_addr learned my addr 172.21.15.16:0/1195634789 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:30:02.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.198+0000 7f7469fdb640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f746419c170 0x7f746419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f7468fd9640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74641978a0 0x7f746419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f74697da640 1 -- 172.21.15.16:0/1195634789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74641978a0 msgr2=0x7f746419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f74697da640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74641978a0 0x7f746419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f74697da640 1 -- 172.21.15.16:0/1195634789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f746419c170 msgr2=0x7f746419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f74697da640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f746419c170 0x7f746419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f74697da640 1 -- 172.21.15.16:0/1195634789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74641a3ee0 con 0x7f7464107650 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f74697da640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464197360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f746002b080 tx=0x7f746002f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f745a7fc640 1 -- 172.21.15.16:0/1195634789 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460047070 con 0x7f7464107650 2024-09-15T22:30:02.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f745a7fc640 1 -- 172.21.15.16:0/1195634789 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7460003e20 con 0x7f7464107650 2024-09-15T22:30:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f745a7fc640 1 -- 172.21.15.16:0/1195634789 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460033050 con 0x7f7464107650 2024-09-15T22:30:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74641a40b0 con 0x7f7464107650 2024-09-15T22:30:02.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.199+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74641a4590 con 0x7f7464107650 2024-09-15T22:30:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.200+0000 7f745a7fc640 1 -- 172.21.15.16:0/1195634789 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f746004a070 con 0x7f7464107650 2024-09-15T22:30:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.200+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7464107ab0 con 0x7f7464107650 2024-09-15T22:30:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.200+0000 7f745a7fc640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7450039730 0x7f745003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.200+0000 7f745a7fc640 1 -- 172.21.15.16:0/1195634789 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f746006d610 con 0x7f7464107650 2024-09-15T22:30:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.201+0000 7f7468fd9640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7450039730 0x7f745003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:02.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.201+0000 7f7468fd9640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7450039730 0x7f745003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7454009950 tx=0x7f7454008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:02.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.205+0000 7f745a7fc640 1 -- 172.21.15.16:0/1195634789 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74600373b0 con 0x7f7464107650 2024-09-15T22:30:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.383+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f74641a4890 con 0x7f7464107650 2024-09-15T22:30:02.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:02 smithi016 ceph-mon[31492]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:02 smithi016 ceph-mon[31492]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:02 smithi016 ceph-mon[31492]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:02 smithi016 ceph-mon[31492]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:02.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:02 smithi016 ceph-mon[31492]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:30:02.386 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:02 smithi016 ceph-mon[31492]: Reconfiguring daemon mon.a on smithi016 2024-09-15T22:30:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.383+0000 7f745a7fc640 1 -- 172.21.15.16:0/1195634789 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f7460051130 con 0x7f7464107650 2024-09-15T22:30:02.387 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:30:02.387 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-15T22:30:02.387 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:30:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] 2024-09-15T22:30:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7450039730 msgr2=0x7f745003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7450039730 0x7f745003bbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7454009950 tx=0x7f7454008040 comp rx=0 tx=0).stop 2024-09-15T22:30:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 msgr2=0x7f7464197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464197360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f746002b080 tx=0x7f746002f8a0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 shutdown_connections 2024-09-15T22:30:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7450039730 0x7f745003bbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f746419c170 0x7f746419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f74641978a0 0x7f746419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 --2- 172.21.15.16:0/1195634789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464107650 0x7f7464197360 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 >> 172.21.15.16:0/1195634789 conn(0x7f74640fd260 msgr2=0x7f74640fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 shutdown_connections 2024-09-15T22:30:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:02.386+0000 7f746ba3e640 1 -- 172.21.15.16:0/1195634789 wait complete. 2024-09-15T22:30:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:02 smithi130 ceph-mon[35742]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:02 smithi130 ceph-mon[35742]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:02 smithi130 ceph-mon[35742]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:02 smithi130 ceph-mon[35742]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:02.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:02 smithi130 ceph-mon[35742]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:30:02.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:02 smithi130 ceph-mon[35742]: Reconfiguring daemon mon.a on smithi016 2024-09-15T22:30:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:02 smithi059 ceph-mon[34872]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:02 smithi059 ceph-mon[34872]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:02 smithi059 ceph-mon[34872]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:30:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:02 smithi059 ceph-mon[34872]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:02 smithi059 ceph-mon[34872]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:30:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:02 smithi059 ceph-mon[34872]: Reconfiguring daemon mon.a on smithi016 2024-09-15T22:30:03.383 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:03 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1195634789' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:03.383 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:03.383 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:03.383 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:03.383 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:03.383 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:03 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1195634789' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:03.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:03.413 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-15T22:30:03.413 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:30:03.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:30:03.444 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:30:03.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:30:03.513 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:30:03.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:30:03.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:03 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1195634789' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:03.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:03.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:03.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:03.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:03.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:03.541 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:30:03.541 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:30:03.606 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:30:03.607 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:30:03.640 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:30:03.640 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:30:03.708 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-09-15T22:30:03.709 INFO:tasks.cephadm:Adding mgr.y on smithi059 2024-09-15T22:30:03.709 INFO:tasks.cephadm:Adding mgr.z on smithi130 2024-09-15T22:30:03.709 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi016=x;smithi059=y;smithi130=z' 2024-09-15T22:30:03.986 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:30:04.432 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:04.433 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:30:04.433 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: Reconfiguring daemon mon.b on smithi059 2024-09-15T22:30:04.433 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:04.433 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:04.433 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:04.433 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:04.433 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:04.441 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.439+0000 7f73668f8640 1 -- 172.21.15.130:0/4172385759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601005f0 msgr2=0x7f73601009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:04.441 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.439+0000 7f73668f8640 1 --2- 172.21.15.130:0/4172385759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601005f0 0x7f73601009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7350009880 tx=0x7f735002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.439+0000 7f73668f8640 1 -- 172.21.15.130:0/4172385759 shutdown_connections 2024-09-15T22:30:04.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.439+0000 7f73668f8640 1 --2- 172.21.15.130:0/4172385759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601005f0 0x7f73601009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.439+0000 7f73668f8640 1 -- 172.21.15.130:0/4172385759 >> 172.21.15.130:0/4172385759 conn(0x7f73600fbda0 msgr2=0x7f73600fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:04.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.439+0000 7f73668f8640 1 -- 172.21.15.130:0/4172385759 shutdown_connections 2024-09-15T22:30:04.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.439+0000 7f73668f8640 1 -- 172.21.15.130:0/4172385759 wait complete. 2024-09-15T22:30:04.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 Processor -- start 2024-09-15T22:30:04.442 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 -- start start 2024-09-15T22:30:04.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73601005f0 0x7f7360196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:04.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601974f0 0x7f736019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:04.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7360197970 0x7f736019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:04.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7360106700 con 0x7f73601974f0 2024-09-15T22:30:04.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7360106580 con 0x7f7360197970 2024-09-15T22:30:04.444 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.440+0000 7f73668f8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7360106880 con 0x7f73601005f0 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73601005f0 0x7f7360196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73601005f0 0x7f7360196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:53310/0 (socket says 172.21.15.130:53310) 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601974f0 0x7f736019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 -- 172.21.15.130:0/1420513419 learned_addr learned my addr 172.21.15.130:0/1420513419 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f7364e95640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7360197970 0x7f736019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 -- 172.21.15.130:0/1420513419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7360197970 msgr2=0x7f736019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7360197970 0x7f736019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 -- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601974f0 msgr2=0x7f736019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601974f0 0x7f736019c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 -- 172.21.15.130:0/1420513419 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736019f3b0 con 0x7f73601005f0 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735f7fe640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601974f0 0x7f736019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:30:04.445 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735ffff640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73601005f0 0x7f7360196fb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7350005ab0 tx=0x7f7350032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:04.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735d7fa640 1 -- 172.21.15.130:0/1420513419 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735003f030 con 0x7f73601005f0 2024-09-15T22:30:04.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735d7fa640 1 -- 172.21.15.130:0/1420513419 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7350003ce0 con 0x7f73601005f0 2024-09-15T22:30:04.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736019f640 con 0x7f73601005f0 2024-09-15T22:30:04.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f735d7fa640 1 -- 172.21.15.130:0/1420513419 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350035070 con 0x7f73601005f0 2024-09-15T22:30:04.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.441+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f736019fba0 con 0x7f73601005f0 2024-09-15T22:30:04.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.442+0000 7f735d7fa640 1 -- 172.21.15.130:0/1420513419 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7350042030 con 0x7f73601005f0 2024-09-15T22:30:04.446 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.442+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73601907d0 con 0x7f73601005f0 2024-09-15T22:30:04.447 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.443+0000 7f735d7fa640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7348039730 0x7f734803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:04.447 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.443+0000 7f735d7fa640 1 -- 172.21.15.130:0/1420513419 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f735006cd30 con 0x7f73601005f0 2024-09-15T22:30:04.447 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.443+0000 7f735f7fe640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7348039730 0x7f734803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:04.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:04.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:30:04.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: Reconfiguring daemon mon.b on smithi059 2024-09-15T22:30:04.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:04.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:04.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:04.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:04.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:04.448 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.443+0000 7f735f7fe640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7348039730 0x7f734803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73601984d0 tx=0x7f7354009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:04.449 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.448+0000 7f735d7fa640 1 -- 172.21.15.130:0/1420513419 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f735003bdc0 con 0x7f73601005f0 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: Reconfiguring daemon mon.b on smithi059 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:30:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:04.634 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.633+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi059=y;smithi130=z", "target": ["mon-mgr", ""]}) -- 0x7f73600626c0 con 0x7f7348039730 2024-09-15T22:30:04.641 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.639+0000 7f735d7fa640 1 -- 172.21.15.130:0/1420513419 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f73600626c0 con 0x7f7348039730 2024-09-15T22:30:04.641 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mgr update... 2024-09-15T22:30:04.643 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7348039730 msgr2=0x7f734803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7348039730 0x7f734803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73601984d0 tx=0x7f7354009190 comp rx=0 tx=0).stop 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73601005f0 msgr2=0x7f7360196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73601005f0 0x7f7360196fb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7350005ab0 tx=0x7f7350032d00 comp rx=0 tx=0).stop 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 shutdown_connections 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7348039730 0x7f734803bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7360197970 0x7f736019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73601974f0 0x7f736019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 --2- 172.21.15.130:0/1420513419 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73601005f0 0x7f7360196fb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:04.644 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 >> 172.21.15.130:0/1420513419 conn(0x7f73600fbda0 msgr2=0x7f736006a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:04.645 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 shutdown_connections 2024-09-15T22:30:04.645 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:30:04.642+0000 7f73668f8640 1 -- 172.21.15.130:0/1420513419 wait complete. 2024-09-15T22:30:05.389 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:05 smithi130 ceph-mon[35742]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:30:05.389 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:05 smithi130 ceph-mon[35742]: Reconfiguring daemon mon.c on smithi130 2024-09-15T22:30:05.389 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:05 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:05.459 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:05 smithi016 ceph-mon[31492]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:30:05.459 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:05 smithi016 ceph-mon[31492]: Reconfiguring daemon mon.c on smithi130 2024-09-15T22:30:05.459 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:05 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:05.689 DEBUG:teuthology.orchestra.run.smithi059:mgr.y> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.y.service 2024-09-15T22:30:05.692 DEBUG:teuthology.orchestra.run.smithi130:mgr.z> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.z.service 2024-09-15T22:30:05.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:05 smithi059 ceph-mon[34872]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:30:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:05 smithi059 ceph-mon[34872]: Reconfiguring daemon mon.c on smithi130 2024-09-15T22:30:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:05 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:05.695 INFO:tasks.cephadm:Deploying OSDs... 2024-09-15T22:30:05.696 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:30:05.696 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:30:05.716 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:30:05.716 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-15T22:30:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:30:05.776 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:05.776 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-15T22:30:05.776 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:05.776 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:05.776 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-15 22:29:21.432489587 +0000 2024-09-15T22:30:05.777 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-15 22:25:05.404864440 +0000 2024-09-15T22:30:05.777 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-15 22:25:05.404864440 +0000 2024-09-15T22:30:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-15 22:25:05.404864440 +0000 2024-09-15T22:30:05.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:30:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-15T22:30:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-15T22:30:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000127216 s, 4.0 MB/s 2024-09-15T22:30:05.848 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:30:05.908 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-15T22:30:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:30:05.966 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:05.966 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T22:30:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-15 22:29:21.459489178 +0000 2024-09-15T22:30:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-15 22:25:05.746859524 +0000 2024-09-15T22:30:05.967 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-15 22:25:05.746859524 +0000 2024-09-15T22:30:05.967 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-15 22:25:05.746859524 +0000 2024-09-15T22:30:05.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:30:06.033 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-15T22:30:06.033 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-15T22:30:06.033 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000120882 s, 4.2 MB/s 2024-09-15T22:30:06.035 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:30:06.093 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-15 22:29:21.487488755 +0000 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-15 22:25:06.073854823 +0000 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-15 22:25:06.073854823 +0000 2024-09-15T22:30:06.151 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-15 22:25:06.073854823 +0000 2024-09-15T22:30:06.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:30:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-15T22:30:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-15T22:30:06.217 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.3925e-05 s, 5.5 MB/s 2024-09-15T22:30:06.218 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:30:06.275 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-15 22:29:21.518488287 +0000 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-15 22:25:06.410849979 +0000 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-15 22:25:06.410849979 +0000 2024-09-15T22:30:06.333 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-15 22:25:06.410849979 +0000 2024-09-15T22:30:06.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:30:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-15T22:30:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-15T22:30:06.398 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000112894 s, 4.5 MB/s 2024-09-15T22:30:06.399 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:30:06.459 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-09-15T22:30:06.459 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:30:06.483 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:30:06.484 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-09-15T22:30:06.522 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi059=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: Saving service mgr spec with placement smithi016=x;smithi059=y;smithi130=z;count:3 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:06.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:30:06.524 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:06.524 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:06.557 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:30:06.557 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:06.557 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-15T22:30:06.558 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:06.558 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:06.558 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-15 22:29:53.196492617 +0000 2024-09-15T22:30:06.558 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-15 22:25:20.068108859 +0000 2024-09-15T22:30:06.558 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-15 22:25:20.068108859 +0000 2024-09-15T22:30:06.558 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-15 22:25:20.068108859 +0000 2024-09-15T22:30:06.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:30:06.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:06.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi059=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:06.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: Saving service mgr spec with placement smithi016=x;smithi059=y;smithi130=z;count:3 2024-09-15T22:30:06.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:06.560 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:06.560 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:06.560 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.560 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:06.560 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:30:06.560 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:06.560 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi016=x;smithi059=y;smithi130=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: Saving service mgr spec with placement smithi016=x;smithi059=y;smithi130=z;count:3 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:06.638 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:06.639 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:30:06.639 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:06.639 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-15T22:30:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-15T22:30:06.640 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000274945 s, 1.9 MB/s 2024-09-15T22:30:06.642 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:30:06.701 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-15 22:29:53.196492617 +0000 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-15 22:25:20.388104822 +0000 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-15 22:25:20.388104822 +0000 2024-09-15T22:30:06.763 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-15 22:25:20.388104822 +0000 2024-09-15T22:30:06.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:30:06.832 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-15T22:30:06.832 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-15T22:30:06.832 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.00019691 s, 2.6 MB/s 2024-09-15T22:30:06.833 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:30:06.894 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-09-15T22:30:06.953 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:30:06.953 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:06.953 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-15T22:30:06.954 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:06.954 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:06.954 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-15 22:29:53.196492617 +0000 2024-09-15T22:30:06.954 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-15 22:25:20.710100759 +0000 2024-09-15T22:30:06.954 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-15 22:25:20.710100759 +0000 2024-09-15T22:30:06.954 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-15 22:25:20.710100759 +0000 2024-09-15T22:30:06.954 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:30:07.021 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-15T22:30:07.021 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-15T22:30:07.022 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000215214 s, 2.4 MB/s 2024-09-15T22:30:07.023 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:30:07.082 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-09-15T22:30:07.139 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:30:07.139 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:07.139 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-15T22:30:07.140 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:07.140 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:07.140 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-09-15 22:29:53.197492604 +0000 2024-09-15T22:30:07.140 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-09-15 22:25:21.045096532 +0000 2024-09-15T22:30:07.140 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-09-15 22:25:21.045096532 +0000 2024-09-15T22:30:07.140 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-09-15 22:25:21.045096532 +0000 2024-09-15T22:30:07.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:30:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-09-15T22:30:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-09-15T22:30:07.206 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000201217 s, 2.5 MB/s 2024-09-15T22:30:07.207 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:30:07.264 DEBUG:teuthology.orchestra.run.smithi130:> set -ex 2024-09-15T22:30:07.264 DEBUG:teuthology.orchestra.run.smithi130:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:30:07.282 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:30:07.282 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_1 2024-09-15T22:30:07.338 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-15 22:29:44.256695805 +0000 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-15 22:25:14.394483127 +0000 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-15 22:25:14.394483127 +0000 2024-09-15T22:30:07.339 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-15 22:25:14.394483127 +0000 2024-09-15T22:30:07.339 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:30:07.405 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-15T22:30:07.405 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-15T22:30:07.405 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000255752 s, 2.0 MB/s 2024-09-15T22:30:07.406 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:30:07.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:07 smithi059 ceph-mon[34872]: Deploying daemon mgr.z on smithi130 2024-09-15T22:30:07.464 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_2 2024-09-15T22:30:07.520 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:30:07.520 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:07.520 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-15T22:30:07.520 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:07.521 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:07.521 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-15 22:30:07.514259861 +0000 2024-09-15T22:30:07.521 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-15 22:25:14.715477136 +0000 2024-09-15T22:30:07.521 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-15 22:25:14.715477136 +0000 2024-09-15T22:30:07.521 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-15 22:25:14.715477136 +0000 2024-09-15T22:30:07.521 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:30:07.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:07 smithi130 ceph-mon[35742]: Deploying daemon mgr.z on smithi130 2024-09-15T22:30:07.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:07 smithi016 ceph-mon[31492]: Deploying daemon mgr.z on smithi130 2024-09-15T22:30:07.563 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-15T22:30:07.564 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-15T22:30:07.564 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000196838 s, 2.6 MB/s 2024-09-15T22:30:07.565 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:30:07.626 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_3 2024-09-15T22:30:07.683 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:30:07.683 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:07.683 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T22:30:07.683 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:07.683 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:07.683 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-15 22:30:07.515259843 +0000 2024-09-15T22:30:07.683 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-15 22:25:15.045470976 +0000 2024-09-15T22:30:07.684 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-15 22:25:15.045470976 +0000 2024-09-15T22:30:07.684 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-15 22:25:15.045470976 +0000 2024-09-15T22:30:07.684 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:30:07.754 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-15T22:30:07.755 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-15T22:30:07.755 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000197777 s, 2.6 MB/s 2024-09-15T22:30:07.756 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:30:07.812 DEBUG:teuthology.orchestra.run.smithi130:> stat /dev/vg_nvme/lv_4 2024-09-15T22:30:07.868 INFO:teuthology.orchestra.run.smithi130.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout:Access: 2024-09-15 22:30:07.695256488 +0000 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout:Modify: 2024-09-15 22:25:15.375464817 +0000 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout:Change: 2024-09-15 22:25:15.375464817 +0000 2024-09-15T22:30:07.869 INFO:teuthology.orchestra.run.smithi130.stdout: Birth: 2024-09-15 22:25:15.375464817 +0000 2024-09-15T22:30:07.869 DEBUG:teuthology.orchestra.run.smithi130:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:30:07.934 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records in 2024-09-15T22:30:07.934 INFO:teuthology.orchestra.run.smithi130.stderr:1+0 records out 2024-09-15T22:30:07.934 INFO:teuthology.orchestra.run.smithi130.stderr:512 bytes copied, 0.000189088 s, 2.7 MB/s 2024-09-15T22:30:07.935 DEBUG:teuthology.orchestra.run.smithi130:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:30:07.992 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-09-15T22:30:07.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:30:08.257 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:08.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:08 smithi016 ceph-mon[31492]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:08 smithi130 ceph-mon[35742]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:08.529 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:08.404+0000 7f0589a98040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:30:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:08 smithi059 ceph-mon[34872]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:08.907 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:08.554+0000 7f0589a98040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:30:08.908 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:08.620+0000 7f0589a98040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:30:08.908 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:08.767+0000 7f0589a98040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:30:08.908 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:08.906+0000 7f0589a98040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:30:09.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:30:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:09.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:09 smithi016 ceph-mon[31492]: Deploying daemon mgr.y on smithi059 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:09.217+0000 7f0589a98040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:09.277+0000 7f0589a98040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:09.464+0000 7f0589a98040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:30:09.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:09.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:09.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:09 smithi130 ceph-mon[35742]: Deploying daemon mgr.y on smithi059 2024-09-15T22:30:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:30:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:09.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:09 smithi059 ceph-mon[34872]: Deploying daemon mgr.y on smithi059 2024-09-15T22:30:10.345 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 systemd[1]: Starting Ceph mgr.y for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:30:10.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:30:10.630 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:10 smithi059 ceph-mon[34872]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:10.630 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 podman[36228]: 2024-09-15 22:30:10.3437585 +0000 UTC m=+0.101664540 container create ac1b67580f656d4c1bafc4f567f56e4d935e7e85fdb71c692f9ddb8d55002fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:30:10.630 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 podman[36228]: 2024-09-15 22:30:10.257839412 +0000 UTC m=+0.015745466 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:30:10.630 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 podman[36228]: 2024-09-15 22:30:10.554433041 +0000 UTC m=+0.312339081 container init ac1b67580f656d4c1bafc4f567f56e4d935e7e85fdb71c692f9ddb8d55002fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T22:30:10.630 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 podman[36228]: 2024-09-15 22:30:10.557148994 +0000 UTC m=+0.315055042 container start ac1b67580f656d4c1bafc4f567f56e4d935e7e85fdb71c692f9ddb8d55002fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-15T22:30:10.630 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 ceph-mgr[36247]: -- 172.21.15.59:0/488351329 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55692f83f0e0 con 0x55692f81c800 2024-09-15T22:30:10.630 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 bash[36228]: ac1b67580f656d4c1bafc4f567f56e4d935e7e85fdb71c692f9ddb8d55002fd4 2024-09-15T22:30:10.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-09-15T22:30:10.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:10 smithi016 ceph-mon[31492]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:10 smithi130 ceph-mon[35742]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:10.779 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:10.343+0000 7f0589a98040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:30:10.779 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:10.410+0000 7f0589a98040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:30:10.900 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 systemd[1]: Started Ceph mgr.y for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:30:10.900 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:10.681+0000 7facc2191040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:30:10.900 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:10.740+0000 7facc2191040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:30:10.900 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:10.898+0000 7facc2191040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:30:10.910 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:30:11.194 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:10.968+0000 7facc2191040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:30:11.194 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:11.132+0000 7facc2191040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 -- 172.21.15.16:0/2218868273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc101d10 msgr2=0x7f32cc102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/2218868273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc101d10 0x7f32cc102190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009900 tx=0x7f32b402a410 comp rx=0 tx=0).stop 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 -- 172.21.15.16:0/2218868273 shutdown_connections 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/2218868273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32cc1026d0 0x7f32cc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/2218868273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc101d10 0x7f32cc102190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/2218868273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32cc100b10 0x7f32cc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 -- 172.21.15.16:0/2218868273 >> 172.21.15.16:0/2218868273 conn(0x7f32cc0fc2a0 msgr2=0x7f32cc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 -- 172.21.15.16:0/2218868273 shutdown_connections 2024-09-15T22:30:11.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.379+0000 7f32d0a4c640 1 -- 172.21.15.16:0/2218868273 wait complete. 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 Processor -- start 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 -- start start 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc100b10 0x7f32cc104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32cc101d10 0x7f32cc1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32cc1026d0 0x7f32cc103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10c790 con 0x7f32cc100b10 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10c610 con 0x7f32cc101d10 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32d0a4c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10c910 con 0x7f32cc1026d0 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc100b10 0x7f32cc104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc100b10 0x7f32cc104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51476/0 (socket says 172.21.15.16:51476) 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32ca59c640 1 -- 172.21.15.16:0/132469634 learned_addr learned my addr 172.21.15.16:0/132469634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32c3fff640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32cc101d10 0x7f32cc1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:11.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.380+0000 7f32cad9d640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32cc1026d0 0x7f32cc103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:11.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32ca59c640 1 -- 172.21.15.16:0/132469634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32cc1026d0 msgr2=0x7f32cc103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:11.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32ca59c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32cc1026d0 0x7f32cc103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:11.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32ca59c640 1 -- 172.21.15.16:0/132469634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32cc101d10 msgr2=0x7f32cc1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:11.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32ca59c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32cc101d10 0x7f32cc1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:11.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32ca59c640 1 -- 172.21.15.16:0/132469634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc103ef0 con 0x7f32cc100b10 2024-09-15T22:30:11.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32ca59c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc100b10 0x7f32cc104aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e9c0 tx=0x7f32bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:11.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00cd80 con 0x7f32cc100b10 2024-09-15T22:30:11.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc0035f0 con 0x7f32cc100b10 2024-09-15T22:30:11.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc010690 con 0x7f32cc100b10 2024-09-15T22:30:11.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1a2090 con 0x7f32cc100b10 2024-09-15T22:30:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.381+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a25a0 con 0x7f32cc100b10 2024-09-15T22:30:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.382+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f32bc010830 con 0x7f32cc100b10 2024-09-15T22:30:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.382+0000 7f32c37fe640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f32b80394d0 0x7f32b803b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.382+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc1a2830 con 0x7f32cc100b10 2024-09-15T22:30:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.382+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f32bc04caa0 con 0x7f32cc100b10 2024-09-15T22:30:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.382+0000 7f32c3fff640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f32b80394d0 0x7f32b803b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.383+0000 7f32c3fff640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f32b80394d0 0x7f32b803b990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32b40314e0 tx=0x7f32b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:11.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.387+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32bc0173d0 con 0x7f32cc100b10 2024-09-15T22:30:11.419 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.162+0000 7f0589a98040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:30:11.420 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.285+0000 7f0589a98040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:30:11.542 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:11.285+0000 7facc2191040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:30:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:11.572+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f32cc06aed0 con 0x7f32b80394d0 2024-09-15T22:30:11.699 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.433+0000 7f0589a98040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:30:11.699 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.497+0000 7f0589a98040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:30:11.699 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.652+0000 7f0589a98040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:30:11.888 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:11.889 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:11.620+0000 7facc2191040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:30:11.889 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:11.683+0000 7facc2191040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:30:11.889 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:11.885+0000 7facc2191040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:30:11.956 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.723+0000 7f0589a98040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.784+0000 7f0589a98040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:11.955+0000 7f0589a98040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:30:11.957 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:11.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:11.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:11.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:11.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:30:11.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:30:11.988 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:12.232 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:12.023+0000 7f0589a98040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:30:12.232 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:12.090+0000 7f0589a98040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:30:12.233 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:12.172+0000 7f0589a98040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:30:12.233 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:12.230+0000 7f0589a98040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:30:12.528 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:12.290+0000 7f0589a98040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:30:13.194 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:12.807+0000 7facc2191040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:30:13.194 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:12.876+0000 7facc2191040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:30:13.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:13.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:13.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:12 smithi059 ceph-mon[34872]: Reconfiguring daemon mgr.x on smithi016 2024-09-15T22:30:13.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:13.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:13.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:13.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:30:13.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:13.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:13.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:13.275 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:12 smithi016 ceph-mon[31492]: Reconfiguring daemon mgr.x on smithi016 2024-09-15T22:30:13.278 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:30:13 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:30:13.080+0000 7f0589a98040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:30:13.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:30:13.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:13.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:12 smithi130 ceph-mon[35742]: Reconfiguring daemon mgr.x on smithi016 2024-09-15T22:30:13.896 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:13.627+0000 7facc2191040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:30:13.896 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:13.747+0000 7facc2191040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:30:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:13.919+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7f32bc01dd20 con 0x7f32cc100b10 2024-09-15T22:30:14.191 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-mon[34872]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:14.191 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-mon[34872]: Standby manager daemon z started 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:13.895+0000 7facc2191040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:13.962+0000 7facc2191040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.119+0000 7facc2191040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:30:14.192 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.190+0000 7facc2191040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:30:14.197 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:13 smithi016 ceph-mon[31492]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:14.197 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:13 smithi016 ceph-mon[31492]: Standby manager daemon z started 2024-09-15T22:30:14.197 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:13 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:30:14.197 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:13 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:30:14.197 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:13 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:30:14.197 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:13 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:30:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:13 smithi130 ceph-mon[35742]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:13 smithi130 ceph-mon[35742]: Standby manager daemon z started 2024-09-15T22:30:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:13 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:30:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:13 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:30:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:13 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:30:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:13 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/3542671953' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:30:14.444 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.252+0000 7facc2191040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:30:14.444 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.417+0000 7facc2191040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:30:14.764 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.484+0000 7facc2191040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:30:14.764 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.554+0000 7facc2191040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:30:14.764 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.640+0000 7facc2191040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:30:14.764 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.701+0000 7facc2191040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:30:14.764 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:14.762+0000 7facc2191040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:15.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:15.920 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:30:15.586+0000 7facc2191040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3634227016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61d5e373-f70b-4535-bbb2-815a227c3404"}]: dispatch 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3634227016' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61d5e373-f70b-4535-bbb2-815a227c3404"}]': finished 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: Standby manager daemon y started 2024-09-15T22:30:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:30:16.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:30:16.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:30:16.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:30:16.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:15 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3495822333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3634227016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61d5e373-f70b-4535-bbb2-815a227c3404"}]: dispatch 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3634227016' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61d5e373-f70b-4535-bbb2-815a227c3404"}]': finished 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: Standby manager daemon y started 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:30:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:30:16.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:15 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3495822333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3634227016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61d5e373-f70b-4535-bbb2-815a227c3404"}]: dispatch 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3634227016' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61d5e373-f70b-4535-bbb2-815a227c3404"}]': finished 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: Standby manager daemon y started 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:30:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:30:16.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:30:16.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/2060162327' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:30:16.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:15 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3495822333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:30:16.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:16.532+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f32bc018ce0 con 0x7f32cc100b10 2024-09-15T22:30:17.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:17 smithi016 ceph-mon[31492]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:17.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:17 smithi016 ceph-mon[31492]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-15T22:30:17.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:30:17.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:17 smithi059 ceph-mon[34872]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:17.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:17 smithi059 ceph-mon[34872]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-15T22:30:17.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:30:18.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:17 smithi130 ceph-mon[35742]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:18.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:17 smithi130 ceph-mon[35742]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-15T22:30:18.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:30:19.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:19 smithi016 ceph-mon[31492]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:19.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:19 smithi059 ceph-mon[34872]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:19 smithi130 ceph-mon[35742]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:21.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:21 smithi016 ceph-mon[31492]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:21.831 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:21 smithi130 ceph-mon[35742]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:21.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:21 smithi059 ceph-mon[34872]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:23.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:23 smithi016 ceph-mon[31492]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:23.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:30:23.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:23.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:23 smithi016 ceph-mon[31492]: Deploying daemon osd.0 on smithi016 2024-09-15T22:30:23.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:23 smithi059 ceph-mon[34872]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:23.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:30:23.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:23.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:23 smithi059 ceph-mon[34872]: Deploying daemon osd.0 on smithi016 2024-09-15T22:30:24.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:23 smithi130 ceph-mon[35742]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:24.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:30:24.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:24.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:23 smithi130 ceph-mon[35742]: Deploying daemon osd.0 on smithi016 2024-09-15T22:30:25.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:25 smithi016 ceph-mon[31492]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:25.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:25 smithi059 ceph-mon[34872]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:25 smithi130 ceph-mon[35742]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:27.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:27 smithi016 ceph-mon[31492]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:27.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:27 smithi059 ceph-mon[34872]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:28.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:27 smithi130 ceph-mon[35742]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:29.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:29 smithi016 ceph-mon[31492]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:29.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:29 smithi059 ceph-mon[34872]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:30.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:29 smithi130 ceph-mon[35742]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:30.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:30.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:30.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:30.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:30.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:30.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:31 smithi016 ceph-mon[31492]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:32.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:31 smithi059 ceph-mon[34872]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:32.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:31 smithi130 ceph-mon[35742]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:33.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:34.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.026+0000 7f32c37fe640 1 -- 172.21.15.16:0/132469634 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f32cc06aed0 con 0x7f32b80394d0 2024-09-15T22:30:34.029 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-09-15T22:30:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f32b80394d0 msgr2=0x7f32b803b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f32b80394d0 0x7f32b803b990 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32b40314e0 tx=0x7f32b4000bc0 comp rx=0 tx=0).stop 2024-09-15T22:30:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc100b10 msgr2=0x7f32cc104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc100b10 0x7f32cc104aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e9c0 tx=0x7f32bc00ee80 comp rx=0 tx=0).stop 2024-09-15T22:30:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 shutdown_connections 2024-09-15T22:30:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f32b80394d0 0x7f32b803b990 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:34.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f32cc1026d0 0x7f32cc103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32cc101d10 0x7f32cc1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 --2- 172.21.15.16:0/132469634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32cc100b10 0x7f32cc104aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 >> 172.21.15.16:0/132469634 conn(0x7f32cc0fc2a0 msgr2=0x7f32cc06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 shutdown_connections 2024-09-15T22:30:34.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:30:34.028+0000 7f32d0a4c640 1 -- 172.21.15.16:0/132469634 wait complete. 2024-09-15T22:30:34.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:34 smithi016 ceph-mon[31492]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:34.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:34.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:34.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:34.647 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.0.service 2024-09-15T22:30:34.690 INFO:tasks.cephadm:Deploying osd.1 on smithi059 with /dev/vg_nvme/lv_4... 2024-09-15T22:30:34.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:30:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:34 smithi059 ceph-mon[34872]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:34.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:34 smithi130 ceph-mon[35742]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:34.949 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:30:36.041 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:30:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0[45374]: 2024-09-15T22:30:35.656+0000 7f5cbec35640 -1 osd.0 0 log_to_monitors true 2024-09-15T22:30:36.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:36 smithi059 ceph-mon[34872]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:36.648 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:36 smithi059 ceph-mon[34872]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:30:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:36 smithi130 ceph-mon[35742]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:36 smithi130 ceph-mon[35742]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:30:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:36 smithi016 ceph-mon[31492]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:36 smithi016 ceph-mon[31492]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:30:37.349 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:30:37.375 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2024-09-15T22:30:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-mon[31492]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:30:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-mon[31492]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:30:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-mon[31492]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:30:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-mon[31492]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:30:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-mon[31492]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:30:37.666 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:30:37.668 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:37 smithi059 ceph-mon[34872]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:30:37.668 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:37 smithi059 ceph-mon[34872]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:30:37.669 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:37 smithi059 ceph-mon[34872]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:30:37.669 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:37 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:37.669 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:37 smithi059 ceph-mon[34872]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:30:37.669 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:37 smithi059 ceph-mon[34872]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:30:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:37 smithi130 ceph-mon[35742]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:30:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:37 smithi130 ceph-mon[35742]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:30:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:37 smithi130 ceph-mon[35742]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:30:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:37 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:37 smithi130 ceph-mon[35742]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:30:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:37 smithi130 ceph-mon[35742]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:30:38.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.148+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/1225714034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4100ad0 msgr2=0x7f3df4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:38.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.148+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/1225714034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4100ad0 0x7f3df4100ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009900 tx=0x7f3df002a410 comp rx=0 tx=0).stop 2024-09-15T22:30:38.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.148+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/1225714034 shutdown_connections 2024-09-15T22:30:38.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.148+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/1225714034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3df4102690 0x7f3df4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:38.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.148+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/1225714034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4101cd0 0x7f3df4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:38.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.148+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/1225714034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4100ad0 0x7f3df4100ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:38.153 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.148+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/1225714034 >> 172.21.15.59:0/1225714034 conn(0x7f3df40fc280 msgr2=0x7f3df40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/1225714034 shutdown_connections 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/1225714034 wait complete. 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 Processor -- start 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 -- start start 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4100ad0 0x7f3df406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4101cd0 0x7f3df419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3df4102690 0x7f3df419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410c730 con 0x7f3df4100ad0 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410c5b0 con 0x7f3df4101cd0 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.149+0000 7f3dfb4a5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3df410c8b0 con 0x7f3df4102690 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4101cd0 0x7f3df419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4101cd0 0x7f3df419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49710/0 (socket says 172.21.15.59:49710) 2024-09-15T22:30:38.154 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 -- 172.21.15.59:0/2197929525 learned_addr learned my addr 172.21.15.59:0/2197929525 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df9a42640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3df4102690 0x7f3df419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df9241640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4100ad0 0x7f3df406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 -- 172.21.15.59:0/2197929525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3df4102690 msgr2=0x7f3df419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3df4102690 0x7f3df419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 -- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4100ad0 msgr2=0x7f3df406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4100ad0 0x7f3df406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 -- 172.21.15.59:0/2197929525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df419fb40 con 0x7f3df4101cd0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3df8a40640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4101cd0 0x7f3df419cc10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d950 tx=0x7f3de400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3dea7fc640 1 -- 172.21.15.59:0/2197929525 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4017070 con 0x7f3df4101cd0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3dea7fc640 1 -- 172.21.15.59:0/2197929525 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de4009c30 con 0x7f3df4101cd0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df419fdd0 con 0x7f3df4101cd0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3dea7fc640 1 -- 172.21.15.59:0/2197929525 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de400f6b0 con 0x7f3df4101cd0 2024-09-15T22:30:38.155 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.150+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df41a0280 con 0x7f3df4101cd0 2024-09-15T22:30:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.151+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df4107e70 con 0x7f3df4101cd0 2024-09-15T22:30:38.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.154+0000 7f3dea7fc640 1 -- 172.21.15.59:0/2197929525 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f3de40109d0 con 0x7f3df4101cd0 2024-09-15T22:30:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.154+0000 7f3dea7fc640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3de00a31b0 0x7f3de00a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:30:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.155+0000 7f3dea7fc640 1 -- 172.21.15.59:0/2197929525 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f3de40cda60 con 0x7f3df4101cd0 2024-09-15T22:30:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.155+0000 7f3df9241640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3de00a31b0 0x7f3de00a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:30:38.157 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.155+0000 7f3df9241640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3de00a31b0 0x7f3de00a5670 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3df0006010 tx=0x7f3df0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:30:38.159 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.157+0000 7f3dea7fc640 1 -- 172.21.15.59:0/2197929525 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3de4009de0 con 0x7f3df4101cd0 2024-09-15T22:30:38.307 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0[45374]: 2024-09-15T22:30:37.903+0000 7f5cbabb8640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T22:30:38.307 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:30:37 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0[45374]: 2024-09-15T22:30:37.907+0000 7f5cb29d8640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:30:38.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:38.340+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3df4062630 con 0x7f3de00a31b0 2024-09-15T22:30:38.683 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' 2024-09-15T22:30:38.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:30:38.685 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:30:38.685 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:38.685 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:38 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:38.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:38.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' 2024-09-15T22:30:38.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:30:38.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:30:38.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:38.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:38 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038]' entity='osd.0' 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:38.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:38 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:30:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:30:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi016 2024-09-15T22:30:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-15T22:30:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] boot 2024-09-15T22:30:39.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:30:39.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi016 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] boot 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:30:39.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi016 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: osd.0 [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] boot 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:30:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:30:40.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:40 smithi059 ceph-mon[34872]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:40.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:40 smithi130 ceph-mon[35742]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:40.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:40 smithi016 ceph-mon[31492]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:30:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:30:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/2659927679' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]: dispatch 2024-09-15T22:30:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]: dispatch 2024-09-15T22:30:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]': finished 2024-09-15T22:30:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:30:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:30:41.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:41 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/2863939970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/2659927679' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]: dispatch 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]: dispatch 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]': finished 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:30:41.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:41 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/2863939970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/2659927679' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]: dispatch 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]: dispatch 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9d5166a-3b14-4810-aa73-0fc1f5defad1"}]': finished 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:30:41.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:41 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/2863939970' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:30:43.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:43 smithi016 ceph-mon[31492]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:43.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:43 smithi059 ceph-mon[34872]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:44.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:43 smithi130 ceph-mon[35742]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:45.834 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:45 smithi059 ceph-mon[34872]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:46.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:45 smithi130 ceph-mon[35742]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:46.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:45 smithi016 ceph-mon[31492]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:47.819 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:47 smithi059 ceph-mon[34872]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:47 smithi130 ceph-mon[35742]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:47 smithi016 ceph-mon[31492]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:49.819 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:49 smithi059 ceph-mon[34872]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:49.819 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:30:49.820 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:49.820 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:49 smithi059 ceph-mon[34872]: Deploying daemon osd.1 on smithi059 2024-09-15T22:30:50.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:49 smithi130 ceph-mon[35742]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:50.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:30:50.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:50.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:49 smithi130 ceph-mon[35742]: Deploying daemon osd.1 on smithi059 2024-09-15T22:30:50.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:49 smithi016 ceph-mon[31492]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:50.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:30:50.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:50.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:49 smithi016 ceph-mon[31492]: Deploying daemon osd.1 on smithi059 2024-09-15T22:30:51.816 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:51 smithi059 ceph-mon[34872]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:51.831 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:51 smithi130 ceph-mon[35742]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:52.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:51 smithi016 ceph-mon[31492]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:53.872 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:53 smithi059 ceph-mon[34872]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:54.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:53 smithi130 ceph-mon[35742]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:54.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:53 smithi016 ceph-mon[31492]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:55.890 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:55 smithi059 ceph-mon[34872]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:55.890 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:55 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:55.891 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:55 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:55.891 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:55 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:56.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:55 smithi130 ceph-mon[35742]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:56.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:55 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:56.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:55 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:56.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:55 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:56.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:55 smithi016 ceph-mon[31492]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:56.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:55 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:30:56.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:55 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:56.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:55 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:58 smithi059 ceph-mon[34872]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:58.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:58.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:58.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:30:58 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:58 smithi130 ceph-mon[35742]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:58.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:58.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:58.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:30:58 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:58 smithi016 ceph-mon[31492]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:30:58.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:58.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:30:58.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:30:58.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:30:58 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:30:59.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.456+0000 7f3dea7fc640 1 -- 172.21.15.59:0/2197929525 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3df4062630 con 0x7f3de00a31b0 2024-09-15T22:30:59.458 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 1 on host 'smithi059' 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3de00a31b0 msgr2=0x7f3de00a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3de00a31b0 0x7f3de00a5670 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3df0006010 tx=0x7f3df0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4101cd0 msgr2=0x7f3df419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4101cd0 0x7f3df419cc10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3de400d950 tx=0x7f3de400de10 comp rx=0 tx=0).stop 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 shutdown_connections 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3de00a31b0 0x7f3de00a5670 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3df4102690 0x7f3df419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3df4101cd0 0x7f3df419cc10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 --2- 172.21.15.59:0/2197929525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df4100ad0 0x7f3df406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:30:59.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.458+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 >> 172.21.15.59:0/2197929525 conn(0x7f3df40fc280 msgr2=0x7f3df4105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:30:59.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.459+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 shutdown_connections 2024-09-15T22:30:59.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:30:59.459+0000 7f3dfb4a5640 1 -- 172.21.15.59:0/2197929525 wait complete. 2024-09-15T22:31:00.069 DEBUG:teuthology.orchestra.run.smithi059:osd.1> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.1.service 2024-09-15T22:31:00.071 INFO:tasks.cephadm:Deploying osd.2 on smithi130 with /dev/vg_nvme/lv_4... 2024-09-15T22:31:00.071 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:31:00.321 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:31:00.648 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:00 smithi130 ceph-mon[35742]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:00.648 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:00.648 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:00.648 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:00 smithi059 ceph-mon[34872]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:00.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:00 smithi016 ceph-mon[31492]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:00.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:00.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:00.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:01.341 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1[39362]: 2024-09-15T22:31:01.011+0000 7f12576b2640 -1 osd.1 0 log_to_monitors true 2024-09-15T22:31:01.669 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:01 smithi130 ceph-mon[35742]: from='osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:31:01.670 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:01 smithi130 ceph-mon[35742]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:31:01.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:01 smithi059 ceph-mon[34872]: from='osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:31:01.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:01 smithi059 ceph-mon[34872]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:31:01.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:01 smithi016 ceph-mon[31492]: from='osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:31:01.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:01 smithi016 ceph-mon[31492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:31:02.675 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:02 smithi130 ceph-mon[35742]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:02.675 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:02 smithi130 ceph-mon[35742]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:31:02.675 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:02 smithi130 ceph-mon[35742]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:31:02.675 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:02.675 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:02 smithi130 ceph-mon[35742]: from='osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:31:02.675 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:02 smithi130 ceph-mon[35742]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:31:02.677 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:31:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:02 smithi059 ceph-mon[34872]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:02 smithi059 ceph-mon[34872]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:31:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:02 smithi059 ceph-mon[34872]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:31:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:02 smithi059 ceph-mon[34872]: from='osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:31:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:02 smithi059 ceph-mon[34872]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:31:02.703 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_4 2024-09-15T22:31:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:02 smithi016 ceph-mon[31492]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:02 smithi016 ceph-mon[31492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:31:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:02 smithi016 ceph-mon[31492]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:31:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:02 smithi016 ceph-mon[31492]: from='osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:31:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:02 smithi016 ceph-mon[31492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:31:02.947 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:31:03.409 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1[39362]: 2024-09-15T22:31:03.080+0000 7f1253635640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T22:31:03.409 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1[39362]: 2024-09-15T22:31:03.086+0000 7f124b455640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.410+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/2310942351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba01078f0 msgr2=0x7f9ba0109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.410+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/2310942351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba01078f0 0x7f9ba0109d40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9b98009ca0 tx=0x7f9b9802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.410+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/2310942351 shutdown_connections 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.410+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/2310942351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba01078f0 0x7f9ba0109d40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.410+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/2310942351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ba0104fc0 0x7f9ba01073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.410+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/2310942351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba0102690 0x7f9ba0104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.410+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/2310942351 >> 172.21.15.130:0/2310942351 conn(0x7f9ba00fc270 msgr2=0x7f9ba00fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.411+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/2310942351 shutdown_connections 2024-09-15T22:31:03.419 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.411+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/2310942351 wait complete. 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.411+0000 7f9ba5cbd640 1 Processor -- start 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.411+0000 7f9ba5cbd640 1 -- start start 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba0102690 0x7f9ba0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba0104fc0 0x7f9ba01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ba01078f0 0x7f9ba019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010d900 con 0x7f9ba0104fc0 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010d780 con 0x7f9ba01078f0 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba010da80 con 0x7f9ba0102690 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba0102690 0x7f9ba0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba0102690 0x7f9ba0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:33522/0 (socket says 172.21.15.130:33522) 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 -- 172.21.15.130:0/1508203905 learned_addr learned my addr 172.21.15.130:0/1508203905 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:31:03.420 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9effd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba0104fc0 0x7f9ba01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9ffff640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ba01078f0 0x7f9ba019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 -- 172.21.15.130:0/1508203905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ba01078f0 msgr2=0x7f9ba019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ba01078f0 0x7f9ba019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 -- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba0104fc0 msgr2=0x7f9ba01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba0104fc0 0x7f9ba01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 -- 172.21.15.130:0/1508203905 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba019fbb0 con 0x7f9ba0102690 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9f7fe640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba0102690 0x7f9ba0197f80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d950 tx=0x7f9b9000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9cff9640 1 -- 172.21.15.130:0/1508203905 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90017070 con 0x7f9ba0102690 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9cff9640 1 -- 172.21.15.130:0/1508203905 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b90009c30 con 0x7f9ba0102690 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba019fea0 con 0x7f9ba0102690 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9b9cff9640 1 -- 172.21.15.130:0/1508203905 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000f6b0 con 0x7f9ba0102690 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.412+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba01a02f0 con 0x7f9ba0102690 2024-09-15T22:31:03.421 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.413+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba00fdd60 con 0x7f9ba0102690 2024-09-15T22:31:03.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.414+0000 7f9b9cff9640 1 -- 172.21.15.130:0/1508203905 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f9b900109d0 con 0x7f9ba0102690 2024-09-15T22:31:03.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.415+0000 7f9b9cff9640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9b880a31e0 0x7f9b880a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:03.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.415+0000 7f9b9cff9640 1 -- 172.21.15.130:0/1508203905 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f9b900cdad0 con 0x7f9ba0102690 2024-09-15T22:31:03.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.416+0000 7f9b9effd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9b880a31e0 0x7f9b880a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:03.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.416+0000 7f9b9effd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9b880a31e0 0x7f9b880a56a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ba01994a0 tx=0x7f9b94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:03.422 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.420+0000 7f9b9cff9640 1 -- 172.21.15.130:0/1508203905 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b9009bb10 con 0x7f9ba0102690 2024-09-15T22:31:03.595 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:03.594+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9ba0062630 con 0x7f9b880a31e0 2024-09-15T22:31:03.595 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.596 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:03 smithi130 ceph-mon[35742]: from='osd.1 ' entity='osd.1' 2024-09-15T22:31:03.674 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.675 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:03 smithi059 ceph-mon[34872]: from='osd.1 ' entity='osd.1' 2024-09-15T22:31:03.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:03.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:03 smithi016 ceph-mon[31492]: from='osd.1 ' entity='osd.1' 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi059 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi059 to 17092M 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:04.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:04.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] boot 2024-09-15T22:31:04.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:31:04.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:31:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:31:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi059 2024-09-15T22:31:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi059 to 17092M 2024-09-15T22:31:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:04.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:04.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:04.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:04.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] boot 2024-09-15T22:31:04.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:31:04.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:31:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:31:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:31:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi059 2024-09-15T22:31:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi059 to 17092M 2024-09-15T22:31:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:04.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:04.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:04.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:04.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: osd.1 [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] boot 2024-09-15T22:31:04.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:31:04.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:31:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:05 smithi059 ceph-mon[34872]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:05.708 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:05 smithi016 ceph-mon[31492]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:05 smithi130 ceph-mon[35742]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: pgmap v60: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/1238295266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]: dispatch 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]: dispatch 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]': finished 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:06 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/4128020316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: pgmap v60: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:31:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/1238295266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]: dispatch 2024-09-15T22:31:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]: dispatch 2024-09-15T22:31:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]': finished 2024-09-15T22:31:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:31:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:06.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:06 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/4128020316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: pgmap v60: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/1238295266' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]: dispatch 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]: dispatch 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad8259e8-4042-4f91-8ca4-10bb8d40315a"}]': finished 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:06.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:06 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/4128020316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:08 smithi059 ceph-mon[34872]: pgmap v63: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:08.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:08 smithi130 ceph-mon[35742]: pgmap v63: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:08.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:08 smithi016 ceph-mon[31492]: pgmap v63: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:10.645 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:10 smithi130 ceph-mon[35742]: pgmap v64: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:10.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:10 smithi059 ceph-mon[34872]: pgmap v64: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:10.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:10 smithi016 ceph-mon[31492]: pgmap v64: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:12.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:12 smithi059 ceph-mon[34872]: pgmap v65: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:12.742 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:12 smithi130 ceph-mon[35742]: pgmap v65: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:12 smithi016 ceph-mon[31492]: pgmap v65: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:14.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:14 smithi059 ceph-mon[34872]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:14.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:31:14.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:14.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:14 smithi130 ceph-mon[35742]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:14.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:31:14.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:14 smithi016 ceph-mon[31492]: pgmap v66: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:31:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:15.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:15 smithi130 ceph-mon[35742]: Deploying daemon osd.2 on smithi130 2024-09-15T22:31:15.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:15 smithi059 ceph-mon[34872]: Deploying daemon osd.2 on smithi130 2024-09-15T22:31:15.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:15 smithi016 ceph-mon[31492]: Deploying daemon osd.2 on smithi130 2024-09-15T22:31:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:16 smithi059 ceph-mon[34872]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:16.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:16 smithi130 ceph-mon[35742]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:16.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:16 smithi016 ceph-mon[31492]: pgmap v67: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:18.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:18 smithi059 ceph-mon[34872]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:18.719 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:18 smithi130 ceph-mon[35742]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:18.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:18 smithi016 ceph-mon[31492]: pgmap v68: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:20.507 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:20 smithi130 ceph-mon[35742]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:20.508 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:20 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:20.508 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:20 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:20.508 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:20 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:20 smithi059 ceph-mon[34872]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:20 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:20 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:20.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:20 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:20.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:20 smithi016 ceph-mon[31492]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:20 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:20 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:20 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:21.620 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:21 smithi130 ceph-mon[35742]: pgmap v70: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:21.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:21 smithi016 ceph-mon[31492]: pgmap v70: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:21.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:21 smithi059 ceph-mon[34872]: pgmap v70: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:23.411 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.411 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.411 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:23.411 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:23.411 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:23.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:23.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:23.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:23.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:23.726 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.725+0000 7f9b9cff9640 1 -- 172.21.15.130:0/1508203905 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9ba0062630 con 0x7f9b880a31e0 2024-09-15T22:31:23.727 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 2 on host 'smithi130' 2024-09-15T22:31:23.729 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.727+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9b880a31e0 msgr2=0x7f9b880a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:23.729 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.727+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9b880a31e0 0x7f9b880a56a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ba01994a0 tx=0x7f9b94005f90 comp rx=0 tx=0).stop 2024-09-15T22:31:23.729 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.727+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba0102690 msgr2=0x7f9ba0197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.727+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba0102690 0x7f9ba0197f80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b9000d950 tx=0x7f9b9000de10 comp rx=0 tx=0).stop 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 shutdown_connections 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9b880a31e0 0x7f9b880a56a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ba01078f0 0x7f9ba019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ba0104fc0 0x7f9ba01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 --2- 172.21.15.130:0/1508203905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ba0102690 0x7f9ba0197f80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 >> 172.21.15.130:0/1508203905 conn(0x7f9ba00fc270 msgr2=0x7f9ba0102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 shutdown_connections 2024-09-15T22:31:23.730 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:31:23.728+0000 7f9ba5cbd640 1 -- 172.21.15.130:0/1508203905 wait complete. 2024-09-15T22:31:24.328 DEBUG:teuthology.orchestra.run.smithi130:osd.2> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.2.service 2024-09-15T22:31:24.331 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-09-15T22:31:24.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:31:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:24 smithi059 ceph-mon[34872]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:24.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:24 smithi130 ceph-mon[35742]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:24.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:24.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:24.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:24.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:24 smithi016 ceph-mon[31492]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:24.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:24.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:24.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:26.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:26 smithi059 ceph-mon[34872]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:26.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:26 smithi016 ceph-mon[31492]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:26 smithi130 ceph-mon[35742]: pgmap v72: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:26.529 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2[40701]: 2024-09-15T22:31:26.293+0000 7fe68d0f8640 -1 osd.2 0 log_to_monitors true 2024-09-15T22:31:27.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:31:27.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-09-15T22:31:27.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:27 smithi130 ceph-mon[35742]: from='osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:31:27.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:27 smithi130 ceph-mon[35742]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:31:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:31:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:27 smithi016 ceph-mon[31492]: from='osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:31:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:27 smithi016 ceph-mon[31492]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:31:27.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:27 smithi059 ceph-mon[34872]: from='osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:31:27.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:27 smithi059 ceph-mon[34872]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 -- 172.21.15.16:0/2871531349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f0104fd0 msgr2=0x7f21f01073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/2871531349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f0104fd0 0x7f21f01073c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009900 tx=0x7f21d802a410 comp rx=0 tx=0).stop 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 -- 172.21.15.16:0/2871531349 shutdown_connections 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/2871531349 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21f0107900 0x7f21f0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/2871531349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f0104fd0 0x7f21f01073c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/2871531349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f01026a0 0x7f21f0104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 -- 172.21.15.16:0/2871531349 >> 172.21.15.16:0/2871531349 conn(0x7f21f00fc2a0 msgr2=0x7f21f00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:28.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 -- 172.21.15.16:0/2871531349 shutdown_connections 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 -- 172.21.15.16:0/2871531349 wait complete. 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 Processor -- start 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.262+0000 7f21f5aa2640 1 -- start start 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21f5aa2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f01026a0 0x7f21f0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21f5aa2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f0104fd0 0x7f21f0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21f5aa2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21f0107900 0x7f21f019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21f5aa2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f21f010d8a0 con 0x7f21f01026a0 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21f5aa2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21f010d720 con 0x7f21f0104fd0 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21f5aa2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f21f010da20 con 0x7f21f0107900 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f01026a0 0x7f21f0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f01026a0 0x7f21f0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38132/0 (socket says 172.21.15.16:38132) 2024-09-15T22:31:28.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 -- 172.21.15.16:0/3908857418 learned_addr learned my addr 172.21.15.16:0/3908857418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21effff640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21f0107900 0x7f21f019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21e6ffd640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f0104fd0 0x7f21f0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 -- 172.21.15.16:0/3908857418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21f0107900 msgr2=0x7f21f019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21f0107900 0x7f21f019f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 -- 172.21.15.16:0/3908857418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f0104fd0 msgr2=0x7f21f0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f0104fd0 0x7f21f0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 -- 172.21.15.16:0/3908857418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f019fb80 con 0x7f21f01026a0 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ef7fe640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f01026a0 0x7f21f0197f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d950 tx=0x7f21e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0017070 con 0x7f21f01026a0 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.263+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f019fe70 con 0x7f21f01026a0 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.264+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e0009c30 con 0x7f21f01026a0 2024-09-15T22:31:28.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.264+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000f6b0 con 0x7f21f01026a0 2024-09-15T22:31:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.264+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01a02c0 con 0x7f21f01026a0 2024-09-15T22:31:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.265+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f00fdd90 con 0x7f21f01026a0 2024-09-15T22:31:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.266+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f21e00109d0 con 0x7f21f01026a0 2024-09-15T22:31:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.267+0000 7f21ed7fa640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f21dc0a31e0 0x7f21dc0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.267+0000 7f21e6ffd640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f21dc0a31e0 0x7f21dc0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.267+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f21e00cdca0 con 0x7f21f01026a0 2024-09-15T22:31:28.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.267+0000 7f21e6ffd640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f21dc0a31e0 0x7f21dc0a56a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21d8005ab0 tx=0x7f21d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:28.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.271+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21e009bac0 con 0x7f21f01026a0 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi130 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:28.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi130 to 17088M 2024-09-15T22:31:28.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:28.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:28.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:28.458+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f21f0062630 con 0x7f21dc0a31e0 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi130 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:28.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi130 to 17088M 2024-09-15T22:31:28.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:28.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:28.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:31:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:31:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:31:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:31:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi130 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi130 to 17088M 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:28.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:29.206 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2[40701]: 2024-09-15T22:31:28.823+0000 7fe68907b640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T22:31:29.207 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2[40701]: 2024-09-15T22:31:28.827+0000 7fe680e9b640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:29 smithi130 ceph-mon[35742]: from='osd.2 ' entity='osd.2' 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:29 smithi016 ceph-mon[31492]: from='osd.2 ' entity='osd.2' 2024-09-15T22:31:29.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:31:29.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:31:29.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:29.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:29.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:29.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:29.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:29.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:29 smithi059 ceph-mon[34872]: from='osd.2 ' entity='osd.2' 2024-09-15T22:31:30.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:31:30.496 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:31:30.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:30.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:30.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:30.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] boot 2024-09-15T22:31:30.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:31:30.497 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] boot 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:31:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: osd.2 [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] boot 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:31:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:31:31.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:31:31.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:31 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/610951161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157fd25f-1750-4f7e-8745-5131ea71a1a4"}]: dispatch 2024-09-15T22:31:31.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:31:31.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:31 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/610951161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "157fd25f-1750-4f7e-8745-5131ea71a1a4"}]': finished 2024-09-15T22:31:31.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:31 smithi016 ceph-mon[31492]: osdmap e19: 4 total, 3 up, 4 in 2024-09-15T22:31:31.527 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:31.528 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:31:31.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:31:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:31 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/610951161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157fd25f-1750-4f7e-8745-5131ea71a1a4"}]: dispatch 2024-09-15T22:31:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:31:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:31 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/610951161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "157fd25f-1750-4f7e-8745-5131ea71a1a4"}]': finished 2024-09-15T22:31:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:31 smithi130 ceph-mon[35742]: osdmap e19: 4 total, 3 up, 4 in 2024-09-15T22:31:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:31:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:31:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:31 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/610951161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157fd25f-1750-4f7e-8745-5131ea71a1a4"}]: dispatch 2024-09-15T22:31:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:31:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:31 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/610951161' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "157fd25f-1750-4f7e-8745-5131ea71a1a4"}]': finished 2024-09-15T22:31:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:31 smithi059 ceph-mon[34872]: osdmap e19: 4 total, 3 up, 4 in 2024-09-15T22:31:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:31:32.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:32 smithi130 ceph-mon[35742]: pgmap v78: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:32.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:32 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2967021287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:32.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:32 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:31:32.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:32 smithi130 ceph-mon[35742]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:31:32.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:32 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:32.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:32 smithi016 ceph-mon[31492]: pgmap v78: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:32.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:32 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2967021287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:32.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:31:32.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:32 smithi016 ceph-mon[31492]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:31:32.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:32 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:32 smithi059 ceph-mon[34872]: pgmap v78: 0 pgs: ; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:32 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2967021287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:32 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:31:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:32 smithi059 ceph-mon[34872]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:31:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:32 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:33.057 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:32 smithi016 sudo[50047]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:31:33.057 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:32 smithi016 sudo[50047]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:33.057 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:32 smithi016 sudo[50047]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:33.057 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:32 smithi016 sudo[50047]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:33.220+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f21e01031b0 con 0x7f21f01026a0 2024-09-15T22:31:33.556 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:33 smithi016 sudo[50234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:31:33.557 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:33 smithi016 sudo[50234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:33.557 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:33 smithi016 sudo[50234]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:33.557 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:31:33 smithi016 sudo[50234]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:33.757 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43506]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:31:33.757 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43506]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:33.757 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43506]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:33.757 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43506]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:34.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43509]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:31:34.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43509]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:34.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43509]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:34.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:31:33 smithi059 sudo[43509]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:34.279 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44780]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:31:34.279 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44780]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:34.279 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44780]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:34.279 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44780]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:34.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:34 smithi130 ceph-mon[35742]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:34.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:34 smithi130 ceph-mon[35742]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:31:34.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:34.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:34 smithi130 ceph-mon[35742]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:31:34.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:34.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:34.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 ceph-mon[31492]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:34.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 ceph-mon[31492]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:31:34.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:34.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 ceph-mon[31492]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:31:34.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:34.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:34 smithi059 ceph-mon[34872]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:34 smithi059 ceph-mon[34872]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:31:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:34 smithi059 ceph-mon[34872]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:31:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:34.779 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44783]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:31:34.780 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44783]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:34.780 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44783]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:34.780 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:31:34 smithi130 sudo[44783]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 sudo[50797]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:31:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 sudo[50797]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 sudo[50797]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:34 smithi016 sudo[50797]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:35.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:35 smithi016 sudo[50986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:31:35.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:35 smithi016 sudo[50986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:35.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:35 smithi016 sudo[50986]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:35.336 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:35 smithi016 sudo[50986]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:35.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44786]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:31:35.714 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44786]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:35.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44786]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:35.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44786]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44789]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:31:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44789]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44789]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:35 smithi130 sudo[44789]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:36.241 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:35 smithi059 sudo[43512]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:31:36.242 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:35 smithi059 sudo[43512]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:36.242 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:35 smithi059 sudo[43512]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:36.242 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:35 smithi059 sudo[43512]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:36.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:36.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:36.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:36.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:36.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 sudo[43515]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 sudo[43515]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 sudo[43515]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 sudo[43515]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:36.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:36.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:36.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:36.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:36.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:37 smithi130 ceph-mon[35742]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:37.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:37.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:37 smithi016 ceph-mon[31492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 102 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:31:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:37 smithi059 ceph-mon[34872]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:31:38.434 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:38 smithi016 ceph-mon[31492]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:38.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:38 smithi130 ceph-mon[35742]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:38 smithi059 ceph-mon[34872]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:39.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:31:39.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:39.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:31:39.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:31:39.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:40.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:40 smithi130 ceph-mon[35742]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:40.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:40 smithi130 ceph-mon[35742]: Deploying daemon osd.3 on smithi016 2024-09-15T22:31:40.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:40 smithi016 ceph-mon[31492]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:40.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:40 smithi016 ceph-mon[31492]: Deploying daemon osd.3 on smithi016 2024-09-15T22:31:40.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:40 smithi059 ceph-mon[34872]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 103 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:40.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:40 smithi059 ceph-mon[34872]: Deploying daemon osd.3 on smithi016 2024-09-15T22:31:42.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:42 smithi130 ceph-mon[35742]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-15T22:31:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:42 smithi016 ceph-mon[31492]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-15T22:31:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:42 smithi059 ceph-mon[34872]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-15T22:31:44.361 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:44 smithi016 ceph-mon[31492]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-15T22:31:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:44 smithi130 ceph-mon[35742]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-15T22:31:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:44 smithi059 ceph-mon[34872]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-15T22:31:46.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:46 smithi016 ceph-mon[31492]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-15T22:31:46.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:46 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:46.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:46 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:46.431 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:46 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:46 smithi130 ceph-mon[35742]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-15T22:31:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:46 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:46 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:46 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:46 smithi059 ceph-mon[34872]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-15T22:31:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:46 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:46 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:46 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:48.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:48 smithi130 ceph-mon[35742]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:48.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:48 smithi016 ceph-mon[31492]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:48.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:48 smithi059 ceph-mon[34872]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:49.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:49.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:49.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:49.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:49.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.759+0000 7f21ed7fa640 1 -- 172.21.15.16:0/3908857418 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f21f0062630 con 0x7f21dc0a31e0 2024-09-15T22:31:49.761 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f21dc0a31e0 msgr2=0x7f21dc0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f21dc0a31e0 0x7f21dc0a56a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f21d8005ab0 tx=0x7f21d8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f01026a0 msgr2=0x7f21f0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f01026a0 0x7f21f0197f50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f21e000d950 tx=0x7f21e000de10 comp rx=0 tx=0).stop 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 shutdown_connections 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f21dc0a31e0 0x7f21dc0a56a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f21f0107900 0x7f21f019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f0104fd0 0x7f21f0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 --2- 172.21.15.16:0/3908857418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f01026a0 0x7f21f0197f50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 >> 172.21.15.16:0/3908857418 conn(0x7f21f00fc2a0 msgr2=0x7f21f0103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 shutdown_connections 2024-09-15T22:31:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:31:49.761+0000 7f21f5aa2640 1 -- 172.21.15.16:0/3908857418 wait complete. 2024-09-15T22:31:49.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:49.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:49.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:49.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:50.411 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.3.service 2024-09-15T22:31:50.413 INFO:tasks.cephadm:Deploying osd.4 on smithi059 with /dev/vg_nvme/lv_3... 2024-09-15T22:31:50.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:31:50.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:50 smithi016 ceph-mon[31492]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:50.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:50 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:50.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:50 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:50.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:50 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:50.672 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:31:50.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:50 smithi059 ceph-mon[34872]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:50.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:50 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:50.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:50 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:50.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:50 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:50.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:50 smithi130 ceph-mon[35742]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:50.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:50 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:31:50.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:50 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:50.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:50 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:50.972 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:31:50 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:31:50.749+0000 7f06a9aca640 -1 osd.3 0 log_to_monitors true 2024-09-15T22:31:51.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:51 smithi016 ceph-mon[31492]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:31:51.680 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:51 smithi130 ceph-mon[35742]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:31:51.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:51 smithi059 ceph-mon[34872]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:31:52.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:52 smithi016 ceph-mon[31492]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:52.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:52 smithi016 ceph-mon[31492]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:31:52.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:52 smithi016 ceph-mon[31492]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:31:52.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:52 smithi016 ceph-mon[31492]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:31:52.693 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:52 smithi059 ceph-mon[34872]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:52 smithi059 ceph-mon[34872]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:31:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:52 smithi059 ceph-mon[34872]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:31:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:52 smithi059 ceph-mon[34872]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:31:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:52.701 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:52 smithi130 ceph-mon[35742]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:31:52.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:52 smithi130 ceph-mon[35742]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:31:52.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:52 smithi130 ceph-mon[35742]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:31:52.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:52 smithi130 ceph-mon[35742]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:31:52.702 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:53.222 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:31:53.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2024-09-15T22:31:53.307 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:31:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:31:52.961+0000 7f06a5a4d640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T22:31:53.307 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:31:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:31:52.967+0000 7f069d86d640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:31:53.503 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:31:53.627 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:53 smithi059 ceph-mon[34872]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:31:53.628 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:53 smithi059 ceph-mon[34872]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:31:53.628 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:53.628 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:53.628 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:53 smithi059 ceph-mon[34872]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' 2024-09-15T22:31:53.628 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:53.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:53 smithi130 ceph-mon[35742]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:31:53.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:53 smithi130 ceph-mon[35742]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:31:53.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:53.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:53.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:53 smithi130 ceph-mon[35742]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' 2024-09-15T22:31:53.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:53.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:53 smithi016 ceph-mon[31492]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:31:53.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:53 smithi016 ceph-mon[31492]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:31:53.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:53.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:53.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:53 smithi016 ceph-mon[31492]: from='osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164]' entity='osd.3' 2024-09-15T22:31:53.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 -- 172.21.15.59:0/63712727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b4102e00 msgr2=0x7f94b4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 --2- 172.21.15.59:0/63712727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b4102e00 0x7f94b4103280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009900 tx=0x7f94a802a410 comp rx=0 tx=0).stop 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 -- 172.21.15.59:0/63712727 shutdown_connections 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 --2- 172.21.15.59:0/63712727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b41037c0 0x7f94b410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 --2- 172.21.15.59:0/63712727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b4102e00 0x7f94b4103280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 --2- 172.21.15.59:0/63712727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4101c00 0x7f94b4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 -- 172.21.15.59:0/63712727 >> 172.21.15.59:0/63712727 conn(0x7f94b40fd390 msgr2=0x7f94b40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 -- 172.21.15.59:0/63712727 shutdown_connections 2024-09-15T22:31:53.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.979+0000 7f94b99ac640 1 -- 172.21.15.59:0/63712727 wait complete. 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 Processor -- start 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 -- start start 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b4101c00 0x7f94b4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4102e00 0x7f94b41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b41037c0 0x7f94b419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94b410d900 con 0x7f94b4101c00 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f94b410d780 con 0x7f94b41037c0 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b99ac640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f94b410da80 con 0x7f94b4102e00 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b37fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b41037c0 0x7f94b419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b37fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b41037c0 0x7f94b419f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43696/0 (socket says 172.21.15.59:43696) 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b37fe640 1 -- 172.21.15.59:0/2269258806 learned_addr learned my addr 172.21.15.59:0/2269258806 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.980+0000 7f94b2ffd640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b4101c00 0x7f94b4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b27fc640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4102e00 0x7f94b41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b37fe640 1 -- 172.21.15.59:0/2269258806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4102e00 msgr2=0x7f94b41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b37fe640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4102e00 0x7f94b41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b37fe640 1 -- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b4101c00 msgr2=0x7f94b4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b37fe640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b4101c00 0x7f94b4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b37fe640 1 -- 172.21.15.59:0/2269258806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b419fbb0 con 0x7f94b41037c0 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b27fc640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4102e00 0x7f94b41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b37fe640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b41037c0 0x7f94b419f540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00eaa0 tx=0x7f94ac00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b89aa640 1 -- 172.21.15.59:0/2269258806 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac004d20 con 0x7f94b41037c0 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b89aa640 1 -- 172.21.15.59:0/2269258806 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ac005020 con 0x7f94b41037c0 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b89aa640 1 -- 172.21.15.59:0/2269258806 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac0116d0 con 0x7f94b41037c0 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b419fea0 con 0x7f94b41037c0 2024-09-15T22:31:53.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.981+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41a03d0 con 0x7f94b41037c0 2024-09-15T22:31:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.982+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b4108fa0 con 0x7f94b41037c0 2024-09-15T22:31:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.983+0000 7f94b89aa640 1 -- 172.21.15.59:0/2269258806 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f94ac0051d0 con 0x7f94b41037c0 2024-09-15T22:31:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.984+0000 7f94b89aa640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f949c0a32f0 0x7f949c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:31:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.984+0000 7f94b89aa640 1 -- 172.21.15.59:0/2269258806 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7f94ac0ce860 con 0x7f94b41037c0 2024-09-15T22:31:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.984+0000 7f94b2ffd640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f949c0a32f0 0x7f949c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:31:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.985+0000 7f94b2ffd640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f949c0a32f0 0x7f949c0a57b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f94b4102c60 tx=0x7f94a4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:31:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:53.989+0000 7f94b89aa640 1 -- 172.21.15.59:0/2269258806 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94ac016030 con 0x7f94b41037c0 2024-09-15T22:31:54.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:31:54.174+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f94b4062630 con 0x7f949c0a32f0 2024-09-15T22:31:54.730 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi016 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] boot 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:31:54.731 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:54.732 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:54 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:31:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:31:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi016 2024-09-15T22:31:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] boot 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:54.781 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:54.781 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:54 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:54.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:31:54.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:31:54.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:31:54.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi016 2024-09-15T22:31:54.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:54.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:54.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:31:54.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-15T22:31:54.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: osd.3 [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] boot 2024-09-15T22:31:54.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:31:54.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:31:54.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:54.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:31:54.809 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:31:54.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:31:54.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:31:54.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:31:54.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:31:54.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:54 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:31:56.165 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:56 smithi059 ceph-mon[34872]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:31:56.165 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:56 smithi059 ceph-mon[34872]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:31:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:56 smithi130 ceph-mon[35742]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:31:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:56 smithi130 ceph-mon[35742]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:31:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:56 smithi016 ceph-mon[31492]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:31:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:56 smithi016 ceph-mon[31492]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:31:57.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: osdmap e27: 4 total, 4 up, 4 in 2024-09-15T22:31:57.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:31:57.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/2363023126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]: dispatch 2024-09-15T22:31:57.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]: dispatch 2024-09-15T22:31:57.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]': finished 2024-09-15T22:31:57.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:31:57.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:31:57.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:57 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/1502101026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: osdmap e27: 4 total, 4 up, 4 in 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/2363023126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]: dispatch 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]: dispatch 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]': finished 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:31:57.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:57 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/1502101026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: osdmap e27: 4 total, 4 up, 4 in 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/2363023126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]: dispatch 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]: dispatch 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b8e735-5627-4a56-b82c-8a9477c44fa9"}]': finished 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:31:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:57 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/1502101026' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:31:59.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:31:59 smithi059 ceph-mon[34872]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:00.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:31:59 smithi130 ceph-mon[35742]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:00.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:31:59 smithi016 ceph-mon[31492]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:01.686 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:01 smithi059 ceph-mon[34872]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:32:02.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:01 smithi130 ceph-mon[35742]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:32:02.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:01 smithi016 ceph-mon[31492]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:32:03.775 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:03 smithi059 ceph-mon[34872]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:32:03.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:03 smithi130 ceph-mon[35742]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:32:04.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:03 smithi016 ceph-mon[31492]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:32:05.619 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:05 smithi059 ceph-mon[34872]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-15T22:32:05.620 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:05 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:32:05.620 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:05 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:05.620 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:05 smithi059 ceph-mon[34872]: Deploying daemon osd.4 on smithi059 2024-09-15T22:32:05.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:05 smithi016 ceph-mon[31492]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-15T22:32:05.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:05 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:32:05.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:05 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:05.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:05 smithi016 ceph-mon[31492]: Deploying daemon osd.4 on smithi059 2024-09-15T22:32:06.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:05 smithi130 ceph-mon[35742]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-15T22:32:06.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:05 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:32:06.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:05 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:06.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:05 smithi130 ceph-mon[35742]: Deploying daemon osd.4 on smithi059 2024-09-15T22:32:07.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:07 smithi059 ceph-mon[34872]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:32:08.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:07 smithi130 ceph-mon[35742]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:32:08.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:07 smithi016 ceph-mon[31492]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:32:09.837 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:09 smithi059 ceph-mon[34872]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:32:10.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:09 smithi130 ceph-mon[35742]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:32:10.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:09 smithi016 ceph-mon[31492]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:32:11.902 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:11 smithi059 ceph-mon[34872]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:32:11.903 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:11.903 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:11.903 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:12.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:11 smithi130 ceph-mon[35742]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:32:12.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:12.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:12.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:12.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:11 smithi016 ceph-mon[31492]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:32:12.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:12.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:12.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:13.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:13 smithi059 ceph-mon[34872]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:14.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:13 smithi130 ceph-mon[35742]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:14.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:13 smithi016 ceph-mon[31492]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:14.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:14.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:14 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:14.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.987+0000 7f94b89aa640 1 -- 172.21.15.59:0/2269258806 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f94b4062630 con 0x7f949c0a32f0 2024-09-15T22:32:14.989 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 4 on host 'smithi059' 2024-09-15T22:32:14.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f949c0a32f0 msgr2=0x7f949c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:14.991 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f949c0a32f0 0x7f949c0a57b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f94b4102c60 tx=0x7f94a4009110 comp rx=0 tx=0).stop 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b41037c0 msgr2=0x7f94b419f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b41037c0 0x7f94b419f540 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00eaa0 tx=0x7f94ac00eea0 comp rx=0 tx=0).stop 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 shutdown_connections 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f949c0a32f0 0x7f949c0a57b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94b41037c0 0x7f94b419f540 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94b4102e00 0x7f94b41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 --2- 172.21.15.59:0/2269258806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94b4101c00 0x7f94b4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 >> 172.21.15.59:0/2269258806 conn(0x7f94b40fd390 msgr2=0x7f94b4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 shutdown_connections 2024-09-15T22:32:14.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:32:14.989+0000 7f94b99ac640 1 -- 172.21.15.59:0/2269258806 wait complete. 2024-09-15T22:32:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:14 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:15.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:14 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:15.630 DEBUG:teuthology.orchestra.run.smithi059:osd.4> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.4.service 2024-09-15T22:32:15.632 INFO:tasks.cephadm:Deploying osd.5 on smithi130 with /dev/vg_nvme/lv_3... 2024-09-15T22:32:15.632 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:32:15.884 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:32:15.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:15 smithi059 ceph-mon[34872]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:15.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:15 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:15.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:15 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:15.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:15 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:16.004 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:15 smithi130 ceph-mon[35742]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:16.004 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:15 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:16.005 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:15 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:16.005 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:15 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:16.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:15 smithi016 ceph-mon[31492]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:16.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:15 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:16.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:15 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:16.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:15 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:17.444 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:32:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4[46256]: 2024-09-15T22:32:17.217+0000 7fd43b7f7640 -1 osd.4 0 log_to_monitors true 2024-09-15T22:32:18.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:17 smithi130 ceph-mon[35742]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:18.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:17 smithi130 ceph-mon[35742]: from='osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:32:18.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:17 smithi130 ceph-mon[35742]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:32:18.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:17 smithi016 ceph-mon[31492]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:18.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:17 smithi016 ceph-mon[31492]: from='osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:32:18.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:17 smithi016 ceph-mon[31492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:32:18.113 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:32:18.139 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_3 2024-09-15T22:32:18.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:17 smithi059 ceph-mon[34872]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:18.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:17 smithi059 ceph-mon[34872]: from='osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:32:18.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:17 smithi059 ceph-mon[34872]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:32:18.394 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:32:18.846 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 -- 172.21.15.130:0/2803826693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc1026c0 msgr2=0x7fc6bc108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:18.846 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 --2- 172.21.15.130:0/2803826693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc1026c0 0x7fc6bc108f50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4009ca0 tx=0x7fc6b402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:32:18.846 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 -- 172.21.15.130:0/2803826693 shutdown_connections 2024-09-15T22:32:18.846 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 --2- 172.21.15.130:0/2803826693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc1026c0 0x7fc6bc108f50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:18.846 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 --2- 172.21.15.130:0/2803826693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc101d00 0x7fc6bc102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:18.846 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 --2- 172.21.15.130:0/2803826693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6bc100b00 0x7fc6bc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:18.846 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 -- 172.21.15.130:0/2803826693 >> 172.21.15.130:0/2803826693 conn(0x7fc6bc0fc290 msgr2=0x7fc6bc0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 -- 172.21.15.130:0/2803826693 shutdown_connections 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.815+0000 7fc6c149d640 1 -- 172.21.15.130:0/2803826693 wait complete. 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 Processor -- start 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 -- start start 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6bc100b00 0x7fc6bc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc101d00 0x7fc6bc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc1026c0 0x7fc6bc19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10c760 con 0x7fc6bc1026c0 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10c5e0 con 0x7fc6bc100b00 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6c149d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc6bc10c8e0 con 0x7fc6bc101d00 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc101d00 0x7fc6bc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6baffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6bc100b00 0x7fc6bc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:18.847 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6b3fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc101d00 0x7fc6bc198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:40686/0 (socket says 172.21.15.130:40686) 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6b3fff640 1 -- 172.21.15.130:0/450257715 learned_addr learned my addr 172.21.15.130:0/450257715 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.816+0000 7fc6bb7fe640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc1026c0 0x7fc6bc19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b3fff640 1 -- 172.21.15.130:0/450257715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6bc100b00 msgr2=0x7fc6bc197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b3fff640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6bc100b00 0x7fc6bc197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b3fff640 1 -- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc1026c0 msgr2=0x7fc6bc19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b3fff640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc1026c0 0x7fc6bc19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b3fff640 1 -- 172.21.15.130:0/450257715 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6bc19fb10 con 0x7fc6bc101d00 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6bb7fe640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc1026c0 0x7fc6bc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b3fff640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc101d00 0x7fc6bc198420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400d950 tx=0x7fc6a400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:18.848 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b8ff9640 1 -- 172.21.15.130:0/450257715 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4017030 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6bc19fe00 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b8ff9640 1 -- 172.21.15.130:0/450257715 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a4004020 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6b8ff9640 1 -- 172.21.15.130:0/450257715 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40047a0 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.817+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6bc1a02b0 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.818+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6bc1a05a0 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.820+0000 7fc6b8ff9640 1 -- 172.21.15.130:0/450257715 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc6a40041d0 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.821+0000 7fc6b8ff9640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc6a80a3270 0x7fc6a80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.821+0000 7fc6b8ff9640 1 -- 172.21.15.130:0/450257715 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7fc6a40ce150 con 0x7fc6bc101d00 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.821+0000 7fc6baffd640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc6a80a3270 0x7fc6a80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.821+0000 7fc6baffd640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc6a80a3270 0x7fc6a80a5730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac005fd0 tx=0x7fc6ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:18.849 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:18.825+0000 7fc6b8ff9640 1 -- 172.21.15.130:0/450257715 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6a409bf60 con 0x7fc6bc101d00 2024-09-15T22:32:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:32:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:32:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:32:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:32:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:18.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:18.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:18.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:18.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:18.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:18.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:18.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.009 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:19.007+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc6bc1a0770 con 0x7fc6a80a3270 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.009 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:19.010 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:19.010 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.010 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:19.010 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:19.010 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:19.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:19.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:19.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:19.694 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4[46256]: 2024-09-15T22:32:19.407+0000 7fd43777a640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T22:32:19.694 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4[46256]: 2024-09-15T22:32:19.413+0000 7fd42f59a640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi059 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi059 to 8546M 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:32:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:32:20.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:20.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:19 smithi130 ceph-mon[35742]: from='osd.4 ' entity='osd.4' 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi059 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi059 to 8546M 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:32:20.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:20.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:19 smithi016 ceph-mon[31492]: from='osd.4 ' entity='osd.4' 2024-09-15T22:32:20.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:20.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi059 2024-09-15T22:32:20.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi059 to 8546M 2024-09-15T22:32:20.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:32:20.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:32:20.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:20.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:20.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:32:20.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:32:20.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:32:20.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:20.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:19 smithi059 ceph-mon[34872]: from='osd.4 ' entity='osd.4' 2024-09-15T22:32:21.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:20 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:32:21.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:20 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:32:21.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:20 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:21.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:20 smithi130 ceph-mon[35742]: osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] boot 2024-09-15T22:32:21.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:20 smithi130 ceph-mon[35742]: osdmap e31: 5 total, 5 up, 5 in 2024-09-15T22:32:21.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:20 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:21.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:20 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:32:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:20 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:32:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:20 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:20 smithi016 ceph-mon[31492]: osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] boot 2024-09-15T22:32:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:20 smithi016 ceph-mon[31492]: osdmap e31: 5 total, 5 up, 5 in 2024-09-15T22:32:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:20 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:21.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:20 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:32:21.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:20 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:32:21.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:20 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:21.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:20 smithi059 ceph-mon[34872]: osd.4 [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] boot 2024-09-15T22:32:21.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:20 smithi059 ceph-mon[34872]: osdmap e31: 5 total, 5 up, 5 in 2024-09-15T22:32:21.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:20 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:32:21.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:21 smithi130 ceph-mon[35742]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:21.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:21 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/2443867637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]: dispatch 2024-09-15T22:32:21.955 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:21 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]: dispatch 2024-09-15T22:32:21.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:21 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]': finished 2024-09-15T22:32:21.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:21 smithi130 ceph-mon[35742]: osdmap e32: 6 total, 5 up, 6 in 2024-09-15T22:32:21.956 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:21 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:22.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:21 smithi016 ceph-mon[31492]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:22.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:21 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/2443867637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]: dispatch 2024-09-15T22:32:22.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:21 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]: dispatch 2024-09-15T22:32:22.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:21 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]': finished 2024-09-15T22:32:22.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:21 smithi016 ceph-mon[31492]: osdmap e32: 6 total, 5 up, 6 in 2024-09-15T22:32:22.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:21 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:21 smithi059 ceph-mon[34872]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:32:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:21 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/2443867637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]: dispatch 2024-09-15T22:32:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:21 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]: dispatch 2024-09-15T22:32:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:21 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a628733-0612-40c3-97b1-cf54a3d990d4"}]': finished 2024-09-15T22:32:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:21 smithi059 ceph-mon[34872]: osdmap e32: 6 total, 5 up, 6 in 2024-09-15T22:32:22.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:21 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:22.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:22 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/1960117251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:32:23.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:22 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/1960117251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:32:23.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:22 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/1960117251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:32:24.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:23 smithi130 ceph-mon[35742]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:24.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:23 smithi016 ceph-mon[31492]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:24.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:23 smithi059 ceph-mon[34872]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:25 smithi130 ceph-mon[35742]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:26.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:25 smithi016 ceph-mon[31492]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:26.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:25 smithi059 ceph-mon[34872]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:28.019 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:27 smithi130 ceph-mon[35742]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:27 smithi016 ceph-mon[31492]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:28.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:27 smithi059 ceph-mon[34872]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:29.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:29 smithi130 ceph-mon[35742]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:29.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:29 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:32:29.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:29 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:29.904 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:29 smithi130 ceph-mon[35742]: Deploying daemon osd.5 on smithi130 2024-09-15T22:32:30.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:29 smithi016 ceph-mon[31492]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:30.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:32:30.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:29 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:30.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:29 smithi016 ceph-mon[31492]: Deploying daemon osd.5 on smithi130 2024-09-15T22:32:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:29 smithi059 ceph-mon[34872]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:29 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:32:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:29 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:29 smithi059 ceph-mon[34872]: Deploying daemon osd.5 on smithi130 2024-09-15T22:32:31.951 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:31 smithi130 ceph-mon[35742]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:31 smithi016 ceph-mon[31492]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:32.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:31 smithi059 ceph-mon[34872]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:34.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:33 smithi130 ceph-mon[35742]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:34.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:33 smithi016 ceph-mon[31492]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:34.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:33 smithi059 ceph-mon[34872]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:35 smithi130 ceph-mon[35742]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:35 smithi016 ceph-mon[31492]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:35 smithi059 ceph-mon[34872]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:37.978 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:37 smithi130 ceph-mon[35742]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:38.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:37 smithi016 ceph-mon[31492]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:37 smithi059 ceph-mon[34872]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:39.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:39.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:39.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.496 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.496 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.497 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:39.497 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:39.497 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.538 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.536+0000 7fc6b8ff9640 1 -- 172.21.15.130:0/450257715 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc6bc1a0770 con 0x7fc6a80a3270 2024-09-15T22:32:39.538 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 5 on host 'smithi130' 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc6a80a3270 msgr2=0x7fc6a80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc6a80a3270 0x7fc6a80a5730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac005fd0 tx=0x7fc6ac00b410 comp rx=0 tx=0).stop 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc101d00 msgr2=0x7fc6bc198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc101d00 0x7fc6bc198420 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400d950 tx=0x7fc6a400de10 comp rx=0 tx=0).stop 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 shutdown_connections 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc6a80a3270 0x7fc6a80a5730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6bc1026c0 0x7fc6bc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc6bc101d00 0x7fc6bc198420 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 --2- 172.21.15.130:0/450257715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6bc100b00 0x7fc6bc197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:39.540 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 >> 172.21.15.130:0/450257715 conn(0x7fc6bc0fc290 msgr2=0x7fc6bc06a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:39.541 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 shutdown_connections 2024-09-15T22:32:39.541 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:32:39.538+0000 7fc6c149d640 1 -- 172.21.15.130:0/450257715 wait complete. 2024-09-15T22:32:39.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:39.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:39.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:39.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:40.169 DEBUG:teuthology.orchestra.run.smithi130:osd.5> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.5.service 2024-09-15T22:32:40.171 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-09-15T22:32:40.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:32:40.400 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:40 smithi130 ceph-mon[35742]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:40.401 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:40 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:40.401 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:40 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:40.401 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:40 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:40.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:40 smithi059 ceph-mon[34872]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:40.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:40 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:40.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:40 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:40.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:40 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:40.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:40 smithi016 ceph-mon[31492]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:40.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:32:40.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:40.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:41.581 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:32:41 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:32:41.368+0000 7f142e7e5640 -1 osd.5 0 log_to_monitors true 2024-09-15T22:32:42.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:42 smithi016 ceph-mon[31492]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:42.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:42 smithi016 ceph-mon[31492]: from='osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:32:42.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:42 smithi016 ceph-mon[31492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:32:42.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:42 smithi059 ceph-mon[34872]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:42.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:42 smithi059 ceph-mon[34872]: from='osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:32:42.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:42 smithi059 ceph-mon[34872]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:32:42.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:42 smithi130 ceph-mon[35742]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:42.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:42 smithi130 ceph-mon[35742]: from='osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:32:42.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:42 smithi130 ceph-mon[35742]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:32:43.105 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:32:43.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-09-15T22:32:43.279 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:32:43.165+0000 7f142a768640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T22:32:43.279 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:32:43.171+0000 7f1426590640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:32:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:32:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T22:32:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:43.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:43 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:43.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:43.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:43.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:43 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:32:43.461 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.462 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:43.462 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:32:43.462 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:43 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:32:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.860+0000 7f137d90c640 1 -- 172.21.15.16:0/140546898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13781037c0 msgr2=0x7f137810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.860+0000 7f137d90c640 1 --2- 172.21.15.16:0/140546898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13781037c0 0x7f137810a050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1370009ca0 tx=0x7f137002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.860+0000 7f137d90c640 1 -- 172.21.15.16:0/140546898 shutdown_connections 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.860+0000 7f137d90c640 1 --2- 172.21.15.16:0/140546898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13781037c0 0x7f137810a050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.860+0000 7f137d90c640 1 --2- 172.21.15.16:0/140546898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1378102e00 0x7f1378103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.860+0000 7f137d90c640 1 --2- 172.21.15.16:0/140546898 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1378101c00 0x7f1378102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 -- 172.21.15.16:0/140546898 >> 172.21.15.16:0/140546898 conn(0x7f13780fd390 msgr2=0x7f13780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 -- 172.21.15.16:0/140546898 shutdown_connections 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 -- 172.21.15.16:0/140546898 wait complete. 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 Processor -- start 2024-09-15T22:32:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 -- start start 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1378101c00 0x7f1378197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378102e00 0x7f13781983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13781037c0 0x7f137819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f137810d860 con 0x7f1378102e00 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f137810d6e0 con 0x7f1378101c00 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.861+0000 7f137d90c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f137810d9e0 con 0x7f13781037c0 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378102e00 0x7f13781983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378102e00 0x7f13781983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40080/0 (socket says 172.21.15.16:40080) 2024-09-15T22:32:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 -- 172.21.15.16:0/1364923060 learned_addr learned my addr 172.21.15.16:0/1364923060 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f1376ffd640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1378101c00 0x7f1378197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f13777fe640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13781037c0 0x7f137819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 -- 172.21.15.16:0/1364923060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13781037c0 msgr2=0x7f137819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13781037c0 0x7f137819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 -- 172.21.15.16:0/1364923060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1378101c00 msgr2=0x7f1378197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1378101c00 0x7f1378197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 -- 172.21.15.16:0/1364923060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137819fab0 con 0x7f1378102e00 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f13777fe640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13781037c0 0x7f137819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f136ffff640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378102e00 0x7f13781983c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f136000d950 tx=0x7f136000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f137c90a640 1 -- 172.21.15.16:0/1364923060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360017070 con 0x7f1378102e00 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137819fda0 con 0x7f1378102e00 2024-09-15T22:32:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f137c90a640 1 -- 172.21.15.16:0/1364923060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1360009c30 con 0x7f1378102e00 2024-09-15T22:32:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f137c90a640 1 -- 172.21.15.16:0/1364923060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136000f6b0 con 0x7f1378102e00 2024-09-15T22:32:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.862+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13781a0260 con 0x7f1378102e00 2024-09-15T22:32:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.863+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1378102000 con 0x7f1378102e00 2024-09-15T22:32:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.865+0000 7f137c90a640 1 -- 172.21.15.16:0/1364923060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f13600109d0 con 0x7f1378102e00 2024-09-15T22:32:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.866+0000 7f137c90a640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f13640a3270 0x7f13640a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:32:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.866+0000 7f1376ffd640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f13640a3270 0x7f13640a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:32:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.866+0000 7f137c90a640 1 -- 172.21.15.16:0/1364923060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7f13600ce4a0 con 0x7f1378102e00 2024-09-15T22:32:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.867+0000 7f1376ffd640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f13640a3270 0x7f13640a5730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1368005fd0 tx=0x7f136800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:32:43.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:43.870+0000 7f137c90a640 1 -- 172.21.15.16:0/1364923060 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1360009de0 con 0x7f1378102e00 2024-09-15T22:32:44.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:32:44.054+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f13780626f0 con 0x7f13640a3270 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi130 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi130 to 8544M 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:32:44.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:32:44.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:44.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] boot 2024-09-15T22:32:44.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: osdmap e35: 6 total, 6 up, 6 in 2024-09-15T22:32:44.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi130 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi130 to 8544M 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] boot 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: osdmap e35: 6 total, 6 up, 6 in 2024-09-15T22:32:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:32:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi130 2024-09-15T22:32:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi130 to 8544M 2024-09-15T22:32:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:32:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:32:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:32:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:32:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: osd.5 [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] boot 2024-09-15T22:32:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: osdmap e35: 6 total, 6 up, 6 in 2024-09-15T22:32:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:32:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:45 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:32:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:45 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:32:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:45 smithi059 ceph-mon[34872]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:32:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:45 smithi059 ceph-mon[34872]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T22:32:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:45 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:32:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:45 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:32:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:45 smithi130 ceph-mon[35742]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:32:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:45 smithi130 ceph-mon[35742]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T22:32:45.541 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:45 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:32:45.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:45 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:32:45.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:45 smithi016 ceph-mon[31492]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:32:45.542 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:45 smithi016 ceph-mon[31492]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T22:32:46.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:46 smithi059 ceph-mon[34872]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:46.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:46 smithi059 ceph-mon[34872]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:32:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:46 smithi130 ceph-mon[35742]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:46 smithi130 ceph-mon[35742]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:32:46.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:46 smithi016 ceph-mon[31492]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:46.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:46 smithi016 ceph-mon[31492]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:32:47.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:47 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/392048483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da934b1-44dc-4944-87e9-21e7b09dddfe"}]: dispatch 2024-09-15T22:32:47.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:47 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/392048483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da934b1-44dc-4944-87e9-21e7b09dddfe"}]': finished 2024-09-15T22:32:47.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:47 smithi059 ceph-mon[34872]: osdmap e38: 7 total, 6 up, 7 in 2024-09-15T22:32:47.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:32:47.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:47 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2228089625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:32:47.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:47 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/392048483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da934b1-44dc-4944-87e9-21e7b09dddfe"}]: dispatch 2024-09-15T22:32:47.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:47 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/392048483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da934b1-44dc-4944-87e9-21e7b09dddfe"}]': finished 2024-09-15T22:32:47.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:47 smithi130 ceph-mon[35742]: osdmap e38: 7 total, 6 up, 7 in 2024-09-15T22:32:47.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:32:47.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:47 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2228089625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:32:47.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:47 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/392048483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8da934b1-44dc-4944-87e9-21e7b09dddfe"}]: dispatch 2024-09-15T22:32:47.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:47 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/392048483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8da934b1-44dc-4944-87e9-21e7b09dddfe"}]': finished 2024-09-15T22:32:47.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:47 smithi016 ceph-mon[31492]: osdmap e38: 7 total, 6 up, 7 in 2024-09-15T22:32:47.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:32:47.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:47 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2228089625' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:32:48.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:48 smithi059 ceph-mon[34872]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:48.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:48 smithi059 ceph-mon[34872]: osdmap e39: 7 total, 6 up, 7 in 2024-09-15T22:32:48.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:48 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:32:48.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:48 smithi130 ceph-mon[35742]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:48.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:48 smithi130 ceph-mon[35742]: osdmap e39: 7 total, 6 up, 7 in 2024-09-15T22:32:48.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:48 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:32:48.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:48 smithi016 ceph-mon[31492]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:48.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:48 smithi016 ceph-mon[31492]: osdmap e39: 7 total, 6 up, 7 in 2024-09-15T22:32:48.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:48 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:32:50.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:50 smithi059 ceph-mon[34872]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:50.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:50 smithi130 ceph-mon[35742]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:50.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:50 smithi016 ceph-mon[31492]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 198 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:32:52.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:52 smithi059 ceph-mon[34872]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-15T22:32:52.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:52 smithi130 ceph-mon[35742]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-15T22:32:52.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:52 smithi016 ceph-mon[31492]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-15T22:32:54.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:54 smithi059 ceph-mon[34872]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-15T22:32:54.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:54 smithi130 ceph-mon[35742]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-15T22:32:54.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:54 smithi016 ceph-mon[31492]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-15T22:32:55.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:55 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:32:55.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:55 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:55.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:55 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:32:55.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:55 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:55.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:55 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:32:55.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:55 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:32:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:56 smithi059 ceph-mon[34872]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:32:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:56 smithi059 ceph-mon[34872]: Deploying daemon osd.6 on smithi016 2024-09-15T22:32:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:56 smithi130 ceph-mon[35742]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:32:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:56 smithi130 ceph-mon[35742]: Deploying daemon osd.6 on smithi016 2024-09-15T22:32:56.541 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:56 smithi016 ceph-mon[31492]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:32:56.541 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:56 smithi016 ceph-mon[31492]: Deploying daemon osd.6 on smithi016 2024-09-15T22:32:58.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:32:58 smithi059 ceph-mon[34872]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:32:58.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:32:58 smithi130 ceph-mon[35742]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:32:58.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:32:58 smithi016 ceph-mon[31492]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:33:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:00 smithi016 ceph-mon[31492]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-15T22:33:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:00 smithi130 ceph-mon[35742]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-15T22:33:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:00 smithi059 ceph-mon[34872]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-15T22:33:02.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:02 smithi016 ceph-mon[31492]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:33:02.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:02.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:02.523 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:02.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:02 smithi130 ceph-mon[35742]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:33:02.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:02 smithi059 ceph-mon[34872]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:33:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:04 smithi016 ceph-mon[31492]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:04.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:04.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:04.519 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:04 smithi130 ceph-mon[35742]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:04 smithi059 ceph-mon[34872]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.281+0000 7f137c90a640 1 -- 172.21.15.16:0/1364923060 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f13780626f0 con 0x7f13640a3270 2024-09-15T22:33:05.283 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-09-15T22:33:05.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f13640a3270 msgr2=0x7f13640a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:05.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f13640a3270 0x7f13640a5730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1368005fd0 tx=0x7f136800b410 comp rx=0 tx=0).stop 2024-09-15T22:33:05.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378102e00 msgr2=0x7f13781983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:05.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378102e00 0x7f13781983c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f136000d950 tx=0x7f136000de10 comp rx=0 tx=0).stop 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 shutdown_connections 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f13640a3270 0x7f13640a5730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13781037c0 0x7f137819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1378102e00 0x7f13781983c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 --2- 172.21.15.16:0/1364923060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1378101c00 0x7f1378197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 >> 172.21.15.16:0/1364923060 conn(0x7f13780fd390 msgr2=0x7f13781061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 shutdown_connections 2024-09-15T22:33:05.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:33:05.284+0000 7f137d90c640 1 -- 172.21.15.16:0/1364923060 wait complete. 2024-09-15T22:33:06.071 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.6.service 2024-09-15T22:33:06.073 INFO:tasks.cephadm:Deploying osd.7 on smithi059 with /dev/vg_nvme/lv_2... 2024-09-15T22:33:06.073 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:33:06.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:06 smithi016 ceph-mon[31492]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:06.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:06.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:06.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:06.336 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:33:06.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:06 smithi059 ceph-mon[34872]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:06.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:06.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:06.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:06.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:06 smithi130 ceph-mon[35742]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:06.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:06.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:06.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:07.885 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:33:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:33:07.613+0000 7f76c10ca640 -1 osd.6 0 log_to_monitors true 2024-09-15T22:33:08.293 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:08 smithi059 ceph-mon[34872]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:08.293 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:08 smithi059 ceph-mon[34872]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:33:08.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:08 smithi016 ceph-mon[31492]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:08.510 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:08 smithi016 ceph-mon[31492]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:33:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:08 smithi130 ceph-mon[35742]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:08 smithi130 ceph-mon[35742]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:33:09.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:09 smithi016 ceph-mon[31492]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:33:09.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:09 smithi016 ceph-mon[31492]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:33:09.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:09 smithi016 ceph-mon[31492]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:33:09.289 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:09.290 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:09 smithi016 ceph-mon[31492]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:33:09.309 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:33:09.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2024-09-15T22:33:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:09 smithi130 ceph-mon[35742]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:33:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:09 smithi130 ceph-mon[35742]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:33:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:09 smithi130 ceph-mon[35742]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:33:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:09 smithi130 ceph-mon[35742]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:33:09.598 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:33:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:09 smithi059 ceph-mon[34872]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:33:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:09 smithi059 ceph-mon[34872]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:33:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:09 smithi059 ceph-mon[34872]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:33:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:09 smithi059 ceph-mon[34872]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:33:10.056 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:33:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:33:09.743+0000 7f76bd04d640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T22:33:10.057 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:33:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:33:09.753+0000 7f76b4e6d640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:33:10.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 -- 172.21.15.59:0/2022486392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 msgr2=0x7fa4a0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 --2- 172.21.15.59:0/2022486392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a0103280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa494009900 tx=0x7fa49402a410 comp rx=0 tx=0).stop 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 -- 172.21.15.59:0/2022486392 shutdown_connections 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 --2- 172.21.15.59:0/2022486392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a01037c0 0x7fa4a010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 --2- 172.21.15.59:0/2022486392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a0103280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 --2- 172.21.15.59:0/2022486392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4a0101c00 0x7fa4a0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 -- 172.21.15.59:0/2022486392 >> 172.21.15.59:0/2022486392 conn(0x7fa4a00fd390 msgr2=0x7fa4a00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 -- 172.21.15.59:0/2022486392 shutdown_connections 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.050+0000 7fa4a6781640 1 -- 172.21.15.59:0/2022486392 wait complete. 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 Processor -- start 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 -- start start 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4a0101c00 0x7fa4a0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:10.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a01037c0 0x7fa4a019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a010d900 con 0x7fa4a01037c0 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a010d780 con 0x7fa4a0102e00 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a6781640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a010da80 con 0x7fa4a0101c00 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa49f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa49f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53728/0 (socket says 172.21.15.59:53728) 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa49f7fe640 1 -- 172.21.15.59:0/1670821707 learned_addr learned my addr 172.21.15.59:0/1670821707 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa4a4d1e640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a01037c0 0x7fa4a019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49ffff640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4a0101c00 0x7fa4a0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.051+0000 7fa49f7fe640 1 -- 172.21.15.59:0/1670821707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4a0101c00 msgr2=0x7fa4a0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49f7fe640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4a0101c00 0x7fa4a0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:10.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49f7fe640 1 -- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a01037c0 msgr2=0x7fa4a019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49f7fe640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a01037c0 0x7fa4a019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49f7fe640 1 -- 172.21.15.59:0/1670821707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a019fbe0 con 0x7fa4a0102e00 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49ffff640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4a0101c00 0x7fa4a0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa4a4d1e640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a01037c0 0x7fa4a019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49f7fe640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a01984f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa494009230 tx=0x7fa494004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49d7fa640 1 -- 172.21.15.59:0/1670821707 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49403f030 con 0x7fa4a0102e00 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49d7fa640 1 -- 172.21.15.59:0/1670821707 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494003ce0 con 0x7fa4a0102e00 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a019fe70 con 0x7fa4a0102e00 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa49d7fa640 1 -- 172.21.15.59:0/1670821707 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494035070 con 0x7fa4a0102e00 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.052+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a01a02c0 con 0x7fa4a0102e00 2024-09-15T22:33:10.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.053+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a0109050 con 0x7fa4a0102e00 2024-09-15T22:33:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.055+0000 7fa49d7fa640 1 -- 172.21.15.59:0/1670821707 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa494042030 con 0x7fa4a0102e00 2024-09-15T22:33:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.056+0000 7fa49d7fa640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa4880a3270 0x7fa4880a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.056+0000 7fa49d7fa640 1 -- 172.21.15.59:0/1670821707 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4685+0+0 (secure 0 0 0) 0x7fa4940ef590 con 0x7fa4a0102e00 2024-09-15T22:33:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.056+0000 7fa49ffff640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa4880a3270 0x7fa4880a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.056+0000 7fa49ffff640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa4880a3270 0x7fa4880a5730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:10.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.059+0000 7fa49d7fa640 1 -- 172.21.15.59:0/1670821707 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4940bd0e0 con 0x7fa4a0102e00 2024-09-15T22:33:10.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:10.246+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa4a0062630 con 0x7fa4880a3270 2024-09-15T22:33:10.249 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:10.249 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:33:10.249 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:10.249 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:10.249 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi016 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.250 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:10 smithi059 ceph-mon[34872]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' 2024-09-15T22:33:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:33:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi016 2024-09-15T22:33:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:10 smithi130 ceph-mon[35742]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi016 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:10.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-15T22:33:10.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:10.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:10.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:10.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:10 smithi016 ceph-mon[31492]: from='osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672]' entity='osd.6' 2024-09-15T22:33:11.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:33:11.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:33:11.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:11.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:33:11.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:33:11.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:33:11.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:11.300 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] boot 2024-09-15T22:33:11.300 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: osdmap e42: 7 total, 7 up, 7 in 2024-09-15T22:33:11.300 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] boot 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: osdmap e42: 7 total, 7 up, 7 in 2024-09-15T22:33:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: osd.6 [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] boot 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: osdmap e42: 7 total, 7 up, 7 in 2024-09-15T22:33:11.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:33:12.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:12 smithi130 ceph-mon[35742]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:12.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:12 smithi016 ceph-mon[31492]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:12.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:12 smithi059 ceph-mon[34872]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 199 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:33:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:13 smithi130 ceph-mon[35742]: osdmap e43: 7 total, 7 up, 7 in 2024-09-15T22:33:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:13 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/661790348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]: dispatch 2024-09-15T22:33:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:13 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]: dispatch 2024-09-15T22:33:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:13 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]': finished 2024-09-15T22:33:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:13 smithi130 ceph-mon[35742]: osdmap e44: 8 total, 7 up, 8 in 2024-09-15T22:33:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:13.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:13 smithi016 ceph-mon[31492]: osdmap e43: 7 total, 7 up, 7 in 2024-09-15T22:33:13.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:13 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/661790348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]: dispatch 2024-09-15T22:33:13.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:13 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]: dispatch 2024-09-15T22:33:13.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:13 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]': finished 2024-09-15T22:33:13.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:13 smithi016 ceph-mon[31492]: osdmap e44: 8 total, 7 up, 8 in 2024-09-15T22:33:13.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:13 smithi059 ceph-mon[34872]: osdmap e43: 7 total, 7 up, 7 in 2024-09-15T22:33:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:13 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/661790348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]: dispatch 2024-09-15T22:33:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:13 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]: dispatch 2024-09-15T22:33:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:13 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4d481db-97f6-4231-9da0-fc59f0dd4f27"}]': finished 2024-09-15T22:33:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:13 smithi059 ceph-mon[34872]: osdmap e44: 8 total, 7 up, 8 in 2024-09-15T22:33:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:14.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:14 smithi130 ceph-mon[35742]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:14.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:14 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/3896022614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:33:14.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:14 smithi016 ceph-mon[31492]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:14.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:14 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/3896022614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:33:14.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:14 smithi059 ceph-mon[34872]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:14.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:14 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/3896022614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:33:16.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:16 smithi130 ceph-mon[35742]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:16.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:16 smithi016 ceph-mon[31492]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:16 smithi059 ceph-mon[34872]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:18.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:18 smithi059 ceph-mon[34872]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:18.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:18 smithi130 ceph-mon[35742]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:18.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:18 smithi016 ceph-mon[31492]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:20.299 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:20 smithi059 ceph-mon[34872]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:20.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:20 smithi130 ceph-mon[35742]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:20.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:20 smithi016 ceph-mon[31492]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:21.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:21 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:33:21.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:21 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:21.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:21 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:33:21.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:21 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:21.574 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:21 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:33:21.574 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:21 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:22.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:22 smithi059 ceph-mon[34872]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:22.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:22 smithi059 ceph-mon[34872]: Deploying daemon osd.7 on smithi059 2024-09-15T22:33:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:22 smithi130 ceph-mon[35742]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:22 smithi130 ceph-mon[35742]: Deploying daemon osd.7 on smithi059 2024-09-15T22:33:22.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:22 smithi016 ceph-mon[31492]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:22.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:22 smithi016 ceph-mon[31492]: Deploying daemon osd.7 on smithi059 2024-09-15T22:33:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:24 smithi059 ceph-mon[34872]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:24.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:24 smithi130 ceph-mon[35742]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:24.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:24 smithi016 ceph-mon[31492]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:26 smithi130 ceph-mon[35742]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:26.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:26 smithi016 ceph-mon[31492]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:26.678 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:26 smithi059 ceph-mon[34872]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:28 smithi130 ceph-mon[35742]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:28 smithi016 ceph-mon[31492]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:28 smithi059 ceph-mon[34872]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:30.424 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:30 smithi059 ceph-mon[34872]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:30 smithi130 ceph-mon[35742]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:30.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:30 smithi016 ceph-mon[31492]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:30.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.777+0000 7fa49d7fa640 1 -- 172.21.15.59:0/1670821707 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4a0062630 con 0x7fa4880a3270 2024-09-15T22:33:30.780 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 7 on host 'smithi059' 2024-09-15T22:33:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa4880a3270 msgr2=0x7fa4880a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa4880a3270 0x7fa4880a5730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa490006fd0 tx=0x7fa490008040 comp rx=0 tx=0).stop 2024-09-15T22:33:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 msgr2=0x7fa4a01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a01984f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa494009230 tx=0x7fa494004360 comp rx=0 tx=0).stop 2024-09-15T22:33:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 shutdown_connections 2024-09-15T22:33:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa4880a3270 0x7fa4880a5730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4a01037c0 0x7fa4a019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4a0102e00 0x7fa4a01984f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 --2- 172.21.15.59:0/1670821707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa4a0101c00 0x7fa4a0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 >> 172.21.15.59:0/1670821707 conn(0x7fa4a00fd390 msgr2=0x7fa4a0106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 shutdown_connections 2024-09-15T22:33:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:33:30.780+0000 7fa4a6781640 1 -- 172.21.15.59:0/1670821707 wait complete. 2024-09-15T22:33:31.432 DEBUG:teuthology.orchestra.run.smithi059:osd.7> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.7.service 2024-09-15T22:33:31.435 INFO:tasks.cephadm:Deploying osd.8 on smithi130 with /dev/vg_nvme/lv_2... 2024-09-15T22:33:31.435 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:33:31.689 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:33:31.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:31 smithi059 ceph-mon[34872]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:31.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:31.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:31.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:31.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:31 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:31 smithi130 ceph-mon[35742]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:32.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:31 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:31 smithi016 ceph-mon[31492]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:31 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:33.625 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:33:33 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:33:33.291+0000 7fe222d32640 -1 osd.7 0 log_to_monitors true 2024-09-15T22:33:33.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:33 smithi059 ceph-mon[34872]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:33.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:33 smithi059 ceph-mon[34872]: from='osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:33:33.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:33 smithi059 ceph-mon[34872]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:33:34.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:33 smithi130 ceph-mon[35742]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:34.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:33 smithi130 ceph-mon[35742]: from='osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:33:34.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:33 smithi130 ceph-mon[35742]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:33:34.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:33 smithi016 ceph-mon[31492]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:34.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:33 smithi016 ceph-mon[31492]: from='osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:33:34.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:33 smithi016 ceph-mon[31492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:33:34.297 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:33:34.323 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_2 2024-09-15T22:33:34.574 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:33:34.909 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:33:34.909 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:33:34.909 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T22:33:34.909 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:34.909 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:33:34.909 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi059 2024-09-15T22:33:34.909 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi059 to 5697M 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:34.910 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:34 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:33:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:33:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T22:33:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:33:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi059 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi059 to 5697M 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:34.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:34 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:33:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:33:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T22:33:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:33:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi059 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi059 to 5697M 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:35.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:34 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:35.123 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 -- 172.21.15.130:0/1952869483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 msgr2=0x7fdb10104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 --2- 172.21.15.130:0/1952869483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10104ab0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009900 tx=0x7fdb0c02a410 comp rx=0 tx=0).stop 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 -- 172.21.15.130:0/1952869483 shutdown_connections 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 --2- 172.21.15.130:0/1952869483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10107920 0x7fdb10109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 --2- 172.21.15.130:0/1952869483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb10104ff0 0x7fdb101073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 --2- 172.21.15.130:0/1952869483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10104ab0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 -- 172.21.15.130:0/1952869483 >> 172.21.15.130:0/1952869483 conn(0x7fdb100fc280 msgr2=0x7fdb100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 -- 172.21.15.130:0/1952869483 shutdown_connections 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.116+0000 7fdb1796c640 1 -- 172.21.15.130:0/1952869483 wait complete. 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 Processor -- start 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 -- start start 2024-09-15T22:33:35.124 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10104ff0 0x7fdb10100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb10107920 0x7fdb10100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010d900 con 0x7fdb10104ff0 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010d780 con 0x7fdb10107920 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb1796c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010da80 con 0x7fdb101026c0 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb15708640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb15708640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10102020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:38692/0 (socket says 172.21.15.130:38692) 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb15708640 1 -- 172.21.15.130:0/705638343 learned_addr learned my addr 172.21.15.130:0/705638343 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.117+0000 7fdb15f09640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb10107920 0x7fdb10100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb14f07640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10104ff0 0x7fdb10100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.125 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb15708640 1 -- 172.21.15.130:0/705638343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb10107920 msgr2=0x7fdb10100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb15708640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb10107920 0x7fdb10100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb15708640 1 -- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10104ff0 msgr2=0x7fdb10100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb15708640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10104ff0 0x7fdb10100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb15708640 1 -- 172.21.15.130:0/705638343 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb10101470 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb15708640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10102020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009230 tx=0x7fdb0c031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb067fc640 1 -- 172.21.15.130:0/705638343 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c03f030 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb101a20b0 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb067fc640 1 -- 172.21.15.130:0/705638343 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb0c003ad0 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb067fc640 1 -- 172.21.15.130:0/705638343 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c035070 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.118+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb101a25e0 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.119+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb100fdd70 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.120+0000 7fdb067fc640 1 -- 172.21.15.130:0/705638343 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdb0c042030 con 0x7fdb101026c0 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.121+0000 7fdb067fc640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdafc0a3240 0x7fdafc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:33:35.126 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.121+0000 7fdb067fc640 1 -- 172.21.15.130:0/705638343 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5117+0+0 (secure 0 0 0) 0x7fdb0c0ef5c0 con 0x7fdb101026c0 2024-09-15T22:33:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.121+0000 7fdb14f07640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdafc0a3240 0x7fdafc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:33:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.122+0000 7fdb14f07640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdafc0a3240 0x7fdafc0a5700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb10101da0 tx=0x7fdb00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:33:35.127 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.124+0000 7fdb067fc640 1 -- 172.21.15.130:0/705638343 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb0c0bcf60 con 0x7fdb101026c0 2024-09-15T22:33:35.307 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:35.305+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdb10062630 con 0x7fdafc0a3240 2024-09-15T22:33:35.797 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:33:35.446+0000 7fe21f4c9640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T22:33:35.797 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:33:35.457+0000 7fe216ad5640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:33:35.848 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:35.848 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:35.849 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:35 smithi130 ceph-mon[35742]: from='osd.7 ' entity='osd.7' 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:35 smithi016 ceph-mon[31492]: from='osd.7 ' entity='osd.7' 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:35 smithi059 ceph-mon[34872]: from='osd.7 ' entity='osd.7' 2024-09-15T22:33:36.950 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:36 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:33:36.951 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:36 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:33:36.951 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:36 smithi130 ceph-mon[35742]: osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] boot 2024-09-15T22:33:36.951 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:36 smithi130 ceph-mon[35742]: osdmap e47: 8 total, 8 up, 8 in 2024-09-15T22:33:36.951 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:37.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:36 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:33:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:36 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:33:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:36 smithi016 ceph-mon[31492]: osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] boot 2024-09-15T22:33:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:36 smithi016 ceph-mon[31492]: osdmap e47: 8 total, 8 up, 8 in 2024-09-15T22:33:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:36 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:33:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:36 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:33:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:36 smithi059 ceph-mon[34872]: osd.7 [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] boot 2024-09-15T22:33:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:36 smithi059 ceph-mon[34872]: osdmap e47: 8 total, 8 up, 8 in 2024-09-15T22:33:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:33:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:37 smithi016 ceph-mon[31492]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:37 smithi016 ceph-mon[31492]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T22:33:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:37 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/2758529844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]: dispatch 2024-09-15T22:33:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:37 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]: dispatch 2024-09-15T22:33:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:37 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]': finished 2024-09-15T22:33:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:37 smithi016 ceph-mon[31492]: osdmap e49: 9 total, 8 up, 9 in 2024-09-15T22:33:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:37 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:37 smithi059 ceph-mon[34872]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:37 smithi059 ceph-mon[34872]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T22:33:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:37 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/2758529844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]: dispatch 2024-09-15T22:33:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:37 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]: dispatch 2024-09-15T22:33:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:37 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]': finished 2024-09-15T22:33:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:37 smithi059 ceph-mon[34872]: osdmap e49: 9 total, 8 up, 9 in 2024-09-15T22:33:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:37 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:38.202 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:37 smithi130 ceph-mon[35742]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:33:38.202 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:37 smithi130 ceph-mon[35742]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T22:33:38.202 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:37 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/2758529844' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]: dispatch 2024-09-15T22:33:38.202 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:37 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]: dispatch 2024-09-15T22:33:38.202 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:37 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "07aa015e-5371-433a-b080-07fef3dd8c8a"}]': finished 2024-09-15T22:33:38.202 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:37 smithi130 ceph-mon[35742]: osdmap e49: 9 total, 8 up, 9 in 2024-09-15T22:33:38.203 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:37 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:39.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:38 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/223895705' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:33:39.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:38 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/223895705' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:33:39.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:38 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/223895705' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:33:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:39 smithi059 ceph-mon[34872]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:39 smithi059 ceph-mon[34872]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T22:33:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:40.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:39 smithi130 ceph-mon[35742]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:39 smithi130 ceph-mon[35742]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T22:33:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:39 smithi016 ceph-mon[31492]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:39 smithi016 ceph-mon[31492]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T22:33:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:41.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:40 smithi059 ceph-mon[34872]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T22:33:41.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:40 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:41.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:40 smithi130 ceph-mon[35742]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T22:33:41.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:40 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:41.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:40 smithi016 ceph-mon[31492]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T22:33:41.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:40 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:33:42.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:41 smithi059 ceph-mon[34872]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:41 smithi130 ceph-mon[35742]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:42.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:41 smithi016 ceph-mon[31492]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:44.152 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:43 smithi130 ceph-mon[35742]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:44.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:43 smithi059 ceph-mon[34872]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:44.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:43 smithi016 ceph-mon[31492]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:46.102 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:45 smithi130 ceph-mon[35742]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:46.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:45 smithi059 ceph-mon[34872]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:46.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:45 smithi016 ceph-mon[31492]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:46.968 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:46 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:33:46.968 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:46 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:46.968 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:46 smithi130 ceph-mon[35742]: Deploying daemon osd.8 on smithi130 2024-09-15T22:33:47.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:46 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:33:47.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:46 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:47.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:46 smithi016 ceph-mon[31492]: Deploying daemon osd.8 on smithi130 2024-09-15T22:33:47.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:46 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:33:47.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:46 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:47.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:46 smithi059 ceph-mon[34872]: Deploying daemon osd.8 on smithi130 2024-09-15T22:33:47.985 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:47 smithi130 ceph-mon[35742]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:48.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:47 smithi059 ceph-mon[34872]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:48.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:47 smithi016 ceph-mon[31492]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:49 smithi059 ceph-mon[34872]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:49 smithi130 ceph-mon[35742]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:50.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:49 smithi016 ceph-mon[31492]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:52.007 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:51 smithi130 ceph-mon[35742]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:51 smithi059 ceph-mon[34872]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:52.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:51 smithi016 ceph-mon[31492]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:53.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:53.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:53.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:53.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:53.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:53.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:54.376 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:54 smithi059 ceph-mon[34872]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:54.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:54 smithi016 ceph-mon[31492]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:54 smithi130 ceph-mon[35742]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:56.075 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 8 on host 'smithi130' 2024-09-15T22:33:56.075 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.072+0000 7fdb067fc640 1 -- 172.21.15.130:0/705638343 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdb10062630 con 0x7fdafc0a3240 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdafc0a3240 msgr2=0x7fdafc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdafc0a3240 0x7fdafc0a5700 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb10101da0 tx=0x7fdb00005f90 comp rx=0 tx=0).stop 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 msgr2=0x7fdb10102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10102020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009230 tx=0x7fdb0c031b60 comp rx=0 tx=0).stop 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 shutdown_connections 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdafc0a3240 0x7fdafc0a5700 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb10107920 0x7fdb10100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb10104ff0 0x7fdb10100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:56.076 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 --2- 172.21.15.130:0/705638343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb101026c0 0x7fdb10102020 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:33:56.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.074+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 >> 172.21.15.130:0/705638343 conn(0x7fdb100fc280 msgr2=0x7fdb10102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:33:56.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.075+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 shutdown_connections 2024-09-15T22:33:56.077 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:33:56.075+0000 7fdb1796c640 1 -- 172.21.15.130:0/705638343 wait complete. 2024-09-15T22:33:56.386 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.387 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:33:56.733 DEBUG:teuthology.orchestra.run.smithi130:osd.8> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.8.service 2024-09-15T22:33:56.735 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-09-15T22:33:56.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:33:57.002 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:33:58.391 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:58 smithi130 ceph-mon[35742]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:58.391 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:33:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:33:58.329+0000 7f4c583ee640 -1 osd.8 0 log_to_monitors true 2024-09-15T22:33:58.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:58 smithi059 ceph-mon[34872]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:58.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:58 smithi016 ceph-mon[31492]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:33:59.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:59 smithi130 ceph-mon[35742]: from='osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:33:59.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:33:59 smithi130 ceph-mon[35742]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:33:59.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:59 smithi016 ceph-mon[31492]: from='osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:33:59.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:33:59 smithi016 ceph-mon[31492]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:33:59.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:59 smithi059 ceph-mon[34872]: from='osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:33:59.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:33:59 smithi059 ceph-mon[34872]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:33:59.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:34:00.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-09-15T22:34:00.263 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:34:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:34:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:34:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:34:00.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:00.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:00.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:00 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:00.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:00 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:00.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:00.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:00.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3172625261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 msgr2=0x7f8aac103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3172625261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac103280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009900 tx=0x7f8a9402a410 comp rx=0 tx=0).stop 2024-09-15T22:34:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3172625261 shutdown_connections 2024-09-15T22:34:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3172625261 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aac1037c0 0x7f8aac10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3172625261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac103280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3172625261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac101c00 0x7f8aac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3172625261 >> 172.21.15.16:0/3172625261 conn(0x7f8aac0fd390 msgr2=0x7f8aac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3172625261 shutdown_connections 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3172625261 wait complete. 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.777+0000 7f8ab1d22640 1 Processor -- start 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8ab1d22640 1 -- start start 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8ab1d22640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac101c00 0x7f8aac197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8ab1d22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8ab1d22640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aac1037c0 0x7f8aac19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8ab1d22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac10d860 con 0x7f8aac102e00 2024-09-15T22:34:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8ab1d22640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac10d6e0 con 0x7f8aac101c00 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8ab1d22640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac10d9e0 con 0x7f8aac1037c0 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59072/0 (socket says 172.21.15.16:59072) 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 -- 172.21.15.16:0/3554074767 learned_addr learned my addr 172.21.15.16:0/3554074767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aabfff640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aac1037c0 0x7f8aac19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aab7fe640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac101c00 0x7f8aac197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 -- 172.21.15.16:0/3554074767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aac1037c0 msgr2=0x7f8aac19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aac1037c0 0x7f8aac19f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 -- 172.21.15.16:0/3554074767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac101c00 msgr2=0x7f8aac197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac101c00 0x7f8aac197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 -- 172.21.15.16:0/3554074767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac19fa80 con 0x7f8aac102e00 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.778+0000 7f8aa2ffd640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac198390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009230 tx=0x7f8a94031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:00.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.779+0000 7f8aa97fa640 1 -- 172.21.15.16:0/3554074767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9403f030 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.779+0000 7f8aa97fa640 1 -- 172.21.15.16:0/3554074767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a94003ad0 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.779+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac19fd10 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.779+0000 7f8aa97fa640 1 -- 172.21.15.16:0/3554074767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94035070 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.779+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac1a01e0 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.780+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac108fd0 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.781+0000 7f8aa97fa640 1 -- 172.21.15.16:0/3554074767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8a94042030 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.782+0000 7f8aa97fa640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a980a32f0 0x7f8a980a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.782+0000 7f8aa97fa640 1 -- 172.21.15.16:0/3554074767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7f8a940ef660 con 0x7f8aac102e00 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.782+0000 7f8aab7fe640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a980a32f0 0x7f8a980a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.783+0000 7f8aab7fe640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a980a32f0 0x7f8a980a57b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8aac102c60 tx=0x7f8a9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.785+0000 7f8aa97fa640 1 -- 172.21.15.16:0/3554074767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a940bc800 con 0x7f8aac102e00 2024-09-15T22:34:00.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:00.973+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8aac062630 con 0x7f8a980a32f0 2024-09-15T22:34:01.159 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:34:00.853+0000 7f4c54371640 -1 osd.8 0 waiting for initial osdmap 2024-09-15T22:34:01.159 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:34:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:34:00.862+0000 7f4c4c191640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:34:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi130 2024-09-15T22:34:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi130 to 5696M 2024-09-15T22:34:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:34:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:34:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='osd.8 ' entity='osd.8' 2024-09-15T22:34:01.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:01.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:01.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:01.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] boot 2024-09-15T22:34:01.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: osdmap e54: 9 total, 9 up, 9 in 2024-09-15T22:34:01.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi130 2024-09-15T22:34:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi130 to 5696M 2024-09-15T22:34:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:34:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:34:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='osd.8 ' entity='osd.8' 2024-09-15T22:34:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] boot 2024-09-15T22:34:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: osdmap e54: 9 total, 9 up, 9 in 2024-09-15T22:34:01.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi130 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi130 to 5696M 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='osd.8 ' entity='osd.8' 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: osd.8 [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] boot 2024-09-15T22:34:01.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: osdmap e54: 9 total, 9 up, 9 in 2024-09-15T22:34:01.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:34:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:02 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:34:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:02 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:34:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:02 smithi059 ceph-mon[34872]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:34:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:02 smithi059 ceph-mon[34872]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:02.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:02 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:34:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:02 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:34:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:02 smithi130 ceph-mon[35742]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:34:02.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:02 smithi130 ceph-mon[35742]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:02.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:02 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:34:02.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:02 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:34:02.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:02 smithi016 ceph-mon[31492]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 266 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:34:02.551 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:02 smithi016 ceph-mon[31492]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:03.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:03 smithi059 ceph-mon[34872]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T22:34:03.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:03 smithi130 ceph-mon[35742]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T22:34:03.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:03 smithi016 ceph-mon[31492]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T22:34:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:04 smithi059 ceph-mon[34872]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:04 smithi059 ceph-mon[34872]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T22:34:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:04 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/701337156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "995607d1-c7a6-4ad1-9538-192ed3d439fc"}]: dispatch 2024-09-15T22:34:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:04 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/701337156' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "995607d1-c7a6-4ad1-9538-192ed3d439fc"}]': finished 2024-09-15T22:34:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:04 smithi059 ceph-mon[34872]: osdmap e57: 10 total, 9 up, 10 in 2024-09-15T22:34:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:04 smithi130 ceph-mon[35742]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:04 smithi130 ceph-mon[35742]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T22:34:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:04 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/701337156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "995607d1-c7a6-4ad1-9538-192ed3d439fc"}]: dispatch 2024-09-15T22:34:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:04 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/701337156' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "995607d1-c7a6-4ad1-9538-192ed3d439fc"}]': finished 2024-09-15T22:34:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:04 smithi130 ceph-mon[35742]: osdmap e57: 10 total, 9 up, 10 in 2024-09-15T22:34:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:04 smithi016 ceph-mon[31492]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:04 smithi016 ceph-mon[31492]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T22:34:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:04 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/701337156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "995607d1-c7a6-4ad1-9538-192ed3d439fc"}]: dispatch 2024-09-15T22:34:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:04 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/701337156' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "995607d1-c7a6-4ad1-9538-192ed3d439fc"}]': finished 2024-09-15T22:34:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:04 smithi016 ceph-mon[31492]: osdmap e57: 10 total, 9 up, 10 in 2024-09-15T22:34:04.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:05.437 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:05 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2016641785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:05.437 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:05 smithi059 ceph-mon[34872]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T22:34:05.437 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:05 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:05.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:05 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2016641785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:05.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:05 smithi016 ceph-mon[31492]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T22:34:05.465 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:05 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:05.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:05 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2016641785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:05.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:05 smithi130 ceph-mon[35742]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T22:34:05.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:05 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:06.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:06 smithi059 ceph-mon[34872]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:06.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:06 smithi130 ceph-mon[35742]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:06.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:06 smithi016 ceph-mon[31492]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 564 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:08.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:08 smithi059 ceph-mon[34872]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:08.473 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:08 smithi016 ceph-mon[31492]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:08.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:08 smithi130 ceph-mon[35742]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:10.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:10 smithi016 ceph-mon[31492]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:34:10.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:10 smithi059 ceph-mon[34872]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:34:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:10 smithi130 ceph-mon[35742]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:34:12.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:12 smithi059 ceph-mon[34872]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:34:12.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:12 smithi016 ceph-mon[31492]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:34:12.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:12 smithi130 ceph-mon[35742]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:34:13.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:34:13.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:34:13.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:34:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:14.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:14 smithi059 ceph-mon[34872]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-15T22:34:14.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:14 smithi059 ceph-mon[34872]: Deploying daemon osd.9 on smithi016 2024-09-15T22:34:14.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:14 smithi016 ceph-mon[31492]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-15T22:34:14.504 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:14 smithi016 ceph-mon[31492]: Deploying daemon osd.9 on smithi016 2024-09-15T22:34:14.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:14 smithi130 ceph-mon[35742]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-15T22:34:14.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:14 smithi130 ceph-mon[35742]: Deploying daemon osd.9 on smithi016 2024-09-15T22:34:16.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:16 smithi016 ceph-mon[31492]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:34:16.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:16 smithi130 ceph-mon[35742]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:34:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:16 smithi059 ceph-mon[34872]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:34:18.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:18 smithi016 ceph-mon[31492]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:34:18.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:18 smithi130 ceph-mon[35742]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:34:18.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:18 smithi059 ceph-mon[34872]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:34:19.528 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:19.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:19.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:19 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:19.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:19.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:19.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:19 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:19.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:19.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:19.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:19 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:20.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:20 smithi016 ceph-mon[31492]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:34:20.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:20 smithi130 ceph-mon[35742]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:34:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:20 smithi059 ceph-mon[34872]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:34:22.472 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:22 smithi016 ceph-mon[31492]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:22.472 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:22 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.472 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:22 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.472 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:22 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:22.472 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:22 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:22.473 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:22 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:22 smithi130 ceph-mon[35742]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:22 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:22 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:22 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:22 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:22.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:22 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:22 smithi059 ceph-mon[34872]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:22 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:22 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:22 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:22 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:22 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:22.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.776+0000 7f8aa97fa640 1 -- 172.21.15.16:0/3554074767 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8aac062630 con 0x7f8a980a32f0 2024-09-15T22:34:22.779 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-09-15T22:34:22.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.778+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a980a32f0 msgr2=0x7f8a980a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:22.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.778+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a980a32f0 0x7f8a980a57b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8aac102c60 tx=0x7f8a9c005f90 comp rx=0 tx=0).stop 2024-09-15T22:34:22.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.778+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 msgr2=0x7f8aac198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:22.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.778+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac198390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009230 tx=0x7f8a94031b60 comp rx=0 tx=0).stop 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 shutdown_connections 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a980a32f0 0x7f8a980a57b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aac1037c0 0x7f8aac19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aac102e00 0x7f8aac198390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 --2- 172.21.15.16:0/3554074767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aac101c00 0x7f8aac197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 >> 172.21.15.16:0/3554074767 conn(0x7f8aac0fd390 msgr2=0x7f8aac1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 shutdown_connections 2024-09-15T22:34:22.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:34:22.779+0000 7f8ab1d22640 1 -- 172.21.15.16:0/3554074767 wait complete. 2024-09-15T22:34:23.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:23.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:23.477 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:23 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:23.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:23.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:23.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:23 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:23.595 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:23.595 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:23.595 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:23 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:23.798 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.9.service 2024-09-15T22:34:23.801 INFO:tasks.cephadm:Deploying osd.10 on smithi059 with /dev/vg_nvme/lv_1... 2024-09-15T22:34:23.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:34:24.063 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:34:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:24 smithi016 ceph-mon[31492]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:24.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:24 smithi059 ceph-mon[34872]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:24.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:24 smithi130 ceph-mon[35742]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:24.734 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:34:24 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:34:24.702+0000 7fd0e5a69640 -1 osd.9 0 log_to_monitors true 2024-09-15T22:34:25.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:25 smithi016 ceph-mon[31492]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:34:25.490 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:25 smithi059 ceph-mon[34872]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:34:25.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:25 smithi130 ceph-mon[35742]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:34:26.407 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:26 smithi059 ceph-mon[34872]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:26.407 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:26 smithi059 ceph-mon[34872]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:34:26.407 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:26 smithi059 ceph-mon[34872]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T22:34:26.407 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:26 smithi059 ceph-mon[34872]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:34:26.407 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:26 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:26.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:26 smithi016 ceph-mon[31492]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:26.525 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:26 smithi016 ceph-mon[31492]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:34:26.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:26 smithi016 ceph-mon[31492]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T22:34:26.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:26 smithi016 ceph-mon[31492]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:34:26.526 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:26 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:26.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:26 smithi130 ceph-mon[35742]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:26.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:26 smithi130 ceph-mon[35742]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:34:26.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:26 smithi130 ceph-mon[35742]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T22:34:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:26 smithi130 ceph-mon[35742]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]: dispatch 2024-09-15T22:34:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:26 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:26.747 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-09-15T22:34:26.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2024-09-15T22:34:26.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:34:26 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:34:26.758+0000 7fd0e19ec640 -1 osd.9 0 waiting for initial osdmap 2024-09-15T22:34:26.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:34:26 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:34:26.768+0000 7fd0d980c640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:34:27.028 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:34:27.411 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:34:27.411 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T22:34:27.411 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:27.413 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:27.413 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.413 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:27 smithi059 ceph-mon[34872]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' 2024-09-15T22:34:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:34:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T22:34:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:27 smithi130 ceph-mon[35742]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' 2024-09-15T22:34:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1746814786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec1026d0 msgr2=0x7f8bec108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1746814786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec1026d0 0x7f8bec108f60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009ca0 tx=0x7f8be402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:34:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1746814786 shutdown_connections 2024-09-15T22:34:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1746814786 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec1026d0 0x7f8bec108f60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1746814786 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bec101d10 0x7f8bec102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1746814786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bec100b10 0x7f8bec100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:27.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1746814786 >> 172.21.15.59:0/1746814786 conn(0x7f8bec0fc2a0 msgr2=0x7f8bec0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1746814786 shutdown_connections 2024-09-15T22:34:27.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1746814786 wait complete. 2024-09-15T22:34:27.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.538+0000 7f8bf10b8640 1 Processor -- start 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bf10b8640 1 -- start start 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bf10b8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec100b10 0x7f8bec104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bf10b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bec101d10 0x7f8bec1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bf10b8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bec1026d0 0x7f8bec103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bf10b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10c770 con 0x7f8bec101d10 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bf10b8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10c5f0 con 0x7f8bec100b10 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bf10b8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec10c8f0 con 0x7f8bec1026d0 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec100b10 0x7f8bec104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec100b10 0x7f8bec104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53664/0 (socket says 172.21.15.59:53664) 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 -- 172.21.15.59:0/1983763903 learned_addr learned my addr 172.21.15.59:0/1983763903 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8beb59e640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bec1026d0 0x7f8bec103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bea59c640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bec101d10 0x7f8bec1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:27.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 -- 172.21.15.59:0/1983763903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bec1026d0 msgr2=0x7f8bec103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bec1026d0 0x7f8bec103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 -- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bec101d10 msgr2=0x7f8bec1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bec101d10 0x7f8bec1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.539+0000 7f8bead9d640 1 -- 172.21.15.59:0/1983763903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec103ef0 con 0x7f8bec100b10 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.540+0000 7f8bead9d640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec100b10 0x7f8bec104aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00c970 tx=0x7f8bdc00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.540+0000 7f8bd3fff640 1 -- 172.21.15.59:0/1983763903 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc007b50 con 0x7f8bec100b10 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.540+0000 7f8bd3fff640 1 -- 172.21.15.59:0/1983763903 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bdc007cf0 con 0x7f8bec100b10 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.540+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec1a1f30 con 0x7f8bec100b10 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.540+0000 7f8bd3fff640 1 -- 172.21.15.59:0/1983763903 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00f870 con 0x7f8bec100b10 2024-09-15T22:34:27.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.540+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec1a2320 con 0x7f8bec100b10 2024-09-15T22:34:27.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.541+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bec1a25e0 con 0x7f8bec100b10 2024-09-15T22:34:27.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.542+0000 7f8bd3fff640 1 -- 172.21.15.59:0/1983763903 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8bdc018480 con 0x7f8bec100b10 2024-09-15T22:34:27.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.543+0000 7f8bd3fff640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8bd40a3240 0x7f8bd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:27.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.543+0000 7f8bd3fff640 1 -- 172.21.15.59:0/1983763903 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5981+0+0 (secure 0 0 0) 0x7f8bdc0cf5f0 con 0x7f8bec100b10 2024-09-15T22:34:27.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.543+0000 7f8bea59c640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8bd40a3240 0x7f8bd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:27.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.544+0000 7f8bea59c640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8bd40a3240 0x7f8bd40a5700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bec104820 tx=0x7f8be000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:27.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.547+0000 7f8bd3fff640 1 -- 172.21.15.59:0/1983763903 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bdc09ccb0 con 0x7f8bec100b10 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:27.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:27.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:27.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:27.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:27 smithi016 ceph-mon[31492]: from='osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932]' entity='osd.9' 2024-09-15T22:34:27.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:27.733+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8bec1a27b0 con 0x7f8bd40a3240 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi016 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:28.418 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:28.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:28.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] boot 2024-09-15T22:34:28.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: osdmap e61: 10 total, 10 up, 10 in 2024-09-15T22:34:28.419 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:28 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi016 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] boot 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: osdmap e61: 10 total, 10 up, 10 in 2024-09-15T22:34:28.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:28 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi016 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: osd.9 [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] boot 2024-09-15T22:34:28.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: osdmap e61: 10 total, 10 up, 10 in 2024-09-15T22:34:28.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:28 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:34:29.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:29 smithi059 ceph-mon[34872]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:29 smithi130 ceph-mon[35742]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:29 smithi016 ceph-mon[31492]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:30.493 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:30 smithi059 ceph-mon[34872]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:30.493 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:30 smithi059 ceph-mon[34872]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T22:34:30.494 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:30 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/15514105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]: dispatch 2024-09-15T22:34:30.494 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:30 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]: dispatch 2024-09-15T22:34:30.494 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:30 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]': finished 2024-09-15T22:34:30.494 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:30 smithi059 ceph-mon[34872]: osdmap e63: 11 total, 10 up, 11 in 2024-09-15T22:34:30.494 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:30 smithi130 ceph-mon[35742]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:30 smithi130 ceph-mon[35742]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T22:34:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:30 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/15514105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]: dispatch 2024-09-15T22:34:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:30 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]: dispatch 2024-09-15T22:34:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:30 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]': finished 2024-09-15T22:34:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:30 smithi130 ceph-mon[35742]: osdmap e63: 11 total, 10 up, 11 in 2024-09-15T22:34:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:30 smithi016 ceph-mon[31492]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:30 smithi016 ceph-mon[31492]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T22:34:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:30 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/15514105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]: dispatch 2024-09-15T22:34:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:30 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]: dispatch 2024-09-15T22:34:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:30 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "421e58d7-471b-43aa-908e-828f72805842"}]': finished 2024-09-15T22:34:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:30 smithi016 ceph-mon[31492]: osdmap e63: 11 total, 10 up, 11 in 2024-09-15T22:34:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:31 smithi130 ceph-mon[35742]: from='client.? 172.21.15.59:0/120089864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:31.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:31 smithi016 ceph-mon[31492]: from='client.? 172.21.15.59:0/120089864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:31.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:31 smithi059 ceph-mon[34872]: from='client.? 172.21.15.59:0/120089864' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:32.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:32 smithi130 ceph-mon[35742]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:32.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:32 smithi016 ceph-mon[31492]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:32 smithi059 ceph-mon[34872]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:34.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:34 smithi130 ceph-mon[35742]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:34.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:34 smithi016 ceph-mon[31492]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:34 smithi059 ceph-mon[34872]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:36.521 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:36 smithi059 ceph-mon[34872]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:36.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:36 smithi130 ceph-mon[35742]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:36.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:36 smithi016 ceph-mon[31492]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:38.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:38 smithi059 ceph-mon[34872]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:38.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:38 smithi130 ceph-mon[35742]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:38.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:38 smithi016 ceph-mon[31492]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:39.492 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:34:39.492 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:39.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:34:39.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:39.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:34:39.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:40.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:40 smithi130 ceph-mon[35742]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:40.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:40 smithi130 ceph-mon[35742]: Deploying daemon osd.10 on smithi059 2024-09-15T22:34:40.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:40 smithi016 ceph-mon[31492]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:40.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:40 smithi016 ceph-mon[31492]: Deploying daemon osd.10 on smithi059 2024-09-15T22:34:40.625 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:40 smithi059 ceph-mon[34872]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:40.625 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:40 smithi059 ceph-mon[34872]: Deploying daemon osd.10 on smithi059 2024-09-15T22:34:42.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:42 smithi130 ceph-mon[35742]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:42 smithi016 ceph-mon[31492]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:42.576 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:42 smithi059 ceph-mon[34872]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:44 smithi059 ceph-mon[34872]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:44.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:44 smithi130 ceph-mon[35742]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:44.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:44 smithi016 ceph-mon[31492]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:45 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:45 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:45 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:45 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:45 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:45 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:45.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:45 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:45.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:45 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:45.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:45 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:46.412 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:46 smithi059 ceph-mon[34872]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:46.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:46 smithi130 ceph-mon[35742]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:46.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:46 smithi016 ceph-mon[31492]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:48.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.042+0000 7f8bd3fff640 1 -- 172.21.15.59:0/1983763903 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8bec1a27b0 con 0x7f8bd40a3240 2024-09-15T22:34:48.044 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 10 on host 'smithi059' 2024-09-15T22:34:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8bd40a3240 msgr2=0x7f8bd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8bd40a3240 0x7f8bd40a5700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bec104820 tx=0x7f8be000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:34:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec100b10 msgr2=0x7f8bec104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec100b10 0x7f8bec104aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00c970 tx=0x7f8bdc00cd70 comp rx=0 tx=0).stop 2024-09-15T22:34:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 shutdown_connections 2024-09-15T22:34:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8bd40a3240 0x7f8bd40a5700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:48.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8bec1026d0 0x7f8bec103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:48.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bec101d10 0x7f8bec1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:48.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 --2- 172.21.15.59:0/1983763903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bec100b10 0x7f8bec104aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:48.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 >> 172.21.15.59:0/1983763903 conn(0x7f8bec0fc2a0 msgr2=0x7f8bec0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:48.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 shutdown_connections 2024-09-15T22:34:48.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-09-15T22:34:48.044+0000 7f8bf10b8640 1 -- 172.21.15.59:0/1983763903 wait complete. 2024-09-15T22:34:48.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:48 smithi016 ceph-mon[31492]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:48.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:48 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:48.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:48 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:48.691 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:48 smithi059 ceph-mon[34872]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:48.691 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:48 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:48.691 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:48 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:48.726 DEBUG:teuthology.orchestra.run.smithi059:osd.10> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.10.service 2024-09-15T22:34:48.729 INFO:tasks.cephadm:Deploying osd.11 on smithi130 with /dev/vg_nvme/lv_1... 2024-09-15T22:34:48.729 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:34:48.757 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:48 smithi130 ceph-mon[35742]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:48.757 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:48 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:48.757 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:48 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:48.983 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:34:50.129 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:49 smithi130 ceph-mon[35742]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:50.130 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:50.130 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:49 smithi059 ceph-mon[34872]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:49 smithi016 ceph-mon[31492]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:50.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:50.950 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:50 smithi130 ceph-mon[35742]: from='osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:34:50.950 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:50 smithi130 ceph-mon[35742]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:34:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:50 smithi059 ceph-mon[34872]: from='osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:34:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:50 smithi059 ceph-mon[34872]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:34:51.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:34:50 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:34:50.772+0000 7f92070cd640 -1 osd.10 0 log_to_monitors true 2024-09-15T22:34:51.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:50 smithi016 ceph-mon[31492]: from='osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:34:51.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:50 smithi016 ceph-mon[31492]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:34:51.806 INFO:teuthology.orchestra.run.smithi130.stdout: 2024-09-15T22:34:51.834 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi130:vg_nvme/lv_1 2024-09-15T22:34:52.122 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:34:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-mon[34872]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-mon[34872]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:34:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-mon[34872]: from='osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:34:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-mon[34872]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T22:34:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-mon[34872]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:34:52.195 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:34:51.826+0000 7f9203050640 -1 osd.10 0 waiting for initial osdmap 2024-09-15T22:34:52.195 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:34:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:34:51.836+0000 7f91fee78640 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:34:52.249 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:51 smithi130 ceph-mon[35742]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:52.249 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:51 smithi130 ceph-mon[35742]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:34:52.249 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:51 smithi130 ceph-mon[35742]: from='osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:34:52.249 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:51 smithi130 ceph-mon[35742]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T22:34:52.249 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:51 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:52.249 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:51 smithi130 ceph-mon[35742]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:34:52.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:51 smithi016 ceph-mon[31492]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:52.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:51 smithi016 ceph-mon[31492]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:34:52.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:51 smithi016 ceph-mon[31492]: from='osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:34:52.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:51 smithi016 ceph-mon[31492]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T22:34:52.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:51 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:52.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:51 smithi016 ceph-mon[31492]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-09-15T22:34:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 -- 172.21.15.130:0/3058740303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240101c00 msgr2=0x7f6240102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 --2- 172.21.15.130:0/3058740303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240101c00 0x7f6240102000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6230009900 tx=0x7f623002a410 comp rx=0 tx=0).stop 2024-09-15T22:34:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 -- 172.21.15.130:0/3058740303 shutdown_connections 2024-09-15T22:34:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 --2- 172.21.15.130:0/3058740303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62401037c0 0x7f624010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 --2- 172.21.15.130:0/3058740303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6240102e00 0x7f6240103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 --2- 172.21.15.130:0/3058740303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240101c00 0x7f6240102000 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:52.716 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 -- 172.21.15.130:0/3058740303 >> 172.21.15.130:0/3058740303 conn(0x7f62400fd390 msgr2=0x7f62400ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 -- 172.21.15.130:0/3058740303 shutdown_connections 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.696+0000 7f6246595640 1 -- 172.21.15.130:0/3058740303 wait complete. 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 Processor -- start 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 -- start start 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240101c00 0x7f62401980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240102e00 0x7f6240198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62401037c0 0x7f624019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f624010d980 con 0x7f6240101c00 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f624010d800 con 0x7f62401037c0 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f6246595640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f624010db00 con 0x7f6240102e00 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240102e00 0x7f6240198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f623f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240102e00 0x7f6240198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:35798/0 (socket says 172.21.15.130:35798) 2024-09-15T22:34:52.717 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.697+0000 7f623f7fe640 1 -- 172.21.15.130:0/3442835771 learned_addr learned my addr 172.21.15.130:0/3442835771 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f6244b32640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62401037c0 0x7f624019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623ffff640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240101c00 0x7f62401980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623f7fe640 1 -- 172.21.15.130:0/3442835771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62401037c0 msgr2=0x7f624019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623f7fe640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62401037c0 0x7f624019f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623f7fe640 1 -- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240101c00 msgr2=0x7f62401980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623f7fe640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240101c00 0x7f62401980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623f7fe640 1 -- 172.21.15.130:0/3442835771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624019fd20 con 0x7f6240102e00 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623f7fe640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240102e00 0x7f6240198630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f623400b4a0 tx=0x7f623400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623d7fa640 1 -- 172.21.15.130:0/3442835771 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234014070 con 0x7f6240102e00 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623d7fa640 1 -- 172.21.15.130:0/3442835771 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6234010af0 con 0x7f6240102e00 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624019ffb0 con 0x7f6240102e00 2024-09-15T22:34:52.718 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f623d7fa640 1 -- 172.21.15.130:0/3442835771 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234017d10 con 0x7f6240102e00 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.698+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401a0540 con 0x7f6240102e00 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.699+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62401090c0 con 0x7f6240102e00 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.700+0000 7f623d7fa640 1 -- 172.21.15.130:0/3442835771 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6234017070 con 0x7f6240102e00 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.701+0000 7f623d7fa640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62280a32f0 0x7f62280a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.701+0000 7f623d7fa640 1 -- 172.21.15.130:0/3442835771 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6414+0+0 (secure 0 0 0) 0x7f62340ce8c0 con 0x7f6240102e00 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.701+0000 7f623ffff640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62280a32f0 0x7f62280a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.701+0000 7f623ffff640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62280a32f0 0x7f62280a57b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6230031610 tx=0x7f6230031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:34:52.719 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.704+0000 7f623d7fa640 1 -- 172.21.15.130:0/3442835771 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f623409bd50 con 0x7f6240102e00 2024-09-15T22:34:52.886 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:34:52.883+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6240062630 con 0x7f62280a32f0 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi059 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:52.886 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi059 to 4273M 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:52.887 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] boot 2024-09-15T22:34:52.888 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: osdmap e66: 11 total, 11 up, 11 in 2024-09-15T22:34:52.888 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:52 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi059 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi059 to 4273M 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] boot 2024-09-15T22:34:53.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: osdmap e66: 11 total, 11 up, 11 in 2024-09-15T22:34:53.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:52 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-09-15T22:34:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T22:34:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi059 2024-09-15T22:34:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi059 to 4273M 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:34:53.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:34:53.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: osd.10 [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] boot 2024-09-15T22:34:53.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: osdmap e66: 11 total, 11 up, 11 in 2024-09-15T22:34:53.309 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:52 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:34:54.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:53 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:34:54.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:53 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:34:54.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:53 smithi130 ceph-mon[35742]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:54.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:53 smithi130 ceph-mon[35742]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:54.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:54.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:54.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:53 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:34:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:53 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:34:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:53 smithi059 ceph-mon[34872]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:53 smithi059 ceph-mon[34872]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:53 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:34:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:53 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:34:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:53 smithi016 ceph-mon[31492]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:34:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:53 smithi016 ceph-mon[31492]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi130:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:34:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:34:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:34:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:34:55.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:54 smithi130 ceph-mon[35742]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T22:34:55.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:54 smithi059 ceph-mon[34872]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T22:34:55.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:54 smithi016 ceph-mon[31492]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T22:34:55.981 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:55 smithi130 ceph-mon[35742]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:34:55.981 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:55 smithi130 ceph-mon[35742]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T22:34:55.981 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:55 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/3166498327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]: dispatch 2024-09-15T22:34:55.982 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:55 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]: dispatch 2024-09-15T22:34:55.982 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:55 smithi130 ceph-mon[35742]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]': finished 2024-09-15T22:34:55.982 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:55 smithi130 ceph-mon[35742]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T22:34:55.982 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:55 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:34:56.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:55 smithi059 ceph-mon[34872]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:34:56.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:55 smithi059 ceph-mon[34872]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T22:34:56.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:55 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/3166498327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]: dispatch 2024-09-15T22:34:56.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:55 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]: dispatch 2024-09-15T22:34:56.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:55 smithi059 ceph-mon[34872]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]': finished 2024-09-15T22:34:56.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:55 smithi059 ceph-mon[34872]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T22:34:56.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:55 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:34:56.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:55 smithi016 ceph-mon[31492]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:34:56.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:55 smithi016 ceph-mon[31492]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T22:34:56.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:55 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/3166498327' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]: dispatch 2024-09-15T22:34:56.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:55 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]: dispatch 2024-09-15T22:34:56.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:55 smithi016 ceph-mon[31492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa7b2bbd-e5b1-4b42-a4a1-934185f54929"}]': finished 2024-09-15T22:34:56.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:55 smithi016 ceph-mon[31492]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T22:34:56.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:55 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:34:57.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:56 smithi059 ceph-mon[34872]: from='client.? 172.21.15.130:0/844271951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:57.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:56 smithi059 ceph-mon[34872]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T22:34:57.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:34:57.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:56 smithi130 ceph-mon[35742]: from='client.? 172.21.15.130:0/844271951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:57.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:56 smithi130 ceph-mon[35742]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T22:34:57.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:34:57.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:56 smithi016 ceph-mon[31492]: from='client.? 172.21.15.130:0/844271951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:34:57.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:56 smithi016 ceph-mon[31492]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T22:34:57.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:34:58.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:34:58 smithi059 ceph-mon[34872]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:34:58.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:34:58 smithi130 ceph-mon[35742]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:34:58.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:34:58 smithi016 ceph-mon[31492]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 368 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:00.693 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:00 smithi130 ceph-mon[35742]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-15T22:35:00.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:00 smithi059 ceph-mon[34872]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-15T22:35:00.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:00 smithi016 ceph-mon[31492]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-09-15T22:35:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:02 smithi059 ceph-mon[34872]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:35:02.763 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:02 smithi130 ceph-mon[35742]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:35:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:02 smithi016 ceph-mon[31492]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:35:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:04 smithi130 ceph-mon[35742]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-15T22:35:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:35:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:04 smithi059 ceph-mon[34872]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-15T22:35:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:35:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:04.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:04 smithi016 ceph-mon[31492]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-15T22:35:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:35:04.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:05.686 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:05 smithi130 ceph-mon[35742]: Deploying daemon osd.11 on smithi130 2024-09-15T22:35:05.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:05 smithi016 ceph-mon[31492]: Deploying daemon osd.11 on smithi130 2024-09-15T22:35:05.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:05 smithi059 ceph-mon[34872]: Deploying daemon osd.11 on smithi130 2024-09-15T22:35:06.687 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:06 smithi130 ceph-mon[35742]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-15T22:35:06.708 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:06 smithi016 ceph-mon[31492]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-15T22:35:06.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:06 smithi059 ceph-mon[34872]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-15T22:35:08.647 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:08 smithi130 ceph-mon[35742]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-15T22:35:08.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:08 smithi016 ceph-mon[31492]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-15T22:35:08.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:08 smithi059 ceph-mon[34872]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-15T22:35:10.680 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:10 smithi130 ceph-mon[35742]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:35:10.680 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:10.680 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:10.680 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:10.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:10 smithi016 ceph-mon[31492]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:35:10.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:10.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:10.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:10 smithi059 ceph-mon[34872]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:35:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:12.718 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:12 smithi130 ceph-mon[35742]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:12 smithi016 ceph-mon[31492]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:12 smithi059 ceph-mon[34872]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:13.671 INFO:teuthology.orchestra.run.smithi130.stdout:Created osd(s) 11 on host 'smithi130' 2024-09-15T22:35:13.672 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.670+0000 7f623d7fa640 1 -- 172.21.15.130:0/3442835771 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6240062630 con 0x7f62280a32f0 2024-09-15T22:35:13.673 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62280a32f0 msgr2=0x7f62280a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62280a32f0 0x7f62280a57b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6230031610 tx=0x7f6230031400 comp rx=0 tx=0).stop 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240102e00 msgr2=0x7f6240198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240102e00 0x7f6240198630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f623400b4a0 tx=0x7f623400b960 comp rx=0 tx=0).stop 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 shutdown_connections 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62280a32f0 0x7f62280a57b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62401037c0 0x7f624019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6240102e00 0x7f6240198630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 --2- 172.21.15.130:0/3442835771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6240101c00 0x7f62401980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 >> 172.21.15.130:0/3442835771 conn(0x7f62400fd390 msgr2=0x7f6240106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 shutdown_connections 2024-09-15T22:35:13.674 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:13.672+0000 7f6246595640 1 -- 172.21.15.130:0/3442835771 wait complete. 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:14.337 DEBUG:teuthology.orchestra.run.smithi130:osd.11> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.11.service 2024-09-15T22:35:14.339 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-15T22:35:14.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:35:14.606 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- 172.21.15.16:0/147889042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 msgr2=0x7f94ec10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 --2- 172.21.15.16:0/147889042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec10a050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009ca0 tx=0x7f94e402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- 172.21.15.16:0/147889042 shutdown_connections 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 --2- 172.21.15.16:0/147889042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec10a050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 --2- 172.21.15.16:0/147889042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94ec102e00 0x7f94ec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 --2- 172.21.15.16:0/147889042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94ec101c00 0x7f94ec102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- 172.21.15.16:0/147889042 >> 172.21.15.16:0/147889042 conn(0x7f94ec0fd390 msgr2=0x7f94ec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- 172.21.15.16:0/147889042 shutdown_connections 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- 172.21.15.16:0/147889042 wait complete. 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 Processor -- start 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- start start 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94ec101c00 0x7f94ec197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94ec102e00 0x7f94ec1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec10d860 con 0x7f94ec1037c0 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec10d6e0 con 0x7f94ec101c00 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.069+0000 7f94f2f70640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f94ec10d9e0 con 0x7f94ec102e00 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44064/0 (socket says 172.21.15.16:44064) 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 -- 172.21.15.16:0/910314422 learned_addr learned my addr 172.21.15.16:0/910314422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f0d0c640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94ec101c00 0x7f94ec197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94ebfff640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94ec102e00 0x7f94ec1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 -- 172.21.15.16:0/910314422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94ec102e00 msgr2=0x7f94ec1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94ec102e00 0x7f94ec1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 -- 172.21.15.16:0/910314422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94ec101c00 msgr2=0x7f94ec197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94ec101c00 0x7f94ec197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 -- 172.21.15.16:0/910314422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94ec19fab0 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f150d640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec19f440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94e4005e30 tx=0x7f94e4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94e9ffb640 1 -- 172.21.15.16:0/910314422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4003c30 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94ec19fd40 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94e9ffb640 1 -- 172.21.15.16:0/910314422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e4003dd0 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.070+0000 7f94e9ffb640 1 -- 172.21.15.16:0/910314422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e402fde0 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.071+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94ec1a0220 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.072+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94ec109000 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.073+0000 7f94e9ffb640 1 -- 172.21.15.16:0/910314422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f94e4004050 con 0x7f94ec1037c0 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.074+0000 7f94e9ffb640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94dc0a3270 0x7f94dc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:15.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.074+0000 7f94f0d0c640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94dc0a3270 0x7f94dc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.074+0000 7f94e9ffb640 1 -- 172.21.15.16:0/910314422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6809+0+0 (secure 0 0 0) 0x7f94e40f08d0 con 0x7f94ec1037c0 2024-09-15T22:35:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.074+0000 7f94f0d0c640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94dc0a3270 0x7f94dc0a5730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94ec102c60 tx=0x7f94e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.077+0000 7f94e9ffb640 1 -- 172.21.15.16:0/910314422 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94e40bdc50 con 0x7f94ec1037c0 2024-09-15T22:35:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.255+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f94ec062e90 con 0x7f94ec1037c0 2024-09-15T22:35:15.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.256+0000 7f94e9ffb640 1 -- 172.21.15.16:0/910314422 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f94e40bd550 con 0x7f94ec1037c0 2024-09-15T22:35:15.258 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94dc0a3270 msgr2=0x7f94dc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:15.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94dc0a3270 0x7f94dc0a5730 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f94ec102c60 tx=0x7f94e000b390 comp rx=0 tx=0).stop 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 msgr2=0x7f94ec19f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec19f440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94e4005e30 tx=0x7f94e4031ee0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 shutdown_connections 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94dc0a3270 0x7f94dc0a5730 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94ec1037c0 0x7f94ec19f440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f94ec102e00 0x7f94ec1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 --2- 172.21.15.16:0/910314422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94ec101c00 0x7f94ec197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 >> 172.21.15.16:0/910314422 conn(0x7f94ec0fd390 msgr2=0x7f94ec1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 shutdown_connections 2024-09-15T22:35:15.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:15.258+0000 7f94f2f70640 1 -- 172.21.15.16:0/910314422 wait complete. 2024-09-15T22:35:15.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:35:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:35:15.506+0000 7fa8617c6640 -1 osd.11 0 log_to_monitors true 2024-09-15T22:35:16.059 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1726439692,"num_in_osds":12,"osd_in_since":1726439695,"num_remapped_pgs":0} 2024-09-15T22:35:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:15 smithi059 ceph-mon[34872]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:15 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/910314422' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:35:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:15 smithi059 ceph-mon[34872]: from='osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:35:16.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:15 smithi059 ceph-mon[34872]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:35:16.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:15 smithi130 ceph-mon[35742]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:15 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/910314422' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:35:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:15 smithi130 ceph-mon[35742]: from='osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:35:16.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:15 smithi130 ceph-mon[35742]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:35:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:15 smithi016 ceph-mon[31492]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:15 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/910314422' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:35:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:15 smithi016 ceph-mon[31492]: from='osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:35:16.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:15 smithi016 ceph-mon[31492]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:35:17.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:35:17.245 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:35:16.942+0000 7fa85d749640 -1 osd.11 0 waiting for initial osdmap 2024-09-15T22:35:17.245 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:35:16.953+0000 7fa859571640 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:35:17.245 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-mon[35742]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:35:17.245 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-mon[35742]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T22:35:17.246 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-mon[35742]: from='osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:35:17.246 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:17.246 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-mon[35742]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:35:17.246 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-mon[35742]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:35:17.246 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:16 smithi130 ceph-mon[35742]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:35:17.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:16 smithi016 ceph-mon[31492]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:35:17.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:16 smithi016 ceph-mon[31492]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T22:35:17.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:16 smithi016 ceph-mon[31492]: from='osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:35:17.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:16 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:17.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:16 smithi016 ceph-mon[31492]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:35:17.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:16 smithi016 ceph-mon[31492]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:35:17.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:16 smithi016 ceph-mon[31492]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:35:17.334 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:17.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:16 smithi059 ceph-mon[34872]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:35:17.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:16 smithi059 ceph-mon[34872]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T22:35:17.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:16 smithi059 ceph-mon[34872]: from='osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:35:17.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:16 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:17.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:16 smithi059 ceph-mon[34872]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]: dispatch 2024-09-15T22:35:17.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:16 smithi059 ceph-mon[34872]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi130", "root=default"]}]': finished 2024-09-15T22:35:17.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:16 smithi059 ceph-mon[34872]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 -- 172.21.15.16:0/2436840355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124101d00 msgr2=0x7fa124102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/2436840355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124101d00 0x7fa124102180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa118009900 tx=0x7fa11802a410 comp rx=0 tx=0).stop 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 -- 172.21.15.16:0/2436840355 shutdown_connections 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/2436840355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1241026c0 0x7fa124108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/2436840355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124101d00 0x7fa124102180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/2436840355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa124100b00 0x7fa124100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 -- 172.21.15.16:0/2436840355 >> 172.21.15.16:0/2436840355 conn(0x7fa1240fc270 msgr2=0x7fa1240fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:17.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 -- 172.21.15.16:0/2436840355 shutdown_connections 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 -- 172.21.15.16:0/2436840355 wait complete. 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.917+0000 7fa12a8d9640 1 Processor -- start 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa12a8d9640 1 -- start start 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa12a8d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124100b00 0x7fa124198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa12a8d9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa124101d00 0x7fa124198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa12a8d9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1241026c0 0x7fa12419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa12a8d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410c780 con 0x7fa124100b00 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa12a8d9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410c600 con 0x7fa124101d00 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa12a8d9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa12410c900 con 0x7fa1241026c0 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124100b00 0x7fa124198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124100b00 0x7fa124198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44084/0 (socket says 172.21.15.16:44084) 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 -- 172.21.15.16:0/523028916 learned_addr learned my addr 172.21.15.16:0/523028916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa128e76640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1241026c0 0x7fa12419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa1237fe640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa124101d00 0x7fa124198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 -- 172.21.15.16:0/523028916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1241026c0 msgr2=0x7fa12419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1241026c0 0x7fa12419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 -- 172.21.15.16:0/523028916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa124101d00 msgr2=0x7fa124198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa124101d00 0x7fa124198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.918+0000 7fa123fff640 1 -- 172.21.15.16:0/523028916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12419fc60 con 0x7fa124100b00 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.919+0000 7fa123fff640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124100b00 0x7fa124198030 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa11400d770 tx=0x7fa11400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.919+0000 7fa1217fa640 1 -- 172.21.15.16:0/523028916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114017070 con 0x7fa124100b00 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.919+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12419ff50 con 0x7fa124100b00 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.919+0000 7fa1217fa640 1 -- 172.21.15.16:0/523028916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa114009c30 con 0x7fa124100b00 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.919+0000 7fa1217fa640 1 -- 172.21.15.16:0/523028916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114003dd0 con 0x7fa124100b00 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.919+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1241a03a0 con 0x7fa124100b00 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.920+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1241a05d0 con 0x7fa124100b00 2024-09-15T22:35:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.921+0000 7fa1217fa640 1 -- 172.21.15.16:0/523028916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa114003f70 con 0x7fa124100b00 2024-09-15T22:35:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.922+0000 7fa1217fa640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa10c0a3270 0x7fa10c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.922+0000 7fa1237fe640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa10c0a3270 0x7fa10c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.922+0000 7fa1217fa640 1 -- 172.21.15.16:0/523028916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7fa1140cecb0 con 0x7fa124100b00 2024-09-15T22:35:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.922+0000 7fa1237fe640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa10c0a3270 0x7fa10c0a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa1180315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:17.926+0000 7fa1217fa640 1 -- 172.21.15.16:0/523028916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa114009de0 con 0x7fa124100b00 2024-09-15T22:35:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.104+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa1241a0e80 con 0x7fa124100b00 2024-09-15T22:35:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.105+0000 7fa1217fa640 1 -- 172.21.15.16:0/523028916 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fa11409b8a0 con 0x7fa124100b00 2024-09-15T22:35:18.107 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: Detected new or changed devices on smithi130 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi130 to 4272M 2024-09-15T22:35:18.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:18.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:18.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.108 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:17 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa10c0a3270 msgr2=0x7fa10c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa10c0a3270 0x7fa10c0a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa118000c00 tx=0x7fa1180315b0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124100b00 msgr2=0x7fa124198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124100b00 0x7fa124198030 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa11400d770 tx=0x7fa11400dc30 comp rx=0 tx=0).stop 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 shutdown_connections 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa10c0a3270 0x7fa10c0a5730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1241026c0 0x7fa12419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa124101d00 0x7fa124198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.107+0000 7fa12a8d9640 1 --2- 172.21.15.16:0/523028916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa124100b00 0x7fa124198030 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:18.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.108+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 >> 172.21.15.16:0/523028916 conn(0x7fa1240fc270 msgr2=0x7fa12406a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:18.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.108+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 shutdown_connections 2024-09-15T22:35:18.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:18.108+0000 7fa12a8d9640 1 -- 172.21.15.16:0/523028916 wait complete. 2024-09-15T22:35:18.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:18.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: Detected new or changed devices on smithi130 2024-09-15T22:35:18.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi130 to 4272M 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:17 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:18.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: Detected new or changed devices on smithi130 2024-09-15T22:35:18.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi130 to 4272M 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:18.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:17 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:18.757 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1726439717,"num_in_osds":12,"osd_in_since":1726439695,"num_remapped_pgs":0} 2024-09-15T22:35:18.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:35:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:18 smithi016 ceph-mon[31492]: purged_snaps scrub starts 2024-09-15T22:35:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:18 smithi016 ceph-mon[31492]: purged_snaps scrub ok 2024-09-15T22:35:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:18 smithi016 ceph-mon[31492]: osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] boot 2024-09-15T22:35:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:18 smithi016 ceph-mon[31492]: osdmap e73: 12 total, 12 up, 12 in 2024-09-15T22:35:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:18 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:18 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/523028916' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:35:19.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:18 smithi130 ceph-mon[35742]: purged_snaps scrub starts 2024-09-15T22:35:19.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:18 smithi130 ceph-mon[35742]: purged_snaps scrub ok 2024-09-15T22:35:19.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:18 smithi130 ceph-mon[35742]: osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] boot 2024-09-15T22:35:19.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:18 smithi130 ceph-mon[35742]: osdmap e73: 12 total, 12 up, 12 in 2024-09-15T22:35:19.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:18 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:19.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:18 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/523028916' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:35:19.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:18 smithi059 ceph-mon[34872]: purged_snaps scrub starts 2024-09-15T22:35:19.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:18 smithi059 ceph-mon[34872]: purged_snaps scrub ok 2024-09-15T22:35:19.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:18 smithi059 ceph-mon[34872]: osd.11 [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] boot 2024-09-15T22:35:19.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:18 smithi059 ceph-mon[34872]: osdmap e73: 12 total, 12 up, 12 in 2024-09-15T22:35:19.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:18 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:35:19.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:18 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/523028916' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.493+0000 7f898fa63640 1 -- 172.21.15.16:0/3870841848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f898810c180 msgr2=0x7f898810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.493+0000 7f898fa63640 1 --2- 172.21.15.16:0/3870841848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f898810c180 0x7f898810e610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8980009ca0 tx=0x7f898002a950 comp rx=0 tx=0).stop 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 -- 172.21.15.16:0/3870841848 shutdown_connections 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 --2- 172.21.15.16:0/3870841848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f898810c180 0x7f898810e610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 --2- 172.21.15.16:0/3870841848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8988103700 0x7f898810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 --2- 172.21.15.16:0/3870841848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8988102de0 0x7f89881031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 -- 172.21.15.16:0/3870841848 >> 172.21.15.16:0/3870841848 conn(0x7f89880fc7a0 msgr2=0x7f89880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 -- 172.21.15.16:0/3870841848 shutdown_connections 2024-09-15T22:35:19.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 -- 172.21.15.16:0/3870841848 wait complete. 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.494+0000 7f898fa63640 1 Processor -- start 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898fa63640 1 -- start start 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898fa63640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8988102de0 0x7f8988198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898fa63640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988103700 0x7f8988198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898fa63640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898810c180 0x7f898819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898fa63640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8988112000 con 0x7f8988103700 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898fa63640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8988111e80 con 0x7f898810c180 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898fa63640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8988112180 con 0x7f8988102de0 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988103700 0x7f8988198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988103700 0x7f8988198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44104/0 (socket says 172.21.15.16:44104) 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 -- 172.21.15.16:0/3509788614 learned_addr learned my addr 172.21.15.16:0/3509788614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898d7ff640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8988102de0 0x7f8988198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:19.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898e000640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898810c180 0x7f898819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 -- 172.21.15.16:0/3509788614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8988102de0 msgr2=0x7f8988198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8988102de0 0x7f8988198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 -- 172.21.15.16:0/3509788614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898810c180 msgr2=0x7f898819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898810c180 0x7f898819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 -- 172.21.15.16:0/3509788614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898819d3f0 con 0x7f8988103700 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898e000640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898810c180 0x7f898819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.495+0000 7f898cffe640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988103700 0x7f8988198980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b4a0 tx=0x7f897c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.496+0000 7f897a7fc640 1 -- 172.21.15.16:0/3509788614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c014070 con 0x7f8988103700 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.496+0000 7f897a7fc640 1 -- 172.21.15.16:0/3509788614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897c010af0 con 0x7f8988103700 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.496+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898819d6e0 con 0x7f8988103700 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.496+0000 7f897a7fc640 1 -- 172.21.15.16:0/3509788614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c017d10 con 0x7f8988103700 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.496+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89881a4fc0 con 0x7f8988103700 2024-09-15T22:35:19.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.497+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89881031c0 con 0x7f8988103700 2024-09-15T22:35:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.499+0000 7f897a7fc640 1 -- 172.21.15.16:0/3509788614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f897c017070 con 0x7f8988103700 2024-09-15T22:35:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.500+0000 7f897a7fc640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f89740a32c0 0x7f89740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.500+0000 7f898d7ff640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f89740a32c0 0x7f89740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.500+0000 7f897a7fc640 1 -- 172.21.15.16:0/3509788614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f897c0cebe0 con 0x7f8988103700 2024-09-15T22:35:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.500+0000 7f898d7ff640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f89740a32c0 0x7f89740a5780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f898400bbe0 tx=0x7f898400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.503+0000 7f897a7fc640 1 -- 172.21.15.16:0/3509788614 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f897c09bd00 con 0x7f8988103700 2024-09-15T22:35:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.684+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89881997a0 con 0x7f8988103700 2024-09-15T22:35:19.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.685+0000 7f897a7fc640 1 -- 172.21.15.16:0/3509788614 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18446 (secure 0 0 0) 0x7f89881997a0 con 0x7f8988103700 2024-09-15T22:35:19.686 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:19.687 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":74,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:35:18.952837+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:35:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f89740a32c0 msgr2=0x7f89740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f89740a32c0 0x7f89740a5780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f898400bbe0 tx=0x7f898400b390 comp rx=0 tx=0).stop 2024-09-15T22:35:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988103700 msgr2=0x7f8988198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988103700 0x7f8988198980 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f897c00b4a0 tx=0x7f897c00b960 comp rx=0 tx=0).stop 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 shutdown_connections 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f89740a32c0 0x7f89740a5780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f898810c180 0x7f898819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8988103700 0x7f8988198980 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 --2- 172.21.15.16:0/3509788614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8988102de0 0x7f8988198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 >> 172.21.15.16:0/3509788614 conn(0x7f89880fc7a0 msgr2=0x7f89880fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 shutdown_connections 2024-09-15T22:35:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:19.687+0000 7f898fa63640 1 -- 172.21.15.16:0/3509788614 wait complete. 2024-09-15T22:35:20.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:19 smithi130 ceph-mon[35742]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:20.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:19 smithi130 ceph-mon[35742]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T22:35:20.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:19 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3509788614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:35:20.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:19 smithi016 ceph-mon[31492]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:20.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:19 smithi016 ceph-mon[31492]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T22:35:20.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:19 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3509788614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:35:20.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:19 smithi059 ceph-mon[34872]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:35:20.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:19 smithi059 ceph-mon[34872]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T22:35:20.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:19 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3509788614' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:35:20.504 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:31:31.005703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T22:35:20.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-15T22:35:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 -- 172.21.15.16:0/4073272680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 msgr2=0x7fa5c810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/4073272680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c810e600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802a480 comp rx=0 tx=0).stop 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 -- 172.21.15.16:0/4073272680 shutdown_connections 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/4073272680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c810e600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/4073272680 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5c81036c0 0x7fa5c810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/4073272680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5c8102da0 0x7fa5c8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 -- 172.21.15.16:0/4073272680 >> 172.21.15.16:0/4073272680 conn(0x7fa5c80fc7a0 msgr2=0x7fa5c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 -- 172.21.15.16:0/4073272680 shutdown_connections 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.230+0000 7fa5cff4c640 1 -- 172.21.15.16:0/4073272680 wait complete. 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 Processor -- start 2024-09-15T22:35:21.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 -- start start 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5c8102da0 0x7fa5c819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5c81036c0 0x7fa5c819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8111fa0 con 0x7fa5c810c180 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8111e20 con 0x7fa5c8102da0 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cff4c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8112120 con 0x7fa5c81036c0 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5ce4e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5ce4e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44126/0 (socket says 172.21.15.16:44126) 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5ce4e9640 1 -- 172.21.15.16:0/1583136086 learned_addr learned my addr 172.21.15.16:0/1583136086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cdce8640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5c8102da0 0x7fa5c819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.231+0000 7fa5cd4e7640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5c81036c0 0x7fa5c819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5ce4e9640 1 -- 172.21.15.16:0/1583136086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5c81036c0 msgr2=0x7fa5c819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5ce4e9640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5c81036c0 0x7fa5c819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5ce4e9640 1 -- 172.21.15.16:0/1583136086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5c8102da0 msgr2=0x7fa5c819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5ce4e9640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5c8102da0 0x7fa5c819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5ce4e9640 1 -- 172.21.15.16:0/1583136086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c819f600 con 0x7fa5c810c180 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5cdce8640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5c8102da0 0x7fa5c819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5cd4e7640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5c81036c0 0x7fa5c819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5ce4e9640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c819ef20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005ab0 tx=0x7fa5b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5c6ffd640 1 -- 172.21.15.16:0/1583136086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b803f030 con 0x7fa5c810c180 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5c6ffd640 1 -- 172.21.15.16:0/1583136086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b8003ce0 con 0x7fa5c810c180 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c819f890 con 0x7fa5c810c180 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5c6ffd640 1 -- 172.21.15.16:0/1583136086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8035070 con 0x7fa5c810c180 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.232+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c81a71d0 con 0x7fa5c810c180 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.233+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c8103180 con 0x7fa5c810c180 2024-09-15T22:35:21.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.234+0000 7fa5c6ffd640 1 -- 172.21.15.16:0/1583136086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa5b8042030 con 0x7fa5c810c180 2024-09-15T22:35:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.235+0000 7fa5c6ffd640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5b00a32c0 0x7fa5b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.235+0000 7fa5cdce8640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5b00a32c0 0x7fa5b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.236+0000 7fa5c6ffd640 1 -- 172.21.15.16:0/1583136086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa5b80effa0 con 0x7fa5c810c180 2024-09-15T22:35:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.236+0000 7fa5cdce8640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5b00a32c0 0x7fa5b00a5780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005fd0 tx=0x7fa5c00093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:21.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.239+0000 7fa5c6ffd640 1 -- 172.21.15.16:0/1583136086 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5b80bd0c0 con 0x7fa5c810c180 2024-09-15T22:35:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.413+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5c81a73a0 con 0x7fa5c810c180 2024-09-15T22:35:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.414+0000 7fa5c6ffd640 1 -- 172.21.15.16:0/1583136086 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fa5b80bc9c0 con 0x7fa5c810c180 2024-09-15T22:35:21.415 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5b00a32c0 msgr2=0x7fa5b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5b00a32c0 0x7fa5b00a5780 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0005fd0 tx=0x7fa5c00093e0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 msgr2=0x7fa5c819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c819ef20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8005ab0 tx=0x7fa5b80042f0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 shutdown_connections 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5b00a32c0 0x7fa5b00a5780 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5c810c180 0x7fa5c819ef20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa5c81036c0 0x7fa5c819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 --2- 172.21.15.16:0/1583136086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5c8102da0 0x7fa5c819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 >> 172.21.15.16:0/1583136086 conn(0x7fa5c80fc7a0 msgr2=0x7fa5c8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 shutdown_connections 2024-09-15T22:35:21.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:21.416+0000 7fa5cff4c640 1 -- 172.21.15.16:0/1583136086 wait complete. 2024-09-15T22:35:22.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:21 smithi016 ceph-mon[31492]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:22.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:21 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1583136086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:35:22.234 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-09-15T22:35:22.234 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.b on smithi059 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.e on smithi059 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.h on smithi059 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.k on smithi059 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.c on smithi130 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.f on smithi130 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.i on smithi130 2024-09-15T22:35:22.235 INFO:tasks.cephadm:Adding mds.l on smithi130 2024-09-15T22:35:22.235 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l' 2024-09-15T22:35:22.252 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:21 smithi130 ceph-mon[35742]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:22.252 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:21 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1583136086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:35:22.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:21 smithi059 ceph-mon[34872]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:22.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:21 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1583136086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:35:22.510 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:35:23.016 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 -- 172.21.15.130:0/1257457266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278107950 msgr2=0x7ff278109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:23.016 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 --2- 172.21.15.130:0/1257457266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278107950 0x7ff278109d40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff270009ca0 tx=0x7ff27002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.016 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 -- 172.21.15.130:0/1257457266 shutdown_connections 2024-09-15T22:35:23.016 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 --2- 172.21.15.130:0/1257457266 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278107950 0x7ff278109d40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 --2- 172.21.15.130:0/1257457266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff278105020 0x7ff278107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 --2- 172.21.15.130:0/1257457266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2781026f0 0x7ff278104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 -- 172.21.15.130:0/1257457266 >> 172.21.15.130:0/1257457266 conn(0x7ff2780fc290 msgr2=0x7ff2780fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:23.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 -- 172.21.15.130:0/1257457266 shutdown_connections 2024-09-15T22:35:23.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.992+0000 7ff27fd27640 1 -- 172.21.15.130:0/1257457266 wait complete. 2024-09-15T22:35:23.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 Processor -- start 2024-09-15T22:35:23.017 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 -- start start 2024-09-15T22:35:23.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2781026f0 0x7ff278198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:23.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278105020 0x7ff2781985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:23.018 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff278107950 0x7ff27819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:23.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810d980 con 0x7ff278107950 2024-09-15T22:35:23.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810d800 con 0x7ff2781026f0 2024-09-15T22:35:23.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27fd27640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810db00 con 0x7ff278105020 2024-09-15T22:35:23.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27d2c2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278105020 0x7ff2781985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:23.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27d2c2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278105020 0x7ff2781985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.130:60934/0 (socket says 172.21.15.130:60934) 2024-09-15T22:35:23.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27d2c2640 1 -- 172.21.15.130:0/3821270531 learned_addr learned my addr 172.21.15.130:0/3821270531 (peer_addr_for_me v2:172.21.15.130:0/0) 2024-09-15T22:35:23.020 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27dac3640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2781026f0 0x7ff278198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.993+0000 7ff27d2c2640 1 -- 172.21.15.130:0/3821270531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2781026f0 msgr2=0x7ff278198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27e2c4640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff278107950 0x7ff27819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27d2c2640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2781026f0 0x7ff278198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27d2c2640 1 -- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff278107950 msgr2=0x7ff27819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27d2c2640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff278107950 0x7ff27819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27d2c2640 1 -- 172.21.15.130:0/3821270531 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27819fcc0 con 0x7ff278105020 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27dac3640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2781026f0 0x7ff278198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27d2c2640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278105020 0x7ff2781985d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b450 tx=0x7ff26800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff26effd640 1 -- 172.21.15.130:0/3821270531 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268015020 con 0x7ff278105020 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff26effd640 1 -- 172.21.15.130:0/3821270531 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff268011d70 con 0x7ff278105020 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27819ffb0 con 0x7ff278105020 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff26effd640 1 -- 172.21.15.130:0/3821270531 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26800f340 con 0x7ff278105020 2024-09-15T22:35:23.021 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.994+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2781a04e0 con 0x7ff278105020 2024-09-15T22:35:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.995+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2780fdd80 con 0x7ff278105020 2024-09-15T22:35:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.996+0000 7ff26effd640 1 -- 172.21.15.130:0/3821270531 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff268018020 con 0x7ff278105020 2024-09-15T22:35:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.997+0000 7ff26effd640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff2640a32f0 0x7ff2640a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.997+0000 7ff26effd640 1 -- 172.21.15.130:0/3821270531 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7ff2680cef30 con 0x7ff278105020 2024-09-15T22:35:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.997+0000 7ff27dac3640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff2640a32f0 0x7ff2640a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:22.997+0000 7ff27dac3640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff2640a32f0 0x7ff2640a57b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff274009930 tx=0x7ff274008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:23.022 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.001+0000 7ff26effd640 1 -- 172.21.15.130:0/3821270531 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff26801f020 con 0x7ff278105020 2024-09-15T22:35:23.185 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.184+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}) -- 0x7ff278062730 con 0x7ff2640a32f0 2024-09-15T22:35:23.193 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.191+0000 7ff26effd640 1 -- 172.21.15.130:0/3821270531 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff278062730 con 0x7ff2640a32f0 2024-09-15T22:35:23.193 INFO:teuthology.orchestra.run.smithi130.stdout:Scheduled mds.all update... 2024-09-15T22:35:23.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff2640a32f0 msgr2=0x7ff2640a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:23.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff2640a32f0 0x7ff2640a57b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff274009930 tx=0x7ff274008040 comp rx=0 tx=0).stop 2024-09-15T22:35:23.195 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278105020 msgr2=0x7ff2781985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278105020 0x7ff2781985d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b450 tx=0x7ff26800b910 comp rx=0 tx=0).stop 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 shutdown_connections 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff2640a32f0 0x7ff2640a57b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff278107950 0x7ff27819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff278105020 0x7ff2781985d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 --2- 172.21.15.130:0/3821270531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2781026f0 0x7ff278198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 >> 172.21.15.130:0/3821270531 conn(0x7ff2780fc290 msgr2=0x7ff278103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 shutdown_connections 2024-09-15T22:35:23.196 INFO:teuthology.orchestra.run.smithi130.stderr:2024-09-15T22:35:23.194+0000 7ff27fd27640 1 -- 172.21.15.130:0/3821270531 wait complete. 2024-09-15T22:35:23.951 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.a.service 2024-09-15T22:35:23.953 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.d.service 2024-09-15T22:35:23.996 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.g.service 2024-09-15T22:35:24.039 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.j.service 2024-09-15T22:35:24.082 DEBUG:teuthology.orchestra.run.smithi059:mds.b> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.b.service 2024-09-15T22:35:24.085 DEBUG:teuthology.orchestra.run.smithi059:mds.e> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.e.service 2024-09-15T22:35:24.128 DEBUG:teuthology.orchestra.run.smithi059:mds.h> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.h.service 2024-09-15T22:35:24.173 DEBUG:teuthology.orchestra.run.smithi059:mds.k> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.k.service 2024-09-15T22:35:24.216 DEBUG:teuthology.orchestra.run.smithi130:mds.c> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.c.service 2024-09-15T22:35:24.219 DEBUG:teuthology.orchestra.run.smithi130:mds.f> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.f.service 2024-09-15T22:35:24.262 DEBUG:teuthology.orchestra.run.smithi130:mds.i> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.i.service 2024-09-15T22:35:24.305 DEBUG:teuthology.orchestra.run.smithi130:mds.l> sudo journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.l.service 2024-09-15T22:35:24.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:24.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:24 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:24.348 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-15T22:35:24.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:24.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:24.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:24.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:24.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:24 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:24.500 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:24.501 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:24 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:24.612 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 -- 172.21.15.16:0/1613003958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0112640 msgr2=0x7f12a0112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1613003958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0112640 0x7f12a0112a20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1290009900 tx=0x7f129002a410 comp rx=0 tx=0).stop 2024-09-15T22:35:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 -- 172.21.15.16:0/1613003958 shutdown_connections 2024-09-15T22:35:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1613003958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12a010cf30 0x7f12a0119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1613003958 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12a010c590 0x7f12a010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1613003958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a0112640 0x7f12a0112a20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 -- 172.21.15.16:0/1613003958 >> 172.21.15.16:0/1613003958 conn(0x7f12a006cdc0 msgr2=0x7f12a006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 -- 172.21.15.16:0/1613003958 shutdown_connections 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.081+0000 7f12a6a62640 1 -- 172.21.15.16:0/1613003958 wait complete. 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 Processor -- start 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 -- start start 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12a010c590 0x7f12a01a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a010cf30 0x7f12a01a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12a0112640 0x7f12a01a7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12a011c960 con 0x7f12a010cf30 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f12a011c7e0 con 0x7f12a0112640 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.082+0000 7f12a6a62640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f12a011cae0 con 0x7f12a010c590 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a010cf30 0x7f12a01a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a010cf30 0x7f12a01a32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42370/0 (socket says 172.21.15.16:42370) 2024-09-15T22:35:25.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 -- 172.21.15.16:0/1530255067 learned_addr learned my addr 172.21.15.16:0/1530255067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129ffff640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12a010c590 0x7f12a01a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f12a4fff640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12a0112640 0x7f12a01a7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 -- 172.21.15.16:0/1530255067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12a010c590 msgr2=0x7f12a01a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12a010c590 0x7f12a01a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 -- 172.21.15.16:0/1530255067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12a0112640 msgr2=0x7f12a01a7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12a0112640 0x7f12a01a7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 -- 172.21.15.16:0/1530255067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a01a7d30 con 0x7f12a010cf30 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129f7fe640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a010cf30 0x7f12a01a32c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f129400c920 tx=0x7f129400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129d7fa640 1 -- 172.21.15.16:0/1530255067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294007c70 con 0x7f12a010cf30 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a01a8020 con 0x7f12a010cf30 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129d7fa640 1 -- 172.21.15.16:0/1530255067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1294007e10 con 0x7f12a010cf30 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f129d7fa640 1 -- 172.21.15.16:0/1530255067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294018890 con 0x7f12a010cf30 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.083+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a01af900 con 0x7f12a010cf30 2024-09-15T22:35:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.084+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a010c9f0 con 0x7f12a010cf30 2024-09-15T22:35:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.086+0000 7f129d7fa640 1 -- 172.21.15.16:0/1530255067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1294018b30 con 0x7f12a010cf30 2024-09-15T22:35:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.086+0000 7f129d7fa640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f12880a32c0 0x7f12880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.086+0000 7f129ffff640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f12880a32c0 0x7f12880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.086+0000 7f129d7fa640 1 -- 172.21.15.16:0/1530255067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f12940cf560 con 0x7f12a010cf30 2024-09-15T22:35:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.087+0000 7f129ffff640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f12880a32c0 0x7f12880a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f12900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.090+0000 7f129d7fa640 1 -- 172.21.15.16:0/1530255067 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1294014030 con 0x7f12a010cf30 2024-09-15T22:35:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.302+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12a002d7c0 con 0x7f12a010cf30 2024-09-15T22:35:25.304 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:25 smithi016 ceph-mon[31492]: Deploying daemon mds.l on smithi130 2024-09-15T22:35:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.302+0000 7f129d7fa640 1 -- 172.21.15.16:0/1530255067 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f1294018e20 con 0x7f12a010cf30 2024-09-15T22:35:25.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:25.304 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"btime":"2024-09-15T22:28:23:528693+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-15T22:35:25.304 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f12880a32c0 msgr2=0x7f12880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f12880a32c0 0x7f12880a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1290000c00 tx=0x7f12900058d0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a010cf30 msgr2=0x7f12a01a32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a010cf30 0x7f12a01a32c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f129400c920 tx=0x7f129400cde0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 shutdown_connections 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f12880a32c0 0x7f12880a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12a0112640 0x7f12a01a7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12a010cf30 0x7f12a01a32c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 --2- 172.21.15.16:0/1530255067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f12a010c590 0x7f12a01a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 >> 172.21.15.16:0/1530255067 conn(0x7f12a006cdc0 msgr2=0x7f12a010aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 shutdown_connections 2024-09-15T22:35:25.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:25.304+0000 7f12a6a62640 1 -- 172.21.15.16:0/1530255067 wait complete. 2024-09-15T22:35:25.441 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:25 smithi130 ceph-mon[35742]: Deploying daemon mds.l on smithi130 2024-09-15T22:35:25.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:25 smithi059 ceph-mon[34872]: Deploying daemon mds.l on smithi130 2024-09-15T22:35:25.779 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:25 smithi130 systemd[1]: Starting Ceph mds.l for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:26.077 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:25 smithi130 podman[65014]: 2024-09-15 22:35:25.727657197 +0000 UTC m=+0.016058155 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:26.077 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:25 smithi130 podman[65014]: 2024-09-15 22:35:25.849031192 +0000 UTC m=+0.137432134 container create ab862784bd7aea40f5e6336af15c70107b2faa70330278aab21eee7a09189b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-15T22:35:26.078 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:26 smithi130 podman[65014]: 2024-09-15 22:35:26.076652822 +0000 UTC m=+0.365053764 container init ab862784bd7aea40f5e6336af15c70107b2faa70330278aab21eee7a09189b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-15T22:35:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:26 smithi130 ceph-mon[35742]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:26 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1530255067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:26 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6832/2632425547,v1:172.21.15.130:6833/2632425547] up:boot 2024-09-15T22:35:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:26 smithi130 ceph-mon[35742]: fsmap 1 up:standby 2024-09-15T22:35:26.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:26 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:35:26.529 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:26 smithi130 podman[65014]: 2024-09-15 22:35:26.079619074 +0000 UTC m=+0.368020019 container start ab862784bd7aea40f5e6336af15c70107b2faa70330278aab21eee7a09189b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-15T22:35:26.530 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:26 smithi130 ceph-mds[65033]: -- 172.21.15.130:0/2748878119 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e43f68dc20 con 0x55e43f66a800 2024-09-15T22:35:26.530 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l[65029]: starting mds.l at 2024-09-15T22:35:26.530 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:26 smithi130 bash[65014]: ab862784bd7aea40f5e6336af15c70107b2faa70330278aab21eee7a09189b29 2024-09-15T22:35:26.530 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:35:26 smithi130 systemd[1]: Started Ceph mds.l for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:26 smithi016 ceph-mon[31492]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:26 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1530255067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:26 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6832/2632425547,v1:172.21.15.130:6833/2632425547] up:boot 2024-09-15T22:35:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:26 smithi016 ceph-mon[31492]: fsmap 1 up:standby 2024-09-15T22:35:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:26 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:35:26.665 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:26 smithi059 ceph-mon[34872]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:26.666 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:26 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1530255067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:26.666 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:26 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6832/2632425547,v1:172.21.15.130:6833/2632425547] up:boot 2024-09-15T22:35:26.666 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:26 smithi059 ceph-mon[34872]: fsmap 1 up:standby 2024-09-15T22:35:26.666 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:26 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:35:27.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:27.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:27.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:27 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:27.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:27 smithi059 ceph-mon[34872]: Deploying daemon mds.k on smithi059 2024-09-15T22:35:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:27 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:27.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:27 smithi130 ceph-mon[35742]: Deploying daemon mds.k on smithi059 2024-09-15T22:35:27.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:27 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:27.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:27 smithi016 ceph-mon[31492]: Deploying daemon mds.k on smithi059 2024-09-15T22:35:28.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:28.382 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:28 smithi059 ceph-mon[34872]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:28.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:28 smithi016 ceph-mon[31492]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:28.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:28 smithi130 ceph-mon[35742]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:28.694 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:35:28 smithi059 podman[64337]: 2024-09-15 22:35:28.618814911 +0000 UTC m=+0.139370227 container create d51f617d1f26764f330030bed888ff2fbe5099d0cd91b9ad3eca08e9a27d285f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-15T22:35:28.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 -- 172.21.15.16:0/4138712610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c106f10 msgr2=0x7f260c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:28.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 --2- 172.21.15.16:0/4138712610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c106f10 0x7f260c10ea90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2604009ca0 tx=0x7f260402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 -- 172.21.15.16:0/4138712610 shutdown_connections 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 --2- 172.21.15.16:0/4138712610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c106f10 0x7f260c10ea90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 --2- 172.21.15.16:0/4138712610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f260c102790 0x7f260c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 --2- 172.21.15.16:0/4138712610 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f260c101de0 0x7f260c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 -- 172.21.15.16:0/4138712610 >> 172.21.15.16:0/4138712610 conn(0x7f260c0fd8c0 msgr2=0x7f260c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.850+0000 7f261189b640 1 -- 172.21.15.16:0/4138712610 shutdown_connections 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 -- 172.21.15.16:0/4138712610 wait complete. 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 Processor -- start 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 -- start start 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f260c101de0 0x7f260c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c102790 0x7f260c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:28.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f260c106f10 0x7f260c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f260c111f70 con 0x7f260c102790 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f260c111df0 con 0x7f260c101de0 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.851+0000 7f261189b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f260c1120f0 con 0x7f260c106f10 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c102790 0x7f260c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c102790 0x7f260c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42390/0 (socket says 172.21.15.16:42390) 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 -- 172.21.15.16:0/2280652859 learned_addr learned my addr 172.21.15.16:0/2280652859 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260b7fe640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f260c106f10 0x7f260c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260affd640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f260c101de0 0x7f260c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 -- 172.21.15.16:0/2280652859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f260c106f10 msgr2=0x7f260c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f260c106f10 0x7f260c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 -- 172.21.15.16:0/2280652859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f260c101de0 msgr2=0x7f260c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f260c101de0 0x7f260c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:28.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 -- 172.21.15.16:0/2280652859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260c19d2e0 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260b7fe640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f260c106f10 0x7f260c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f260a7fc640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c102790 0x7f260c198870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f260000e9f0 tx=0x7f260000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f2610899640 1 -- 172.21.15.16:0/2280652859 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000cd80 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260c19d5d0 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f2610899640 1 -- 172.21.15.16:0/2280652859 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2600010910 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f2610899640 1 -- 172.21.15.16:0/2280652859 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26000188f0 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.852+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260c1a4eb0 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.853+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260c1021c0 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.855+0000 7f2610899640 1 -- 172.21.15.16:0/2280652859 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2600010ac0 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.855+0000 7f2610899640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f25f40a3240 0x7f25f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.855+0000 7f260affd640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f25f40a3240 0x7f25f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.855+0000 7f2610899640 1 -- 172.21.15.16:0/2280652859 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f26000cfce0 con 0x7f260c102790 2024-09-15T22:35:28.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.856+0000 7f260affd640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f25f40a3240 0x7f25f40a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005e10 tx=0x7f25fc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:28.858+0000 7f2610899640 1 -- 172.21.15.16:0/2280652859 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f260009cea0 con 0x7f260c102790 2024-09-15T22:35:29.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.083+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f260c10a150 con 0x7f260c102790 2024-09-15T22:35:29.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.084+0000 7f2610899640 1 -- 172.21.15.16:0/2280652859 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f260009c7a0 con 0x7f260c102790 2024-09-15T22:35:29.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:29.087 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-15T22:35:26:108055+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T22:35:29.087 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f25f40a3240 msgr2=0x7f25f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f25f40a3240 0x7f25f40a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005e10 tx=0x7f25fc005da0 comp rx=0 tx=0).stop 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c102790 msgr2=0x7f260c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c102790 0x7f260c198870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f260000e9f0 tx=0x7f260000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 shutdown_connections 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f25f40a3240 0x7f25f40a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f260c106f10 0x7f260c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f260c102790 0x7f260c198870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:29.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 --2- 172.21.15.16:0/2280652859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f260c101de0 0x7f260c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:29.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.086+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 >> 172.21.15.16:0/2280652859 conn(0x7f260c0fd8c0 msgr2=0x7f260c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:29.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.087+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 shutdown_connections 2024-09-15T22:35:29.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:29.087+0000 7f261189b640 1 -- 172.21.15.16:0/2280652859 wait complete. 2024-09-15T22:35:29.194 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:35:28 smithi059 podman[64337]: 2024-09-15 22:35:28.888354164 +0000 UTC m=+0.408909480 container init d51f617d1f26764f330030bed888ff2fbe5099d0cd91b9ad3eca08e9a27d285f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-15T22:35:29.194 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:35:28 smithi059 podman[64337]: 2024-09-15 22:35:28.891296491 +0000 UTC m=+0.411851807 container start d51f617d1f26764f330030bed888ff2fbe5099d0cd91b9ad3eca08e9a27d285f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:35:29.194 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:35:28 smithi059 ceph-mds[64356]: -- 172.21.15.59:0/3607086587 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e48ca51860 con 0x55e48ca2f400 2024-09-15T22:35:29.194 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:35:28 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k[64352]: starting mds.k at 2024-09-15T22:35:29.194 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:35:28 smithi059 bash[64337]: d51f617d1f26764f330030bed888ff2fbe5099d0cd91b9ad3eca08e9a27d285f 2024-09-15T22:35:29.194 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:35:28 smithi059 systemd[1]: Started Ceph mds.k for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:30.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:30.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: Deploying daemon mds.j on smithi016 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2280652859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6832/1429864622,v1:172.21.15.59:6833/1429864622] up:boot 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: fsmap 2 up:standby 2024-09-15T22:35:30.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:30 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:35:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:30.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: Deploying daemon mds.j on smithi016 2024-09-15T22:35:30.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2280652859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:30.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6832/1429864622,v1:172.21.15.59:6833/1429864622] up:boot 2024-09-15T22:35:30.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: fsmap 2 up:standby 2024-09-15T22:35:30.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:30 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: Deploying daemon mds.j on smithi016 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2280652859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6832/1429864622,v1:172.21.15.59:6833/1429864622] up:boot 2024-09-15T22:35:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: fsmap 2 up:standby 2024-09-15T22:35:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:30 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:35:31.808 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:31 smithi016 systemd[1]: Starting Ceph mds.j for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:31.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:32.190 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:32 smithi016 ceph-mon[31492]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:32.190 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:31 smithi016 podman[70451]: 2024-09-15 22:35:31.826531064 +0000 UTC m=+0.016528286 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:32.191 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:31 smithi016 podman[70451]: 2024-09-15 22:35:31.945163111 +0000 UTC m=+0.135160321 container create d1044db680afc0c190ce363d435c3ae4f3c6b53319428bc62b266f85dab99385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-15T22:35:32.191 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:32 smithi016 podman[70451]: 2024-09-15 22:35:32.189681487 +0000 UTC m=+0.379678698 container init d1044db680afc0c190ce363d435c3ae4f3c6b53319428bc62b266f85dab99385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-15T22:35:32.393 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:32.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:32 smithi059 ceph-mon[34872]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:32.516 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:32 smithi016 podman[70451]: 2024-09-15 22:35:32.193212113 +0000 UTC m=+0.383209327 container start d1044db680afc0c190ce363d435c3ae4f3c6b53319428bc62b266f85dab99385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:35:32.516 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:32 smithi016 ceph-mds[70500]: -- 172.21.15.16:0/3267156137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd2d495860 con 0x55fd2d473000 2024-09-15T22:35:32.516 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:32 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j[70485]: starting mds.j at 2024-09-15T22:35:32.517 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:32 smithi016 bash[70451]: d1044db680afc0c190ce363d435c3ae4f3c6b53319428bc62b266f85dab99385 2024-09-15T22:35:32.517 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:35:32 smithi016 systemd[1]: Started Ceph mds.j for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:32.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:32 smithi130 ceph-mon[35742]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 -- 172.21.15.16:0/2425531686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102570 msgr2=0x7fddc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 --2- 172.21.15.16:0/2425531686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102570 0x7fddc81029b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009900 tx=0x7fddbc02a480 comp rx=0 tx=0).stop 2024-09-15T22:35:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 -- 172.21.15.16:0/2425531686 shutdown_connections 2024-09-15T22:35:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 --2- 172.21.15.16:0/2425531686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc8102ef0 0x7fddc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 --2- 172.21.15.16:0/2425531686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102570 0x7fddc81029b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 --2- 172.21.15.16:0/2425531686 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc8100e20 0x7fddc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 -- 172.21.15.16:0/2425531686 >> 172.21.15.16:0/2425531686 conn(0x7fddc80fc7d0 msgr2=0x7fddc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 -- 172.21.15.16:0/2425531686 shutdown_connections 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.912+0000 7fddcfb47640 1 -- 172.21.15.16:0/2425531686 wait complete. 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 Processor -- start 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 -- start start 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc8100e20 0x7fddc81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc8102570 0x7fddc8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102ef0 0x7fddc819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8110f80 con 0x7fddc8102ef0 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8110e00 con 0x7fddc8100e20 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcfb47640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8111100 con 0x7fddc8102570 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102ef0 0x7fddc819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102ef0 0x7fddc819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34390/0 (socket says 172.21.15.16:34390) 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 -- 172.21.15.16:0/1882334767 learned_addr learned my addr 172.21.15.16:0/1882334767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 -- 172.21.15.16:0/1882334767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc8102570 msgr2=0x7fddc8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc8102570 0x7fddc8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddcd8e3640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc8100e20 0x7fddc81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 -- 172.21.15.16:0/1882334767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc8100e20 msgr2=0x7fddc81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc8100e20 0x7fddc81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.913+0000 7fddce0e4640 1 -- 172.21.15.16:0/1882334767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc819d490 con 0x7fddc8102ef0 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.914+0000 7fddce0e4640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102ef0 0x7fddc819cdb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fddb800cad0 tx=0x7fddb800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.914+0000 7fddc6ffd640 1 -- 172.21.15.16:0/1882334767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb800ae20 con 0x7fddc8102ef0 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.914+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc819d780 con 0x7fddc8102ef0 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.914+0000 7fddc6ffd640 1 -- 172.21.15.16:0/1882334767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8005020 con 0x7fddc8102ef0 2024-09-15T22:35:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.914+0000 7fddc6ffd640 1 -- 172.21.15.16:0/1882334767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb800f6f0 con 0x7fddc8102ef0 2024-09-15T22:35:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.914+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc81a5060 con 0x7fddc8102ef0 2024-09-15T22:35:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.915+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc8101200 con 0x7fddc8102ef0 2024-09-15T22:35:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.916+0000 7fddc6ffd640 1 -- 172.21.15.16:0/1882334767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fddb80051d0 con 0x7fddc8102ef0 2024-09-15T22:35:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.917+0000 7fddc6ffd640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fddb00a3240 0x7fddb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.917+0000 7fddc6ffd640 1 -- 172.21.15.16:0/1882334767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fddb80cf930 con 0x7fddc8102ef0 2024-09-15T22:35:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.917+0000 7fddcd8e3640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fddb00a3240 0x7fddb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.918+0000 7fddcd8e3640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fddb00a3240 0x7fddb00a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddc00058c0 tx=0x7fddc0005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:32.920+0000 7fddc6ffd640 1 -- 172.21.15.16:0/1882334767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddb8014030 con 0x7fddc8102ef0 2024-09-15T22:35:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.141+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddc8199840 con 0x7fddc8102ef0 2024-09-15T22:35:33.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.141+0000 7fddc6ffd640 1 -- 172.21.15.16:0/1882334767 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7fddc8199840 con 0x7fddc8102ef0 2024-09-15T22:35:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:33.144 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"btime":"2024-09-15T22:35:33:032347+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:35:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-09-15T22:35:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.143+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fddb00a3240 msgr2=0x7fddb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.143+0000 7fddcfb47640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fddb00a3240 0x7fddb00a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fddc00058c0 tx=0x7fddc0005c60 comp rx=0 tx=0).stop 2024-09-15T22:35:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.143+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102ef0 msgr2=0x7fddc819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.143+0000 7fddcfb47640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102ef0 0x7fddc819cdb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fddb800cad0 tx=0x7fddb800cf90 comp rx=0 tx=0).stop 2024-09-15T22:35:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 shutdown_connections 2024-09-15T22:35:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fddb00a3240 0x7fddb00a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc8102ef0 0x7fddc819cdb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc8102570 0x7fddc8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 --2- 172.21.15.16:0/1882334767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc8100e20 0x7fddc81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 >> 172.21.15.16:0/1882334767 conn(0x7fddc80fc7d0 msgr2=0x7fddc806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 shutdown_connections 2024-09-15T22:35:33.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:33.144+0000 7fddcfb47640 1 -- 172.21.15.16:0/1882334767 wait complete. 2024-09-15T22:35:33.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:33.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:33.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:33.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: Deploying daemon mds.d on smithi016 2024-09-15T22:35:33.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.16:6834/2727578831,v1:172.21.15.16:6835/2727578831] up:boot 2024-09-15T22:35:33.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: fsmap 3 up:standby 2024-09-15T22:35:33.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:35:33.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:33 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1882334767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:33.590 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.590 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.590 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.590 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:33.590 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:33.591 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:33.591 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: Deploying daemon mds.d on smithi016 2024-09-15T22:35:33.591 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.16:6834/2727578831,v1:172.21.15.16:6835/2727578831] up:boot 2024-09-15T22:35:33.591 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: fsmap 3 up:standby 2024-09-15T22:35:33.591 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:35:33.591 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:33 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1882334767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: Deploying daemon mds.d on smithi016 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.16:6834/2727578831,v1:172.21.15.16:6835/2727578831] up:boot 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: fsmap 3 up:standby 2024-09-15T22:35:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:35:33.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:33 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1882334767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:34.599 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:34 smithi016 ceph-mon[31492]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:34 smithi059 ceph-mon[34872]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:34 smithi130 ceph-mon[35742]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:35.557 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:35 smithi016 systemd[1]: Starting Ceph mds.d for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:35.865 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:35 smithi016 podman[70862]: 2024-09-15 22:35:35.501128698 +0000 UTC m=+0.016558063 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:35.865 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:35 smithi016 podman[70862]: 2024-09-15 22:35:35.743000588 +0000 UTC m=+0.258429938 container create 0d040ce8a24674f66e816fb38cec170db585454e10006d5e17fbfa2ff3c77053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-15T22:35:36.307 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:35 smithi016 podman[70862]: 2024-09-15 22:35:35.987643103 +0000 UTC m=+0.503072456 container init 0d040ce8a24674f66e816fb38cec170db585454e10006d5e17fbfa2ff3c77053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-15T22:35:36.307 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:35 smithi016 podman[70862]: 2024-09-15 22:35:35.990693987 +0000 UTC m=+0.506123337 container start 0d040ce8a24674f66e816fb38cec170db585454e10006d5e17fbfa2ff3c77053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-15T22:35:36.307 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mds[70881]: -- 172.21.15.16:0/402338329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0a043d860 con 0x55e0a041a800 2024-09-15T22:35:36.307 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d[70877]: starting mds.d at 2024-09-15T22:35:36.307 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:36 smithi016 bash[70862]: 0d040ce8a24674f66e816fb38cec170db585454e10006d5e17fbfa2ff3c77053 2024-09-15T22:35:36.307 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:35:36 smithi016 systemd[1]: Started Ceph mds.d for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:36.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:36 smithi059 ceph-mon[34872]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:36 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:36.727 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:36 smithi130 ceph-mon[35742]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:36.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:36 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mon[31492]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:36.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:36 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/3674479599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38100e20 msgr2=0x7f6f38101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/3674479599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38100e20 0x7f6f38101200 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009900 tx=0x7f6f3402a480 comp rx=0 tx=0).stop 2024-09-15T22:35:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/3674479599 shutdown_connections 2024-09-15T22:35:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/3674479599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f38102ef0 0x7f6f3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/3674479599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38102570 0x7f6f381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/3674479599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38100e20 0x7f6f38101200 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/3674479599 >> 172.21.15.16:0/3674479599 conn(0x7f6f380fc7d0 msgr2=0x7f6f380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/3674479599 shutdown_connections 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.244+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/3674479599 wait complete. 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 Processor -- start 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 -- start start 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f38100e20 0x7f6f38198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38102570 0x7f6f381989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38102ef0 0x7f6f3819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38110f80 con 0x7f6f38102ef0 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38110e00 con 0x7f6f38102570 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3f9b0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38111100 con 0x7f6f38100e20 2024-09-15T22:35:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3df4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38102ef0 0x7f6f3819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3df4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38102ef0 0x7f6f3819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34440/0 (socket says 172.21.15.16:34440) 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3df4d640 1 -- 172.21.15.16:0/2462212303 learned_addr learned my addr 172.21.15.16:0/2462212303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3d74c640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f38100e20 0x7f6f38198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3cf4b640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38102570 0x7f6f381989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3df4d640 1 -- 172.21.15.16:0/2462212303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f38100e20 msgr2=0x7f6f38198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3df4d640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f38100e20 0x7f6f38198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.245+0000 7f6f3df4d640 1 -- 172.21.15.16:0/2462212303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38102570 msgr2=0x7f6f381989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f3df4d640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38102570 0x7f6f381989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f3df4d640 1 -- 172.21.15.16:0/2462212303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3819d440 con 0x7f6f38102ef0 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f3cf4b640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38102570 0x7f6f381989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f3df4d640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38102ef0 0x7f6f3819cd60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000cce0 tx=0x7f6f3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f2e7fc640 1 -- 172.21.15.16:0/2462212303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000f450 con 0x7f6f38102ef0 2024-09-15T22:35:37.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f2e7fc640 1 -- 172.21.15.16:0/2462212303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f3000fad0 con 0x7f6f38102ef0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3819d730 con 0x7f6f38102ef0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f2e7fc640 1 -- 172.21.15.16:0/2462212303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30018c10 con 0x7f6f38102ef0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.246+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f381a5010 con 0x7f6f38102ef0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.247+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f38101200 con 0x7f6f38102ef0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.248+0000 7f6f2e7fc640 1 -- 172.21.15.16:0/2462212303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6f3000f5f0 con 0x7f6f38102ef0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.249+0000 7f6f2e7fc640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f240a32c0 0x7f6f240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.249+0000 7f6f3d74c640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f240a32c0 0x7f6f240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.249+0000 7f6f2e7fc640 1 -- 172.21.15.16:0/2462212303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f6f300cfb60 con 0x7f6f38102ef0 2024-09-15T22:35:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.249+0000 7f6f3d74c640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f240a32c0 0x7f6f240a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005ab0 tx=0x7f6f34005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:37.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.253+0000 7f6f2e7fc640 1 -- 172.21.15.16:0/2462212303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f30014030 con 0x7f6f38102ef0 2024-09-15T22:35:37.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.467+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f381a51e0 con 0x7f6f38102ef0 2024-09-15T22:35:37.469 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:37 smithi016 ceph-mon[31492]: Deploying daemon mds.e on smithi059 2024-09-15T22:35:37.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:37 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.16:6836/4208747900,v1:172.21.15.16:6837/4208747900] up:boot 2024-09-15T22:35:37.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:37 smithi016 ceph-mon[31492]: fsmap 4 up:standby 2024-09-15T22:35:37.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:37 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:35:37.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.468+0000 7f6f2e7fc640 1 -- 172.21.15.16:0/2462212303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f6f3009cd20 con 0x7f6f38102ef0 2024-09-15T22:35:37.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:37.471 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"btime":"2024-09-15T22:35:36:348647+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:35:37.471 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-15T22:35:37.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f240a32c0 msgr2=0x7f6f240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f240a32c0 0x7f6f240a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005ab0 tx=0x7f6f34005a20 comp rx=0 tx=0).stop 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38102ef0 msgr2=0x7f6f3819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38102ef0 0x7f6f3819cd60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000cce0 tx=0x7f6f3000a620 comp rx=0 tx=0).stop 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 shutdown_connections 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6f240a32c0 0x7f6f240a5780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f38102ef0 0x7f6f3819cd60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38102570 0x7f6f381989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 --2- 172.21.15.16:0/2462212303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f38100e20 0x7f6f38198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 >> 172.21.15.16:0/2462212303 conn(0x7f6f380fc7d0 msgr2=0x7f6f3806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 shutdown_connections 2024-09-15T22:35:37.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:37.471+0000 7f6f3f9b0640 1 -- 172.21.15.16:0/2462212303 wait complete. 2024-09-15T22:35:37.689 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:37 smithi059 ceph-mon[34872]: Deploying daemon mds.e on smithi059 2024-09-15T22:35:37.690 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:37 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.16:6836/4208747900,v1:172.21.15.16:6837/4208747900] up:boot 2024-09-15T22:35:37.690 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:37 smithi059 ceph-mon[34872]: fsmap 4 up:standby 2024-09-15T22:35:37.690 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:37 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:35:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:37 smithi130 ceph-mon[35742]: Deploying daemon mds.e on smithi059 2024-09-15T22:35:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:37 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.16:6836/4208747900,v1:172.21.15.16:6837/4208747900] up:boot 2024-09-15T22:35:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:37 smithi130 ceph-mon[35742]: fsmap 4 up:standby 2024-09-15T22:35:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:37 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:35:38.363 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 systemd[1]: Starting Ceph mds.e for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:38.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:38 smithi016 ceph-mon[31492]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:38.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:38 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2462212303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:38.678 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:38 smithi059 ceph-mon[34872]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:38.678 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:38 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2462212303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:38.678 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 podman[64654]: 2024-09-15 22:35:38.296047576 +0000 UTC m=+0.015672418 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:38.678 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 podman[64654]: 2024-09-15 22:35:38.424554421 +0000 UTC m=+0.144179261 container create 02dbb5fb62823d08472024fc191bd83f044a77677f721cfb3b52e16674cd8422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-15T22:35:38.678 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 podman[64654]: 2024-09-15 22:35:38.677563396 +0000 UTC m=+0.397188238 container init 02dbb5fb62823d08472024fc191bd83f044a77677f721cfb3b52e16674cd8422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:35:38.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:38 smithi130 ceph-mon[35742]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:38 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2462212303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:38.944 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 podman[64654]: 2024-09-15 22:35:38.680796267 +0000 UTC m=+0.400421115 container start 02dbb5fb62823d08472024fc191bd83f044a77677f721cfb3b52e16674cd8422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-15T22:35:38.944 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 ceph-mds[64674]: -- 172.21.15.59:0/3586993774 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd1262b860 con 0x55fd12608800 2024-09-15T22:35:38.944 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e[64670]: starting mds.e at 2024-09-15T22:35:38.944 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 bash[64654]: 02dbb5fb62823d08472024fc191bd83f044a77677f721cfb3b52e16674cd8422 2024-09-15T22:35:38.944 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:35:38 smithi059 systemd[1]: Started Ceph mds.e for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: Deploying daemon mds.f on smithi130 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6834/2202482741,v1:172.21.15.59:6835/2202482741] up:boot 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: fsmap 5 up:standby 2024-09-15T22:35:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:39 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:35:40.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: Deploying daemon mds.f on smithi130 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6834/2202482741,v1:172.21.15.59:6835/2202482741] up:boot 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: fsmap 5 up:standby 2024-09-15T22:35:40.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:39 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: Deploying daemon mds.f on smithi130 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6834/2202482741,v1:172.21.15.59:6835/2202482741] up:boot 2024-09-15T22:35:40.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: fsmap 5 up:standby 2024-09-15T22:35:40.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:39 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:35:40.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:40.586 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:40.965 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:40 smithi130 systemd[1]: Starting Ceph mds.f for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:41.233 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:40 smithi130 podman[65314]: 2024-09-15 22:35:40.866518813 +0000 UTC m=+0.015956866 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:41.233 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:40 smithi130 podman[65314]: 2024-09-15 22:35:40.979468211 +0000 UTC m=+0.128906257 container create ef1819beb068ccd49f3afadcd52590c5f7af638d41aea26344182d37d55fd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:35:41.233 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:41 smithi130 podman[65314]: 2024-09-15 22:35:41.232300516 +0000 UTC m=+0.381738571 container init ef1819beb068ccd49f3afadcd52590c5f7af638d41aea26344182d37d55fd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-15T22:35:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 -- 172.21.15.16:0/2671040464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c102790 msgr2=0x7f538c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 --2- 172.21.15.16:0/2671040464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c102790 0x7f538c1067e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5374009900 tx=0x7f537402a410 comp rx=0 tx=0).stop 2024-09-15T22:35:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 -- 172.21.15.16:0/2671040464 shutdown_connections 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 --2- 172.21.15.16:0/2671040464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f538c106f10 0x7f538c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 --2- 172.21.15.16:0/2671040464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c102790 0x7f538c1067e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 --2- 172.21.15.16:0/2671040464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c101de0 0x7f538c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 -- 172.21.15.16:0/2671040464 >> 172.21.15.16:0/2671040464 conn(0x7f538c0fd8c0 msgr2=0x7f538c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 -- 172.21.15.16:0/2671040464 shutdown_connections 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.294+0000 7f539133e640 1 -- 172.21.15.16:0/2671040464 wait complete. 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 Processor -- start 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 -- start start 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c101de0 0x7f538c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f538c102790 0x7f538c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c106f10 0x7f538c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f538c111f70 con 0x7f538c106f10 2024-09-15T22:35:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f538c111df0 con 0x7f538c101de0 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f539133e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f538c1120f0 con 0x7f538c102790 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f538b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c106f10 0x7f538c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f538b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c106f10 0x7f538c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34464/0 (socket says 172.21.15.16:34464) 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.295+0000 7f538b7fe640 1 -- 172.21.15.16:0/1853405345 learned_addr learned my addr 172.21.15.16:0/1853405345 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538affd640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c101de0 0x7f538c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538a7fc640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f538c102790 0x7f538c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538b7fe640 1 -- 172.21.15.16:0/1853405345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f538c102790 msgr2=0x7f538c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538b7fe640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f538c102790 0x7f538c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538b7fe640 1 -- 172.21.15.16:0/1853405345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c101de0 msgr2=0x7f538c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538b7fe640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c101de0 0x7f538c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538b7fe640 1 -- 172.21.15.16:0/1853405345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c19d440 con 0x7f538c106f10 2024-09-15T22:35:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f538b7fe640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c106f10 0x7f538c19cd60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f537c00d750 tx=0x7f537c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f5383fff640 1 -- 172.21.15.16:0/1853405345 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c017070 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538c19d730 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f5383fff640 1 -- 172.21.15.16:0/1853405345 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f537c005500 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f5383fff640 1 -- 172.21.15.16:0/1853405345 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537c004d90 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.296+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538c1a5010 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.297+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538c1021c0 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.299+0000 7f5383fff640 1 -- 172.21.15.16:0/1853405345 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f537c004f30 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.300+0000 7f5383fff640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f53780a3240 0x7f53780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.300+0000 7f538affd640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f53780a3240 0x7f53780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.300+0000 7f5383fff640 1 -- 172.21.15.16:0/1853405345 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f537c0cf180 con 0x7f538c106f10 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.300+0000 7f538affd640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f53780a3240 0x7f53780a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5384005fd0 tx=0x7f5384009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.303+0000 7f5383fff640 1 -- 172.21.15.16:0/1853405345 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f537c09bc50 con 0x7f538c106f10 2024-09-15T22:35:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.519+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f538c10a0f0 con 0x7f538c106f10 2024-09-15T22:35:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.520+0000 7f5383fff640 1 -- 172.21.15.16:0/1853405345 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4738 (secure 0 0 0) 0x7f537c09ff50 con 0x7f538c106f10 2024-09-15T22:35:41.522 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:41.522 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"btime":"2024-09-15T22:35:39:360235+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-15T22:35:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f53780a3240 msgr2=0x7f53780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f53780a3240 0x7f53780a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5384005fd0 tx=0x7f5384009500 comp rx=0 tx=0).stop 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c106f10 msgr2=0x7f538c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c106f10 0x7f538c19cd60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f537c00d750 tx=0x7f537c00dc10 comp rx=0 tx=0).stop 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 shutdown_connections 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f53780a3240 0x7f53780a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f538c106f10 0x7f538c19cd60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f538c102790 0x7f538c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 --2- 172.21.15.16:0/1853405345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f538c101de0 0x7f538c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 >> 172.21.15.16:0/1853405345 conn(0x7f538c0fd8c0 msgr2=0x7f538c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 shutdown_connections 2024-09-15T22:35:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:41.522+0000 7f539133e640 1 -- 172.21.15.16:0/1853405345 wait complete. 2024-09-15T22:35:41.528 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:41 smithi130 podman[65314]: 2024-09-15 22:35:41.235139461 +0000 UTC m=+0.384577509 container start ef1819beb068ccd49f3afadcd52590c5f7af638d41aea26344182d37d55fd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-15T22:35:41.529 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:41 smithi130 ceph-mds[65334]: -- 172.21.15.130:0/1921266494 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d9565fc20 con 0x556d9563d000 2024-09-15T22:35:41.529 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:41 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f[65330]: starting mds.f at 2024-09-15T22:35:41.529 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:41 smithi130 bash[65314]: ef1819beb068ccd49f3afadcd52590c5f7af638d41aea26344182d37d55fd140 2024-09-15T22:35:41.529 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:35:41 smithi130 systemd[1]: Started Ceph mds.f for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: Deploying daemon mds.c on smithi130 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1853405345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6834/2738921095,v1:172.21.15.130:6835/2738921095] up:boot 2024-09-15T22:35:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: fsmap 6 up:standby 2024-09-15T22:35:42.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:42 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:35:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: Deploying daemon mds.c on smithi130 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1853405345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6834/2738921095,v1:172.21.15.130:6835/2738921095] up:boot 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: fsmap 6 up:standby 2024-09-15T22:35:42.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:42 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: Deploying daemon mds.c on smithi130 2024-09-15T22:35:42.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1853405345' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:42.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6834/2738921095,v1:172.21.15.130:6835/2738921095] up:boot 2024-09-15T22:35:42.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: fsmap 6 up:standby 2024-09-15T22:35:42.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:42 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:35:43.529 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 systemd[1]: Starting Ceph mds.c for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:43.881 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 podman[65618]: 2024-09-15 22:35:43.465572173 +0000 UTC m=+0.016543958 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:43.881 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 podman[65618]: 2024-09-15 22:35:43.594349907 +0000 UTC m=+0.145321721 container create 830d2a77a2bcfd8859aae2ff8ab04d0d25c70245762d6e12e4e7692b10899149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:35:43.882 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 podman[65618]: 2024-09-15 22:35:43.880901901 +0000 UTC m=+0.431873674 container init 830d2a77a2bcfd8859aae2ff8ab04d0d25c70245762d6e12e4e7692b10899149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:35:44.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:44.279 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 podman[65618]: 2024-09-15 22:35:43.883845303 +0000 UTC m=+0.434817088 container start 830d2a77a2bcfd8859aae2ff8ab04d0d25c70245762d6e12e4e7692b10899149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-15T22:35:44.279 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 ceph-mds[65638]: -- 172.21.15.130:0/2329935274 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d066b83860 con 0x55d066b60800 2024-09-15T22:35:44.279 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c[65634]: starting mds.c at 2024-09-15T22:35:44.279 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 bash[65618]: 830d2a77a2bcfd8859aae2ff8ab04d0d25c70245762d6e12e4e7692b10899149 2024-09-15T22:35:44.279 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:35:43 smithi130 systemd[1]: Started Ceph mds.c for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:44 smithi016 ceph-mon[31492]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:44 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:44.640 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:44 smithi059 ceph-mon[34872]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:44.640 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.640 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.641 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.641 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:44.641 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:44.641 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:44 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:44 smithi130 ceph-mon[35742]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:44 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 -- 172.21.15.16:0/2699489316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c10e810 msgr2=0x7f1c6c110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 --2- 172.21.15.16:0/2699489316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c10e810 0x7f1c6c110c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009ca0 tx=0x7f1c6402a8d0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 -- 172.21.15.16:0/2699489316 shutdown_connections 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 --2- 172.21.15.16:0/2699489316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c10e810 0x7f1c6c110c00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 --2- 172.21.15.16:0/2699489316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c6c102fc0 0x7f1c6c1005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 --2- 172.21.15.16:0/2699489316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c102b50 0x7f1c6c100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 -- 172.21.15.16:0/2699489316 >> 172.21.15.16:0/2699489316 conn(0x7f1c6c077210 msgr2=0x7f1c6c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:45.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 -- 172.21.15.16:0/2699489316 shutdown_connections 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.001+0000 7f1c73448640 1 -- 172.21.15.16:0/2699489316 wait complete. 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 Processor -- start 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 -- start start 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c102b50 0x7f1c6c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c6c10e810 0x7f1c6c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c19efa0 0x7f1c6c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c114210 con 0x7f1c6c102b50 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c114090 con 0x7f1c6c19efa0 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.002+0000 7f1c73448640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c114390 con 0x7f1c6c10e810 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c102b50 0x7f1c6c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c102b50 0x7f1c6c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46440/0 (socket says 172.21.15.16:46440) 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 -- 172.21.15.16:0/2863427406 learned_addr learned my addr 172.21.15.16:0/2863427406 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:45.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c719e5640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c19efa0 0x7f1c6c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c709e3640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c6c10e810 0x7f1c6c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 -- 172.21.15.16:0/2863427406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c6c10e810 msgr2=0x7f1c6c19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c6c10e810 0x7f1c6c19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 -- 172.21.15.16:0/2863427406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c19efa0 msgr2=0x7f1c6c19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c19efa0 0x7f1c6c19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 -- 172.21.15.16:0/2863427406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c19fa60 con 0x7f1c6c102b50 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c709e3640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c6c10e810 0x7f1c6c19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c711e4640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c102b50 0x7f1c6c19a6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800d950 tx=0x7f1c6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c627fc640 1 -- 172.21.15.16:0/2863427406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68017030 con 0x7f1c6c102b50 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c19fd50 con 0x7f1c6c102b50 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c627fc640 1 -- 172.21.15.16:0/2863427406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c68004020 con 0x7f1c6c102b50 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c627fc640 1 -- 172.21.15.16:0/2863427406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c680047a0 con 0x7f1c6c102b50 2024-09-15T22:35:45.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.003+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a7530 con 0x7f1c6c102b50 2024-09-15T22:35:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.004+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c100090 con 0x7f1c6c102b50 2024-09-15T22:35:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.005+0000 7f1c627fc640 1 -- 172.21.15.16:0/2863427406 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1c680041d0 con 0x7f1c6c102b50 2024-09-15T22:35:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.006+0000 7f1c627fc640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1c580a3240 0x7f1c580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.006+0000 7f1c627fc640 1 -- 172.21.15.16:0/2863427406 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f1c680cefb0 con 0x7f1c6c102b50 2024-09-15T22:35:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.006+0000 7f1c709e3640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1c580a3240 0x7f1c580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.007+0000 7f1c709e3640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1c580a3240 0x7f1c580a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006fd0 tx=0x7f1c5c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:45.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.010+0000 7f1c627fc640 1 -- 172.21.15.16:0/2863427406 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c6809c0d0 con 0x7f1c6c102b50 2024-09-15T22:35:45.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.221+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c6c19ba30 con 0x7f1c6c102b50 2024-09-15T22:35:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.221+0000 7f1c627fc640 1 -- 172.21.15.16:0/2863427406 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6394 (secure 0 0 0) 0x7f1c6809b9d0 con 0x7f1c6c102b50 2024-09-15T22:35:45.223 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:45.223 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"btime":"2024-09-15T22:35:44:383670+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-15T22:35:45.223 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-09-15T22:35:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.223+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1c580a3240 msgr2=0x7f1c580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:45.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.223+0000 7f1c73448640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1c580a3240 0x7f1c580a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006fd0 tx=0x7f1c5c008040 comp rx=0 tx=0).stop 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.223+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c102b50 msgr2=0x7f1c6c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.223+0000 7f1c73448640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c102b50 0x7f1c6c19a6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800d950 tx=0x7f1c6800de10 comp rx=0 tx=0).stop 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 shutdown_connections 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1c580a3240 0x7f1c580a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c6c19efa0 0x7f1c6c19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c6c10e810 0x7f1c6c19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 --2- 172.21.15.16:0/2863427406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6c102b50 0x7f1c6c19a6d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 >> 172.21.15.16:0/2863427406 conn(0x7f1c6c077210 msgr2=0x7f1c6c10fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 shutdown_connections 2024-09-15T22:35:45.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:45.224+0000 7f1c73448640 1 -- 172.21.15.16:0/2863427406 wait complete. 2024-09-15T22:35:45.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:45 smithi016 ceph-mon[31492]: Deploying daemon mds.h on smithi059 2024-09-15T22:35:45.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:45 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6836/1773539140,v1:172.21.15.130:6837/1773539140] up:boot 2024-09-15T22:35:45.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:45 smithi016 ceph-mon[31492]: fsmap 7 up:standby 2024-09-15T22:35:45.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:45 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:35:45.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:45 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2863427406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:45.661 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:45 smithi059 ceph-mon[34872]: Deploying daemon mds.h on smithi059 2024-09-15T22:35:45.661 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:45 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6836/1773539140,v1:172.21.15.130:6837/1773539140] up:boot 2024-09-15T22:35:45.662 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:45 smithi059 ceph-mon[34872]: fsmap 7 up:standby 2024-09-15T22:35:45.662 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:45 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:35:45.662 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:45 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2863427406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:45 smithi130 ceph-mon[35742]: Deploying daemon mds.h on smithi059 2024-09-15T22:35:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:45 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6836/1773539140,v1:172.21.15.130:6837/1773539140] up:boot 2024-09-15T22:35:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:45 smithi130 ceph-mon[35742]: fsmap 7 up:standby 2024-09-15T22:35:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:45 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:35:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:45 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2863427406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:46.194 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 systemd[1]: Starting Ceph mds.h for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:46.625 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:46 smithi059 ceph-mon[34872]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:46.665 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 podman[64961]: 2024-09-15 22:35:46.230514659 +0000 UTC m=+0.016277755 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:46.665 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 podman[64961]: 2024-09-15 22:35:46.355020955 +0000 UTC m=+0.140784037 container create 55618dd24db2f05dc7c381d4fa80af4a9272ab8997686b7d8be0c4f6a3c77882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-15T22:35:46.665 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 podman[64961]: 2024-09-15 22:35:46.624555947 +0000 UTC m=+0.410319030 container init 55618dd24db2f05dc7c381d4fa80af4a9272ab8997686b7d8be0c4f6a3c77882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-15T22:35:46.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:46 smithi130 ceph-mon[35742]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:46.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:46 smithi016 ceph-mon[31492]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:46.944 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 podman[64961]: 2024-09-15 22:35:46.627524545 +0000 UTC m=+0.413287647 container start 55618dd24db2f05dc7c381d4fa80af4a9272ab8997686b7d8be0c4f6a3c77882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-15T22:35:46.944 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 ceph-mds[64980]: -- 172.21.15.59:0/2292384685 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648942e5860 con 0x5648942c2800 2024-09-15T22:35:46.944 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h[64976]: starting mds.h at 2024-09-15T22:35:46.944 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 bash[64961]: 55618dd24db2f05dc7c381d4fa80af4a9272ab8997686b7d8be0c4f6a3c77882 2024-09-15T22:35:46.944 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:35:46 smithi059 systemd[1]: Started Ceph mds.h for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:47.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:48.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:48.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: Deploying daemon mds.i on smithi130 2024-09-15T22:35:48.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] up:boot 2024-09-15T22:35:48.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: fsmap 8 up:standby 2024-09-15T22:35:48.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:47 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: Deploying daemon mds.i on smithi130 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] up:boot 2024-09-15T22:35:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: fsmap 8 up:standby 2024-09-15T22:35:48.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:47 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:35:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:48.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: Deploying daemon mds.i on smithi130 2024-09-15T22:35:48.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] up:boot 2024-09-15T22:35:48.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: fsmap 8 up:standby 2024-09-15T22:35:48.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:47 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:35:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 -- 172.21.15.16:0/1043787027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d81008b0 msgr2=0x7fa7d810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:48.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/1043787027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d81008b0 0x7fa7d810e9b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0009ca0 tx=0x7fa7d002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 -- 172.21.15.16:0/1043787027 shutdown_connections 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/1043787027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d81008b0 0x7fa7d810e9b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/1043787027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d80fff10 0x7fa7d8100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/1043787027 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7d8106af0 0x7fa7d8106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 -- 172.21.15.16:0/1043787027 >> 172.21.15.16:0/1043787027 conn(0x7fa7d8077010 msgr2=0x7fa7d80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 -- 172.21.15.16:0/1043787027 shutdown_connections 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.751+0000 7fa7d7fff640 1 -- 172.21.15.16:0/1043787027 wait complete. 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.752+0000 7fa7d7fff640 1 Processor -- start 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.752+0000 7fa7d7fff640 1 -- start start 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.752+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d80fff10 0x7fa7d8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d81008b0 0x7fa7d81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7d8106af0 0x7fa7d819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d7fff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8111fe0 con 0x7fa7d80fff10 2024-09-15T22:35:48.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d7fff640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8111e60 con 0x7fa7d81008b0 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d7fff640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8112160 con 0x7fa7d8106af0 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d80fff10 0x7fa7d8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d80fff10 0x7fa7d8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46450/0 (socket says 172.21.15.16:46450) 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/3084323053 learned_addr learned my addr 172.21.15.16:0/3084323053 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d67fc640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d81008b0 0x7fa7d81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d77fe640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7d8106af0 0x7fa7d819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/3084323053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7d8106af0 msgr2=0x7fa7d819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7d8106af0 0x7fa7d819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/3084323053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d81008b0 msgr2=0x7fa7d81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d81008b0 0x7fa7d81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 -- 172.21.15.16:0/3084323053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d819d340 con 0x7fa7d80fff10 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d6ffd640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d80fff10 0x7fa7d8198390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800d770 tx=0x7fa7c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7bffff640 1 -- 172.21.15.16:0/3084323053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8017070 con 0x7fa7d80fff10 2024-09-15T22:35:48.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d819d630 con 0x7fa7d80fff10 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7bffff640 1 -- 172.21.15.16:0/3084323053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c8005500 con 0x7fa7d80fff10 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7bffff640 1 -- 172.21.15.16:0/3084323053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8004e00 con 0x7fa7d80fff10 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.753+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81a4f10 con 0x7fa7d80fff10 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.754+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d8100370 con 0x7fa7d80fff10 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.756+0000 7fa7bffff640 1 -- 172.21.15.16:0/3084323053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa7c8004fa0 con 0x7fa7d80fff10 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.756+0000 7fa7bffff640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa7c00a3240 0x7fa7c00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.757+0000 7fa7d67fc640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa7c00a3240 0x7fa7c00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.757+0000 7fa7bffff640 1 -- 172.21.15.16:0/3084323053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa7c80cf270 con 0x7fa7d80fff10 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.757+0000 7fa7d67fc640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa7c00a3240 0x7fa7c00a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80ff550 tx=0x7fa7cc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:48.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.759+0000 7fa7bffff640 1 -- 172.21.15.16:0/3084323053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7c809bde0 con 0x7fa7d80fff10 2024-09-15T22:35:48.778 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:48 smithi130 systemd[1]: Starting Ceph mds.i for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.975+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7d8103310 con 0x7fa7d80fff10 2024-09-15T22:35:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.975+0000 7fa7bffff640 1 -- 172.21.15.16:0/3084323053 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7219 (secure 0 0 0) 0x7fa7c80a00e0 con 0x7fa7d80fff10 2024-09-15T22:35:48.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:48.977 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-15T22:35:47:395985+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:35:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-15T22:35:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa7c00a3240 msgr2=0x7fa7c00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa7c00a3240 0x7fa7c00a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80ff550 tx=0x7fa7cc00b390 comp rx=0 tx=0).stop 2024-09-15T22:35:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d80fff10 msgr2=0x7fa7d8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d80fff10 0x7fa7d8198390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800d770 tx=0x7fa7c800dc30 comp rx=0 tx=0).stop 2024-09-15T22:35:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 shutdown_connections 2024-09-15T22:35:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa7c00a3240 0x7fa7c00a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa7d8106af0 0x7fa7d819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d81008b0 0x7fa7d81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 --2- 172.21.15.16:0/3084323053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7d80fff10 0x7fa7d8198390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 >> 172.21.15.16:0/3084323053 conn(0x7fa7d8077010 msgr2=0x7fa7d8104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 shutdown_connections 2024-09-15T22:35:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:48.978+0000 7fa7d7fff640 1 -- 172.21.15.16:0/3084323053 wait complete. 2024-09-15T22:35:49.164 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:48 smithi130 podman[65923]: 2024-09-15 22:35:48.781640558 +0000 UTC m=+0.017431869 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:49.164 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:48 smithi130 podman[65923]: 2024-09-15 22:35:48.89361141 +0000 UTC m=+0.129402721 container create e5b9681439796e97fb7d0237243a4e711f389b24aa7dc2a45d32c66e84836603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-15T22:35:49.164 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:49 smithi130 podman[65923]: 2024-09-15 22:35:49.163456759 +0000 UTC m=+0.399248071 container init e5b9681439796e97fb7d0237243a4e711f389b24aa7dc2a45d32c66e84836603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:35:49.529 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:49 smithi130 podman[65923]: 2024-09-15 22:35:49.16643542 +0000 UTC m=+0.402226732 container start e5b9681439796e97fb7d0237243a4e711f389b24aa7dc2a45d32c66e84836603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-15T22:35:49.529 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mds[65942]: -- 172.21.15.130:0/3659142401 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5619e62c1860 con 0x5619e629f400 2024-09-15T22:35:49.529 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i[65938]: starting mds.i at 2024-09-15T22:35:49.529 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:49 smithi130 bash[65923]: e5b9681439796e97fb7d0237243a4e711f389b24aa7dc2a45d32c66e84836603 2024-09-15T22:35:49.529 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:35:49 smithi130 systemd[1]: Started Ceph mds.i for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:50.024 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3084323053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:50.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:49 smithi016 ceph-mon[31492]: Deploying daemon mds.g on smithi016 2024-09-15T22:35:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3084323053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:50.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:50.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:49 smithi059 ceph-mon[34872]: Deploying daemon mds.g on smithi016 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3084323053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:50.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:49 smithi130 ceph-mon[35742]: Deploying daemon mds.g on smithi016 2024-09-15T22:35:50.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:50 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] up:boot 2024-09-15T22:35:50.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:50 smithi016 ceph-mon[31492]: fsmap 9 up:standby 2024-09-15T22:35:50.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:50 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:35:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:50 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] up:boot 2024-09-15T22:35:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:50 smithi059 ceph-mon[34872]: fsmap 9 up:standby 2024-09-15T22:35:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:50 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:35:51.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:50 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] up:boot 2024-09-15T22:35:51.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:50 smithi130 ceph-mon[35742]: fsmap 9 up:standby 2024-09-15T22:35:51.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:50 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:35:51.557 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 systemd[1]: Starting Ceph mds.g for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:51.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:51.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:51 smithi016 ceph-mon[31492]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:51.873 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 podman[71638]: 2024-09-15 22:35:51.50807797 +0000 UTC m=+0.016564005 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:51.873 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 podman[71638]: 2024-09-15 22:35:51.627561392 +0000 UTC m=+0.136047464 container create 7f183bd1c09f7869abea84e24e13ce8cfe5a903a7be225f35fa19d11e7c633f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T22:35:51.873 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 podman[71638]: 2024-09-15 22:35:51.871903517 +0000 UTC m=+0.380389546 container init 7f183bd1c09f7869abea84e24e13ce8cfe5a903a7be225f35fa19d11e7c633f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-15T22:35:52.077 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:52.180 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:51 smithi130 ceph-mon[35742]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:52.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:51 smithi059 ceph-mon[34872]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:52.200 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 podman[71638]: 2024-09-15 22:35:51.876702187 +0000 UTC m=+0.385188215 container start 7f183bd1c09f7869abea84e24e13ce8cfe5a903a7be225f35fa19d11e7c633f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:35:52.200 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 ceph-mds[71678]: -- 172.21.15.16:0/4117711155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f72a33860 con 0x561f72a10800 2024-09-15T22:35:52.200 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g[71672]: starting mds.g at 2024-09-15T22:35:52.201 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 bash[71638]: 7f183bd1c09f7869abea84e24e13ce8cfe5a903a7be225f35fa19d11e7c633f2 2024-09-15T22:35:52.201 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:35:51 smithi016 systemd[1]: Started Ceph mds.g for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:52.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.558+0000 7fef9c697640 1 -- 172.21.15.16:0/1337374138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9410e470 msgr2=0x7fef941108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.558+0000 7fef9c697640 1 --2- 172.21.15.16:0/1337374138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9410e470 0x7fef941108a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ca0 tx=0x7fef8402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- 172.21.15.16:0/1337374138 shutdown_connections 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 --2- 172.21.15.16:0/1337374138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9410e470 0x7fef941108a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 --2- 172.21.15.16:0/1337374138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef94074f50 0x7fef9410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 --2- 172.21.15.16:0/1337374138 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef940745a0 0x7fef94074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- 172.21.15.16:0/1337374138 >> 172.21.15.16:0/1337374138 conn(0x7fef940fd8d0 msgr2=0x7fef940ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- 172.21.15.16:0/1337374138 shutdown_connections 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- 172.21.15.16:0/1337374138 wait complete. 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 Processor -- start 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- start start 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef940745a0 0x7fef9419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:52.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef94074f50 0x7fef9419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef9410e470 0x7fef9419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef94114240 con 0x7fef94074f50 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef941140c0 con 0x7fef9410e470 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.559+0000 7fef9c697640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fef941143c0 con 0x7fef940745a0 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef94074f50 0x7fef9419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef94074f50 0x7fef9419ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57480/0 (socket says 172.21.15.16:57480) 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 -- 172.21.15.16:0/4074275997 learned_addr learned my addr 172.21.15.16:0/4074275997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef9a433640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef940745a0 0x7fef9419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef9ac34640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef9410e470 0x7fef9419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 -- 172.21.15.16:0/4074275997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef940745a0 msgr2=0x7fef9419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef940745a0 0x7fef9419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 -- 172.21.15.16:0/4074275997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef9410e470 msgr2=0x7fef9419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:52.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef9410e470 0x7fef9419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 -- 172.21.15.16:0/4074275997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9419f690 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef9ac34640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef9410e470 0x7fef9419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef99c32640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef94074f50 0x7fef9419ac20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef8800b4a0 tx=0x7fef8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef937fe640 1 -- 172.21.15.16:0/4074275997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88014070 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef937fe640 1 -- 172.21.15.16:0/4074275997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef88010af0 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef937fe640 1 -- 172.21.15.16:0/4074275997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88017d10 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9419f980 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.560+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef941a7260 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.561+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef94074980 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.562+0000 7fef937fe640 1 -- 172.21.15.16:0/4074275997 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fef88017070 con 0x7fef94074f50 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.563+0000 7fef937fe640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef7c0a32c0 0x7fef7c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.563+0000 7fef9a433640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef7c0a32c0 0x7fef7c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:52.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.563+0000 7fef937fe640 1 -- 172.21.15.16:0/4074275997 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fef880ce800 con 0x7fef94074f50 2024-09-15T22:35:52.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.564+0000 7fef9a433640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef7c0a32c0 0x7fef7c0a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005d00 tx=0x7fef8c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:52.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.568+0000 7fef937fe640 1 -- 172.21.15.16:0/4074275997 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef8809b920 con 0x7fef94074f50 2024-09-15T22:35:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.783+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef94109060 con 0x7fef94074f50 2024-09-15T22:35:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.784+0000 7fef937fe640 1 -- 172.21.15.16:0/4074275997 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8049 (secure 0 0 0) 0x7fef8809b220 con 0x7fef94074f50 2024-09-15T22:35:52.786 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:52.786 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"btime":"2024-09-15T22:35:49:776075+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:35:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-09-15T22:35:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef7c0a32c0 msgr2=0x7fef7c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef7c0a32c0 0x7fef7c0a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005d00 tx=0x7fef8c00b410 comp rx=0 tx=0).stop 2024-09-15T22:35:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef94074f50 msgr2=0x7fef9419ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef94074f50 0x7fef9419ac20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef8800b4a0 tx=0x7fef8800b960 comp rx=0 tx=0).stop 2024-09-15T22:35:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 shutdown_connections 2024-09-15T22:35:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef7c0a32c0 0x7fef7c0a5780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef9410e470 0x7fef9419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef94074f50 0x7fef9419ac20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 --2- 172.21.15.16:0/4074275997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef940745a0 0x7fef9419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 >> 172.21.15.16:0/4074275997 conn(0x7fef940fd8d0 msgr2=0x7fef940fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 shutdown_connections 2024-09-15T22:35:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:52.786+0000 7fef9c697640 1 -- 172.21.15.16:0/4074275997 wait complete. 2024-09-15T22:35:53.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:53.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:53.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:53.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: Deploying daemon mds.a on smithi016 2024-09-15T22:35:53.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4074275997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:53.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.16:6838/1483095493,v1:172.21.15.16:6839/1483095493] up:boot 2024-09-15T22:35:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: fsmap 10 up:standby 2024-09-15T22:35:53.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:53 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: Deploying daemon mds.a on smithi016 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4074275997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.16:6838/1483095493,v1:172.21.15.16:6839/1483095493] up:boot 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: fsmap 10 up:standby 2024-09-15T22:35:53.403 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:53 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: Deploying daemon mds.a on smithi016 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4074275997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:53.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.16:6838/1483095493,v1:172.21.15.16:6839/1483095493] up:boot 2024-09-15T22:35:53.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: fsmap 10 up:standby 2024-09-15T22:35:53.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:53 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:35:54.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:54 smithi016 ceph-mon[31492]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:54.345 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:54 smithi059 ceph-mon[34872]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:54.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:54 smithi130 ceph-mon[35742]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:55.307 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 systemd[1]: Starting Ceph mds.a for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:55.612 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 podman[72057]: 2024-09-15 22:35:55.252868486 +0000 UTC m=+0.017336982 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:55.613 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 podman[72057]: 2024-09-15 22:35:55.367512585 +0000 UTC m=+0.131981063 container create f678fa8d9cad7b606d746657064fef4fed528b33028cce78057c950ab11bbb34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:35:55.613 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 podman[72057]: 2024-09-15 22:35:55.611912876 +0000 UTC m=+0.376381362 container init f678fa8d9cad7b606d746657064fef4fed528b33028cce78057c950ab11bbb34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True) 2024-09-15T22:35:55.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:56.015 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:56.056 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 podman[72057]: 2024-09-15 22:35:55.615001585 +0000 UTC m=+0.379470251 container start f678fa8d9cad7b606d746657064fef4fed528b33028cce78057c950ab11bbb34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-15T22:35:56.057 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 ceph-mds[72076]: -- 172.21.15.16:0/2314057562 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e36b753860 con 0x55e36b730800 2024-09-15T22:35:56.057 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a[72072]: starting mds.a at 2024-09-15T22:35:56.057 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 bash[72057]: f678fa8d9cad7b606d746657064fef4fed528b33028cce78057c950ab11bbb34 2024-09-15T22:35:56.057 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:35:55 smithi016 systemd[1]: Started Ceph mds.a for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:56 smithi016 ceph-mon[31492]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:56.321 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:56 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:56.329 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:56 smithi059 ceph-mon[34872]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:56.329 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.329 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.329 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.329 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:56.329 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:56.329 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:56 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 -- 172.21.15.16:0/1738415085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f410c180 msgr2=0x7fa3f410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/1738415085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f410c180 0x7fa3f410e610 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009ca0 tx=0x7fa3ec02a950 comp rx=0 tx=0).stop 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 -- 172.21.15.16:0/1738415085 shutdown_connections 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/1738415085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f410c180 0x7fa3f410e610 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/1738415085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3f4103700 0x7fa3f410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/1738415085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3f4102de0 0x7fa3f41031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 -- 172.21.15.16:0/1738415085 >> 172.21.15.16:0/1738415085 conn(0x7fa3f40fc7a0 msgr2=0x7fa3f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 -- 172.21.15.16:0/1738415085 shutdown_connections 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.499+0000 7fa3fa18d640 1 -- 172.21.15.16:0/1738415085 wait complete. 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 Processor -- start 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 -- start start 2024-09-15T22:35:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f4102de0 0x7fa3f4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3f4103700 0x7fa3f41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3f410c180 0x7fa3f419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4111fb0 con 0x7fa3f4102de0 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4111e30 con 0x7fa3f410c180 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3fa18d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4112130 con 0x7fa3f4103700 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f4102de0 0x7fa3f4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f4102de0 0x7fa3f4198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57522/0 (socket says 172.21.15.16:57522) 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3f37fe640 1 -- 172.21.15.16:0/307766852 learned_addr learned my addr 172.21.15.16:0/307766852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.500+0000 7fa3f3fff640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3f410c180 0x7fa3f419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f2ffd640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3f4103700 0x7fa3f41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f37fe640 1 -- 172.21.15.16:0/307766852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3f4103700 msgr2=0x7fa3f41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f37fe640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3f4103700 0x7fa3f41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f37fe640 1 -- 172.21.15.16:0/307766852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3f410c180 msgr2=0x7fa3f419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f37fe640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3f410c180 0x7fa3f419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f37fe640 1 -- 172.21.15.16:0/307766852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f419d410 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f2ffd640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3f4103700 0x7fa3f41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f3fff640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3f410c180 0x7fa3f419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f37fe640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f4102de0 0x7fa3f4198460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c890 tx=0x7fa3e400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f0ff9640 1 -- 172.21.15.16:0/307766852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4004220 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f0ff9640 1 -- 172.21.15.16:0/307766852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3e40043c0 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f419d700 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3f0ff9640 1 -- 172.21.15.16:0/307766852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40049e0 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.501+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a4fe0 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.502+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f41031c0 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.503+0000 7fa3f0ff9640 1 -- 172.21.15.16:0/307766852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa3e40079b0 con 0x7fa3f4102de0 2024-09-15T22:35:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.504+0000 7fa3f0ff9640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa3dc0a32c0 0x7fa3dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.504+0000 7fa3f2ffd640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa3dc0a32c0 0x7fa3dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.504+0000 7fa3f0ff9640 1 -- 172.21.15.16:0/307766852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa3e40cf5e0 con 0x7fa3f4102de0 2024-09-15T22:35:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.504+0000 7fa3f2ffd640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa3dc0a32c0 0x7fa3dc0a5780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8006fd0 tx=0x7fa3e8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.508+0000 7fa3f0ff9640 1 -- 172.21.15.16:0/307766852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3e4014030 con 0x7fa3f4102de0 2024-09-15T22:35:56.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:56 smithi130 ceph-mon[35742]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:35:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:35:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:56 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.720+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f402d7c0 con 0x7fa3f4102de0 2024-09-15T22:35:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.721+0000 7fa3f0ff9640 1 -- 172.21.15.16:0/307766852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9698 (secure 0 0 0) 0x7fa3e409c700 con 0x7fa3f4102de0 2024-09-15T22:35:56.723 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:35:56.723 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"btime":"2024-09-15T22:35:56:059220+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:35:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa3dc0a32c0 msgr2=0x7fa3dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa3dc0a32c0 0x7fa3dc0a5780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8006fd0 tx=0x7fa3e8008040 comp rx=0 tx=0).stop 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f4102de0 msgr2=0x7fa3f4198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f4102de0 0x7fa3f4198460 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c890 tx=0x7fa3e400cd50 comp rx=0 tx=0).stop 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 shutdown_connections 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa3dc0a32c0 0x7fa3dc0a5780 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3f410c180 0x7fa3f419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa3f4103700 0x7fa3f41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 --2- 172.21.15.16:0/307766852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f4102de0 0x7fa3f4198460 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 >> 172.21.15.16:0/307766852 conn(0x7fa3f40fc7a0 msgr2=0x7fa3f40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 shutdown_connections 2024-09-15T22:35:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:35:56.723+0000 7fa3fa18d640 1 -- 172.21.15.16:0/307766852 wait complete. 2024-09-15T22:35:57.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:57 smithi059 ceph-mon[34872]: Deploying daemon mds.b on smithi059 2024-09-15T22:35:57.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:57 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.16:6840/433045305,v1:172.21.15.16:6841/433045305] up:boot 2024-09-15T22:35:57.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:57 smithi059 ceph-mon[34872]: fsmap 11 up:standby 2024-09-15T22:35:57.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:57 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:35:57.435 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:57 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/307766852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:57.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:57 smithi016 ceph-mon[31492]: Deploying daemon mds.b on smithi059 2024-09-15T22:35:57.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:57 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.16:6840/433045305,v1:172.21.15.16:6841/433045305] up:boot 2024-09-15T22:35:57.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:57 smithi016 ceph-mon[31492]: fsmap 11 up:standby 2024-09-15T22:35:57.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:57 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:35:57.455 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:57 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/307766852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:57.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:57 smithi130 ceph-mon[35742]: Deploying daemon mds.b on smithi059 2024-09-15T22:35:57.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:57 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.16:6840/433045305,v1:172.21.15.16:6841/433045305] up:boot 2024-09-15T22:35:57.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:57 smithi130 ceph-mon[35742]: fsmap 11 up:standby 2024-09-15T22:35:57.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:57 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:35:57.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:57 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/307766852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:35:58.072 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:57 smithi059 systemd[1]: Starting Ceph mds.b for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:35:58.372 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:58 smithi059 ceph-mon[34872]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:58.373 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 podman[65274]: 2024-09-15 22:35:58.028179482 +0000 UTC m=+0.015740081 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:35:58.373 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 podman[65274]: 2024-09-15 22:35:58.15305174 +0000 UTC m=+0.140612337 container create 6d8fe8545d836e87531673f6d0c54260c9a02d00d84309bbf24c1a6ed31bd239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-15T22:35:58.373 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 podman[65274]: 2024-09-15 22:35:58.371981135 +0000 UTC m=+0.359541732 container init 6d8fe8545d836e87531673f6d0c54260c9a02d00d84309bbf24c1a6ed31bd239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-15T22:35:58.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:58 smithi130 ceph-mon[35742]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:58.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:58 smithi016 ceph-mon[31492]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:58.694 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 podman[65274]: 2024-09-15 22:35:58.374936354 +0000 UTC m=+0.362496951 container start 6d8fe8545d836e87531673f6d0c54260c9a02d00d84309bbf24c1a6ed31bd239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:35:58.694 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 ceph-mds[65293]: -- 172.21.15.59:0/659500063 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f647b3860 con 0x556f64790800 2024-09-15T22:35:58.694 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: starting mds.b at 2024-09-15T22:35:58.694 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 bash[65274]: 6d8fe8545d836e87531673f6d0c54260c9a02d00d84309bbf24c1a6ed31bd239 2024-09-15T22:35:58.694 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:35:58 smithi059 systemd[1]: Started Ceph mds.b for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:35:59.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:35:59.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] up:boot 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: fsmap 12 up:standby 2024-09-15T22:35:59.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:35:59 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:35:59.543 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.543 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.543 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.544 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.544 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.544 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:59.544 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:59.544 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] up:boot 2024-09-15T22:35:59.544 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: fsmap 12 up:standby 2024-09-15T22:35:59.544 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:35:59 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:35:59.786 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] up:boot 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: fsmap 12 up:standby 2024-09-15T22:35:59.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:35:59 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:36:00.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.270+0000 7f4ab595a640 1 -- 172.21.15.16:0/3507794468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0102570 msgr2=0x7f4ab01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:00.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.270+0000 7f4ab595a640 1 --2- 172.21.15.16:0/3507794468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0102570 0x7f4ab01029b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009900 tx=0x7f4aa402a480 comp rx=0 tx=0).stop 2024-09-15T22:36:00.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 -- 172.21.15.16:0/3507794468 shutdown_connections 2024-09-15T22:36:00.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 --2- 172.21.15.16:0/3507794468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ab0102ef0 0x7f4ab010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 --2- 172.21.15.16:0/3507794468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0102570 0x7f4ab01029b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 --2- 172.21.15.16:0/3507794468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ab0100e20 0x7f4ab0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 -- 172.21.15.16:0/3507794468 >> 172.21.15.16:0/3507794468 conn(0x7f4ab00fc7d0 msgr2=0x7f4ab00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 -- 172.21.15.16:0/3507794468 shutdown_connections 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 -- 172.21.15.16:0/3507794468 wait complete. 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 Processor -- start 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.271+0000 7f4ab595a640 1 -- start start 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0100e20 0x7f4ab019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ab0102570 0x7f4ab019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ab0102ef0 0x7f4ab019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0110f50 con 0x7f4ab0100e20 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab0110dd0 con 0x7f4ab0102570 2024-09-15T22:36:00.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab01110d0 con 0x7f4ab0102ef0 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0100e20 0x7f4ab019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0100e20 0x7f4ab019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57546/0 (socket says 172.21.15.16:57546) 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 -- 172.21.15.16:0/570393598 learned_addr learned my addr 172.21.15.16:0/570393598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaf7fe640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ab0102ef0 0x7f4ab019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aae7fc640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ab0102570 0x7f4ab019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 -- 172.21.15.16:0/570393598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ab0102ef0 msgr2=0x7f4ab019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ab0102ef0 0x7f4ab019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 -- 172.21.15.16:0/570393598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ab0102570 msgr2=0x7f4ab019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ab0102570 0x7f4ab019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 -- 172.21.15.16:0/570393598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab019f690 con 0x7f4ab0100e20 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaf7fe640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ab0102ef0 0x7f4ab019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4aaeffd640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0100e20 0x7f4ab019a6e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000d950 tx=0x7f4aa000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab4958640 1 -- 172.21.15.16:0/570393598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0017070 con 0x7f4ab0100e20 2024-09-15T22:36:00.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab019f980 con 0x7f4ab0100e20 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab4958640 1 -- 172.21.15.16:0/570393598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa0009c30 con 0x7f4ab0100e20 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab4958640 1 -- 172.21.15.16:0/570393598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000f6b0 con 0x7f4ab0100e20 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.272+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab01a7260 con 0x7f4ab0100e20 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.274+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab0101200 con 0x7f4ab0100e20 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.275+0000 7f4ab4958640 1 -- 172.21.15.16:0/570393598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4aa00109d0 con 0x7f4ab0100e20 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.275+0000 7f4ab4958640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a980a3240 0x7f4a980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.275+0000 7f4aae7fc640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a980a3240 0x7f4a980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.276+0000 7f4ab4958640 1 -- 172.21.15.16:0/570393598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f4aa00cf0f0 con 0x7f4ab0100e20 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.276+0000 7f4aae7fc640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a980a3240 0x7f4a980a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009230 tx=0x7f4aa40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:00.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.280+0000 7f4ab4958640 1 -- 172.21.15.16:0/570393598 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4aa0009de0 con 0x7f4ab0100e20 2024-09-15T22:36:00.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.514+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ab019ba40 con 0x7f4ab0100e20 2024-09-15T22:36:00.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.515+0000 7f4ab4958640 1 -- 172.21.15.16:0/570393598 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10521 (secure 0 0 0) 0x7f4aa009bbc0 con 0x7f4ab0100e20 2024-09-15T22:36:00.516 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:00.516 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"btime":"2024-09-15T22:35:59:071295+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24500,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-15T22:36:00.517 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a980a3240 msgr2=0x7f4a980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a980a3240 0x7f4a980a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009230 tx=0x7f4aa40058d0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0100e20 msgr2=0x7f4ab019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0100e20 0x7f4ab019a6e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000d950 tx=0x7f4aa000de10 comp rx=0 tx=0).stop 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 shutdown_connections 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a980a3240 0x7f4a980a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4ab0102ef0 0x7f4ab019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ab0102570 0x7f4ab019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 --2- 172.21.15.16:0/570393598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab0100e20 0x7f4ab019a6e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 >> 172.21.15.16:0/570393598 conn(0x7f4ab00fc7d0 msgr2=0x7f4ab0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 shutdown_connections 2024-09-15T22:36:00.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:00.518+0000 7f4ab595a640 1 -- 172.21.15.16:0/570393598 wait complete. 2024-09-15T22:36:01.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-15T22:36:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:01 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/570393598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:01.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:01 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:01.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:01 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/570393598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:01.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:01 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:01.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:01 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/570393598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:01.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:01 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 -- 172.21.15.16:0/1238563225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102ef0 msgr2=0x7f358810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:02.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 --2- 172.21.15.16:0/1238563225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102ef0 0x7f358810da40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3580009ca0 tx=0x7f358002a950 comp rx=0 tx=0).stop 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 -- 172.21.15.16:0/1238563225 shutdown_connections 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 --2- 172.21.15.16:0/1238563225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102ef0 0x7f358810da40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 --2- 172.21.15.16:0/1238563225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3588102570 0x7f35881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 --2- 172.21.15.16:0/1238563225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3588100e20 0x7f3588101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 -- 172.21.15.16:0/1238563225 >> 172.21.15.16:0/1238563225 conn(0x7f35880fc7d0 msgr2=0x7f35880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 -- 172.21.15.16:0/1238563225 shutdown_connections 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.004+0000 7f358fe39640 1 -- 172.21.15.16:0/1238563225 wait complete. 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 Processor -- start 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 -- start start 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3588100e20 0x7f3588198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102570 0x7f35881989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3588102ef0 0x7f358819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35881120a0 con 0x7f3588102570 2024-09-15T22:36:02.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3588111f20 con 0x7f3588100e20 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358fe39640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3588112220 con 0x7f3588102ef0 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102570 0x7f35881989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102570 0x7f35881989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50980/0 (socket says 172.21.15.16:50980) 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358e3d6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3588102ef0 0x7f358819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358dbd5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3588100e20 0x7f3588198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 -- 172.21.15.16:0/3045622453 learned_addr learned my addr 172.21.15.16:0/3045622453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 -- 172.21.15.16:0/3045622453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3588102ef0 msgr2=0x7f358819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3588102ef0 0x7f358819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 -- 172.21.15.16:0/3045622453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3588100e20 msgr2=0x7f3588198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3588100e20 0x7f3588198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358d3d4640 1 -- 172.21.15.16:0/3045622453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358819d410 con 0x7f3588102570 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.005+0000 7f358e3d6640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3588102ef0 0x7f358819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:02.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.006+0000 7f358d3d4640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102570 0x7f35881989a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f357800d950 tx=0x7f357800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.006+0000 7f357effd640 1 -- 172.21.15.16:0/3045622453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578017070 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.006+0000 7f357effd640 1 -- 172.21.15.16:0/3045622453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3578009c30 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.006+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358819d700 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.006+0000 7f357effd640 1 -- 172.21.15.16:0/3045622453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357800f6b0 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.006+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35881a4fe0 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.007+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3588101200 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.008+0000 7f357effd640 1 -- 172.21.15.16:0/3045622453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f35780109d0 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.009+0000 7f357effd640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f35740a32c0 0x7f35740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.009+0000 7f358dbd5640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f35740a32c0 0x7f35740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.009+0000 7f357effd640 1 -- 172.21.15.16:0/3045622453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f35780cefb0 con 0x7f3588102570 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.010+0000 7f358dbd5640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f35740a32c0 0x7f35740a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3584005d00 tx=0x7f358400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:02.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.013+0000 7f357effd640 1 -- 172.21.15.16:0/3045622453 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3578009de0 con 0x7f3588102570 2024-09-15T22:36:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.207+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f35881a5210 con 0x7f3588102570 2024-09-15T22:36:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.207+0000 7f357effd640 1 -- 172.21.15.16:0/3045622453 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7f357809ba80 con 0x7f3588102570 2024-09-15T22:36:02.209 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:02.209 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-15T22:36:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:36:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-15T22:36:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-15T22:36:02.210 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:36:02.210 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f35740a32c0 msgr2=0x7f35740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f35740a32c0 0x7f35740a5780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3584005d00 tx=0x7f358400b410 comp rx=0 tx=0).stop 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102570 msgr2=0x7f35881989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102570 0x7f35881989a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f357800d950 tx=0x7f357800de10 comp rx=0 tx=0).stop 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 shutdown_connections 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f35740a32c0 0x7f35740a5780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3588102ef0 0x7f358819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3588102570 0x7f35881989a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 --2- 172.21.15.16:0/3045622453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3588100e20 0x7f3588198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 >> 172.21.15.16:0/3045622453 conn(0x7f35880fc7d0 msgr2=0x7f35881068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 shutdown_connections 2024-09-15T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:02.209+0000 7f358fe39640 1 -- 172.21.15.16:0/3045622453 wait complete. 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:02.323 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:02 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3045622453' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:02.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:02 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3045622453' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:36:02.507 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:02.507 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.507 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.507 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:02.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:02.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:02.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:02.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:02 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3045622453' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:36:03.535 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T22:36:03.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T22:36:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:04.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:04.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:03 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:04.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:04.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:04.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:03 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:36:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:03 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:36:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 -- 172.21.15.16:0/507485341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc100e20 msgr2=0x7fd0dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 --2- 172.21.15.16:0/507485341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc100e20 0x7fd0dc101200 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009900 tx=0x7fd0cc02a480 comp rx=0 tx=0).stop 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 -- 172.21.15.16:0/507485341 shutdown_connections 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 --2- 172.21.15.16:0/507485341 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0dc102ef0 0x7fd0dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 --2- 172.21.15.16:0/507485341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0dc102570 0x7fd0dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 --2- 172.21.15.16:0/507485341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc100e20 0x7fd0dc101200 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 -- 172.21.15.16:0/507485341 >> 172.21.15.16:0/507485341 conn(0x7fd0dc0fc7d0 msgr2=0x7fd0dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 -- 172.21.15.16:0/507485341 shutdown_connections 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.362+0000 7fd0e1964640 1 -- 172.21.15.16:0/507485341 wait complete. 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 Processor -- start 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 -- start start 2024-09-15T22:36:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0dc100e20 0x7fd0dc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc102570 0x7fd0dc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0dc102ef0 0x7fd0dc19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc110fa0 con 0x7fd0dc102570 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc110e20 con 0x7fd0dc102ef0 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0e1964640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc111120 con 0x7fd0dc100e20 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc102570 0x7fd0dc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc102570 0x7fd0dc1979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51004/0 (socket says 172.21.15.16:51004) 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 -- 172.21.15.16:0/132992329 learned_addr learned my addr 172.21.15.16:0/132992329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0daffd640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0dc100e20 0x7fd0dc197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0db7fe640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0dc102ef0 0x7fd0dc19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 -- 172.21.15.16:0/132992329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0dc100e20 msgr2=0x7fd0dc197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0dc100e20 0x7fd0dc197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 -- 172.21.15.16:0/132992329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0dc102ef0 msgr2=0x7fd0dc19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0dc102ef0 0x7fd0dc19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.363+0000 7fd0da7fc640 1 -- 172.21.15.16:0/132992329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc19c410 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.364+0000 7fd0da7fc640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc102570 0x7fd0dc1979a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000b4a0 tx=0x7fd0d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.364+0000 7fd0e0962640 1 -- 172.21.15.16:0/132992329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0014070 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.364+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc19c700 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.364+0000 7fd0e0962640 1 -- 172.21.15.16:0/132992329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0d0010af0 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.364+0000 7fd0e0962640 1 -- 172.21.15.16:0/132992329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0017d10 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.364+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc1a3fe0 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.365+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0dc101200 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.366+0000 7fd0e0962640 1 -- 172.21.15.16:0/132992329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd0d0017070 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.367+0000 7fd0e0962640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0c40a3240 0x7fd0c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.367+0000 7fd0daffd640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0c40a3240 0x7fd0c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.367+0000 7fd0e0962640 1 -- 172.21.15.16:0/132992329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fd0d00cec70 con 0x7fd0dc102570 2024-09-15T22:36:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.367+0000 7fd0daffd640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0c40a3240 0x7fd0c40a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009230 tx=0x7fd0cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:04.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.370+0000 7fd0e0962640 1 -- 172.21.15.16:0/132992329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0d009bd90 con 0x7fd0dc102570 2024-09-15T22:36:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:04.545+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd0dc02d7c0 con 0x7fd0dc102570 2024-09-15T22:36:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.196+0000 7fd0e0962640 1 -- 172.21.15.16:0/132992329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7fd0d009b690 con 0x7fd0dc102570 2024-09-15T22:36:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0c40a3240 msgr2=0x7fd0c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0c40a3240 0x7fd0c40a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009230 tx=0x7fd0cc0058d0 comp rx=0 tx=0).stop 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc102570 msgr2=0x7fd0dc1979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc102570 0x7fd0dc1979a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000b4a0 tx=0x7fd0d000b960 comp rx=0 tx=0).stop 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 shutdown_connections 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fd0c40a3240 0x7fd0c40a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0dc102ef0 0x7fd0dc19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0dc102570 0x7fd0dc1979a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:05.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 --2- 172.21.15.16:0/132992329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd0dc100e20 0x7fd0dc197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 >> 172.21.15.16:0/132992329 conn(0x7fd0dc0fc7d0 msgr2=0x7fd0dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 shutdown_connections 2024-09-15T22:36:05.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:05.199+0000 7fd0e1964640 1 -- 172.21.15.16:0/132992329 wait complete. 2024-09-15T22:36:05.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:05 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/132992329' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:36:05.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:05 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/132992329' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:36:05.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:05 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/132992329' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:36:05.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-15T22:36:06.121 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:06.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:06 smithi016 ceph-mon[31492]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:06.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:06 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/132992329' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:36:06.520 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:06 smithi016 ceph-mon[31492]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T22:36:06.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:06 smithi130 ceph-mon[35742]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:06.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:06 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/132992329' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:36:06.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:06 smithi130 ceph-mon[35742]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 -- 172.21.15.16:0/2508140209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c1008d0 msgr2=0x7fbe3c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/2508140209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c1008d0 0x7fbe3c100cb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009900 tx=0x7fbe3802a410 comp rx=0 tx=0).stop 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 -- 172.21.15.16:0/2508140209 shutdown_connections 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/2508140209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c10e480 0x7fbe3c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/2508140209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe3c101280 0x7fbe3c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/2508140209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c1008d0 0x7fbe3c100cb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 -- 172.21.15.16:0/2508140209 >> 172.21.15.16:0/2508140209 conn(0x7fbe3c0fc7a0 msgr2=0x7fbe3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 -- 172.21.15.16:0/2508140209 shutdown_connections 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.608+0000 7fbe43ea3640 1 -- 172.21.15.16:0/2508140209 wait complete. 2024-09-15T22:36:06.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 Processor -- start 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 -- start start 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c1008d0 0x7fbe3c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c101280 0x7fbe3c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe3c10e480 0x7fbe3c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c114190 con 0x7fbe3c101280 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c114010 con 0x7fbe3c10e480 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe43ea3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c114310 con 0x7fbe3c1008d0 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe4143e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c101280 0x7fbe3c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.609+0000 7fbe4143e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c101280 0x7fbe3c19ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51018/0 (socket says 172.21.15.16:51018) 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe4143e640 1 -- 172.21.15.16:0/998466103 learned_addr learned my addr 172.21.15.16:0/998466103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe41c3f640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c1008d0 0x7fbe3c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe42440640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe3c10e480 0x7fbe3c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:06.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe4143e640 1 -- 172.21.15.16:0/998466103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c1008d0 msgr2=0x7fbe3c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe4143e640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c1008d0 0x7fbe3c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe4143e640 1 -- 172.21.15.16:0/998466103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe3c10e480 msgr2=0x7fbe3c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe4143e640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe3c10e480 0x7fbe3c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe4143e640 1 -- 172.21.15.16:0/998466103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3c19f680 con 0x7fbe3c101280 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe4143e640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c101280 0x7fbe3c19ac10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00d950 tx=0x7fbe2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe32ffd640 1 -- 172.21.15.16:0/998466103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c017070 con 0x7fbe3c101280 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe32ffd640 1 -- 172.21.15.16:0/998466103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2c009c30 con 0x7fbe3c101280 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3c19f970 con 0x7fbe3c101280 2024-09-15T22:36:06.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe32ffd640 1 -- 172.21.15.16:0/998466103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00f6b0 con 0x7fbe3c101280 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.610+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe3c1a7250 con 0x7fbe3c101280 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.611+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3c100cb0 con 0x7fbe3c101280 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.612+0000 7fbe32ffd640 1 -- 172.21.15.16:0/998466103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbe2c0109d0 con 0x7fbe3c101280 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.613+0000 7fbe32ffd640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fbe280a32c0 0x7fbe280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.613+0000 7fbe32ffd640 1 -- 172.21.15.16:0/998466103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fbe2c0cf0f0 con 0x7fbe3c101280 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.613+0000 7fbe41c3f640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fbe280a32c0 0x7fbe280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.614+0000 7fbe41c3f640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fbe280a32c0 0x7fbe280a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009230 tx=0x7fbe380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:06.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.617+0000 7fbe32ffd640 1 -- 172.21.15.16:0/998466103 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe2c009de0 con 0x7fbe3c101280 2024-09-15T22:36:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:06 smithi059 ceph-mon[34872]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:06 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/132992329' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:36:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:06 smithi059 ceph-mon[34872]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T22:36:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:06.791+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fbe3c19ba30 con 0x7fbe3c101280 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.230+0000 7fbe32ffd640 1 -- 172.21.15.16:0/998466103 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v77) ==== 171+0+0 (secure 0 0 0) 0x7fbe2c09bb70 con 0x7fbe3c101280 2024-09-15T22:36:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-15T22:36:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.232+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fbe280a32c0 msgr2=0x7fbe280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.232+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fbe280a32c0 0x7fbe280a5780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009230 tx=0x7fbe380058d0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.232+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c101280 msgr2=0x7fbe3c19ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.232+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c101280 0x7fbe3c19ac10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00d950 tx=0x7fbe2c00de10 comp rx=0 tx=0).stop 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 shutdown_connections 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fbe280a32c0 0x7fbe280a5780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe3c10e480 0x7fbe3c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe3c101280 0x7fbe3c19ac10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 --2- 172.21.15.16:0/998466103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe3c1008d0 0x7fbe3c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 >> 172.21.15.16:0/998466103 conn(0x7fbe3c0fc7a0 msgr2=0x7fbe3c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 shutdown_connections 2024-09-15T22:36:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:07.233+0000 7fbe43ea3640 1 -- 172.21.15.16:0/998466103 wait complete. 2024-09-15T22:36:07.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:07 smithi130 ceph-mon[35742]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:36:07.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:07 smithi016 ceph-mon[31492]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:36:07.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:07 smithi059 ceph-mon[34872]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:36:07.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T22:36:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:08.244 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:08 smithi016 ceph-mon[31492]: pgmap v274: 33 pgs: 1 creating+activating, 12 creating+peering, 17 unknown, 3 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:08.245 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:08 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/998466103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:36:08.245 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:08 smithi016 ceph-mon[31492]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:36:08.245 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:08 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/998466103' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:36:08.245 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:08 smithi016 ceph-mon[31492]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:36:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:08 smithi130 ceph-mon[35742]: pgmap v274: 33 pgs: 1 creating+activating, 12 creating+peering, 17 unknown, 3 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:08 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/998466103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:36:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:08 smithi130 ceph-mon[35742]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:36:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:08 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/998466103' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:36:08.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:08 smithi130 ceph-mon[35742]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:36:08.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 -- 172.21.15.16:0/3319320968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0101de0 msgr2=0x7f76c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 --2- 172.21.15.16:0/3319320968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0101de0 0x7f76c01021c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009900 tx=0x7f76b402a410 comp rx=0 tx=0).stop 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 -- 172.21.15.16:0/3319320968 shutdown_connections 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 --2- 172.21.15.16:0/3319320968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76c0106f10 0x7f76c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 --2- 172.21.15.16:0/3319320968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76c0102790 0x7f76c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 --2- 172.21.15.16:0/3319320968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0101de0 0x7f76c01021c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 -- 172.21.15.16:0/3319320968 >> 172.21.15.16:0/3319320968 conn(0x7f76c00fd8c0 msgr2=0x7f76c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 -- 172.21.15.16:0/3319320968 shutdown_connections 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.649+0000 7f76c6d69640 1 -- 172.21.15.16:0/3319320968 wait complete. 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 Processor -- start 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 -- start start 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76c0101de0 0x7f76c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:08.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102790 0x7f76c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76c0106f10 0x7f76c0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0111f70 con 0x7f76c0102790 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0111df0 con 0x7f76c0106f10 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c6d69640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01120f0 con 0x7f76c0101de0 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102790 0x7f76c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102790 0x7f76c0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51028/0 (socket says 172.21.15.16:51028) 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 -- 172.21.15.16:0/917202539 learned_addr learned my addr 172.21.15.16:0/917202539 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c5306640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76c0106f10 0x7f76c0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76c4b05640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76c0101de0 0x7f76c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 -- 172.21.15.16:0/917202539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76c0101de0 msgr2=0x7f76c00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76c0101de0 0x7f76c00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:08.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 -- 172.21.15.16:0/917202539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76c0106f10 msgr2=0x7f76c0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76c0106f10 0x7f76c0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.650+0000 7f76bffff640 1 -- 172.21.15.16:0/917202539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c01a68f0 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.651+0000 7f76bffff640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102790 0x7f76c0074a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c970 tx=0x7f76ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.651+0000 7f76bdffb640 1 -- 172.21.15.16:0/917202539 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac007bb0 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.651+0000 7f76bdffb640 1 -- 172.21.15.16:0/917202539 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac007d50 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.651+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c01a6be0 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.651+0000 7f76bdffb640 1 -- 172.21.15.16:0/917202539 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00f450 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.651+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a7120 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.652+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c01021c0 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.653+0000 7f76bdffb640 1 -- 172.21.15.16:0/917202539 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f76ac004fe0 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.654+0000 7f76bdffb640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f76b00a32c0 0x7f76b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.654+0000 7f76c4b05640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f76b00a32c0 0x7f76b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.654+0000 7f76bdffb640 1 -- 172.21.15.16:0/917202539 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f76ac0cf900 con 0x7f76c0102790 2024-09-15T22:36:08.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.655+0000 7f76c4b05640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f76b00a32c0 0x7f76b00a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f76b40096c0 tx=0x7f76b4031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:08.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.658+0000 7f76bdffb640 1 -- 172.21.15.16:0/917202539 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76ac014030 con 0x7f76c0102790 2024-09-15T22:36:08.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:08 smithi059 ceph-mon[34872]: pgmap v274: 33 pgs: 1 creating+activating, 12 creating+peering, 17 unknown, 3 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:08 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/998466103' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:36:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:08 smithi059 ceph-mon[34872]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:36:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:08 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/998466103' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:36:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:08 smithi059 ceph-mon[34872]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:36:08.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:08.870+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f76c0075a20 con 0x7f76c0102790 2024-09-15T22:36:09.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.249+0000 7f76bdffb640 1 -- 172.21.15.16:0/917202539 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f76c0075a20 con 0x7f76c0102790 2024-09-15T22:36:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-15T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.251+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f76b00a32c0 msgr2=0x7f76b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.251+0000 7f76c6d69640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f76b00a32c0 0x7f76b00a5780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f76b40096c0 tx=0x7f76b4031490 comp rx=0 tx=0).stop 2024-09-15T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.251+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102790 msgr2=0x7f76c0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.251+0000 7f76c6d69640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102790 0x7f76c0074a30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00c970 tx=0x7f76ac00ce30 comp rx=0 tx=0).stop 2024-09-15T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.251+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 shutdown_connections 2024-09-15T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.252+0000 7f76c6d69640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f76b00a32c0 0x7f76b00a5780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.252+0000 7f76c6d69640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76c0106f10 0x7f76c0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.252+0000 7f76c6d69640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c0102790 0x7f76c0074a30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.252+0000 7f76c6d69640 1 --2- 172.21.15.16:0/917202539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76c0101de0 0x7f76c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.252+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 >> 172.21.15.16:0/917202539 conn(0x7f76c00fd8c0 msgr2=0x7f76c00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.252+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 shutdown_connections 2024-09-15T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:09.252+0000 7f76c6d69640 1 -- 172.21.15.16:0/917202539 wait complete. 2024-09-15T22:36:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:09 smithi130 ceph-mon[35742]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:36:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:09 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/917202539' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:36:09.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:09 smithi130 ceph-mon[35742]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:36:09.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:09 smithi016 ceph-mon[31492]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:36:09.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:09 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/917202539' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:36:09.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:09 smithi016 ceph-mon[31492]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:36:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:09 smithi059 ceph-mon[34872]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:36:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:09 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/917202539' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:36:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:09 smithi059 ceph-mon[34872]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:36:10.058 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-15T22:36:10.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-15T22:36:10.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:10 smithi016 ceph-mon[31492]: pgmap v277: 97 pgs: 1 creating+activating, 18 creating+peering, 66 unknown, 12 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:10.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:10 smithi016 ceph-mon[31492]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:36:10.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:10 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/917202539' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:36:10.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:10 smithi016 ceph-mon[31492]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:36:10.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:10 smithi016 ceph-mon[31492]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:36:10.326 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:10 smithi130 ceph-mon[35742]: pgmap v277: 97 pgs: 1 creating+activating, 18 creating+peering, 66 unknown, 12 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:10 smithi130 ceph-mon[35742]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:36:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:10 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/917202539' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:36:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:10 smithi130 ceph-mon[35742]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:36:10.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:10 smithi130 ceph-mon[35742]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:36:10.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:10 smithi059 ceph-mon[34872]: pgmap v277: 97 pgs: 1 creating+activating, 18 creating+peering, 66 unknown, 12 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:10.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:10 smithi059 ceph-mon[34872]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:36:10.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:10 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/917202539' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:36:10.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:10 smithi059 ceph-mon[34872]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:36:10.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:10 smithi059 ceph-mon[34872]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 -- 172.21.15.16:0/3899306416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1036e0 msgr2=0x7f0b0c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 --2- 172.21.15.16:0/3899306416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1036e0 0x7f0b0c107b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009930 tx=0x7f0b0002a420 comp rx=0 tx=0).stop 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 -- 172.21.15.16:0/3899306416 shutdown_connections 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 --2- 172.21.15.16:0/3899306416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b0c1082a0 0x7f0b0c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 --2- 172.21.15.16:0/3899306416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1036e0 0x7f0b0c107b90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 --2- 172.21.15.16:0/3899306416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b0c102d30 0x7f0b0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 -- 172.21.15.16:0/3899306416 >> 172.21.15.16:0/3899306416 conn(0x7f0b0c0fc7d0 msgr2=0x7f0b0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:10.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 -- 172.21.15.16:0/3899306416 shutdown_connections 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.841+0000 7f0b11eef640 1 -- 172.21.15.16:0/3899306416 wait complete. 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 Processor -- start 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 -- start start 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b0c102d30 0x7f0b0c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b0c1036e0 0x7f0b0c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1082a0 0x7f0b0c19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112060 con 0x7f0b0c1082a0 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c111ee0 con 0x7f0b0c1036e0 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b11eef640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1121e0 con 0x7f0b0c102d30 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1082a0 0x7f0b0c19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1082a0 0x7f0b0c19ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51054/0 (socket says 172.21.15.16:51054) 2024-09-15T22:36:10.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.842+0000 7f0b0bfff640 1 -- 172.21.15.16:0/998065436 learned_addr learned my addr 172.21.15.16:0/998065436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0affd640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b0c1036e0 0x7f0b0c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0b7fe640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b0c102d30 0x7f0b0c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0bfff640 1 -- 172.21.15.16:0/998065436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b0c102d30 msgr2=0x7f0b0c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0bfff640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b0c102d30 0x7f0b0c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0bfff640 1 -- 172.21.15.16:0/998065436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b0c1036e0 msgr2=0x7f0b0c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0bfff640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b0c1036e0 0x7f0b0c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0bfff640 1 -- 172.21.15.16:0/998065436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c19d4f0 con 0x7f0b0c1082a0 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b0bfff640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1082a0 0x7f0b0c19ce10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400cad0 tx=0x7f0b0400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b08ff9640 1 -- 172.21.15.16:0/998065436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400ae20 con 0x7f0b0c1082a0 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c19d7e0 con 0x7f0b0c1082a0 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b08ff9640 1 -- 172.21.15.16:0/998065436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b04005020 con 0x7f0b0c1082a0 2024-09-15T22:36:10.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b08ff9640 1 -- 172.21.15.16:0/998065436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400f6f0 con 0x7f0b0c1082a0 2024-09-15T22:36:10.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.843+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a50c0 con 0x7f0b0c1082a0 2024-09-15T22:36:10.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.844+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c103110 con 0x7f0b0c1082a0 2024-09-15T22:36:10.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.846+0000 7f0b08ff9640 1 -- 172.21.15.16:0/998065436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0b040051d0 con 0x7f0b0c1082a0 2024-09-15T22:36:10.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.847+0000 7f0b08ff9640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0af40a3240 0x7f0af40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:10.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.847+0000 7f0b08ff9640 1 -- 172.21.15.16:0/998065436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f0b040cfb50 con 0x7f0b0c1082a0 2024-09-15T22:36:10.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.847+0000 7f0b0b7fe640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0af40a3240 0x7f0af40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:10.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.847+0000 7f0b0b7fe640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0af40a3240 0x7f0af40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005e10 tx=0x7f0afc005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:10.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:10.850+0000 7f0b08ff9640 1 -- 172.21.15.16:0/998065436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b04014030 con 0x7f0b0c1082a0 2024-09-15T22:36:11.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.033+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0b0c1998a0 con 0x7f0b0c1082a0 2024-09-15T22:36:11.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.268+0000 7f0b08ff9640 1 -- 172.21.15.16:0/998065436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f0b0c1998a0 con 0x7f0b0c1082a0 2024-09-15T22:36:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.270+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0af40a3240 msgr2=0x7f0af40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.270+0000 7f0b11eef640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0af40a3240 0x7f0af40a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005e10 tx=0x7f0afc005da0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.270+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1082a0 msgr2=0x7f0b0c19ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.270+0000 7f0b11eef640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1082a0 0x7f0b0c19ce10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400cad0 tx=0x7f0b0400cf90 comp rx=0 tx=0).stop 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 shutdown_connections 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0af40a3240 0x7f0af40a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b0c1082a0 0x7f0b0c19ce10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b0c1036e0 0x7f0b0c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 --2- 172.21.15.16:0/998065436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b0c102d30 0x7f0b0c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 >> 172.21.15.16:0/998065436 conn(0x7f0b0c0fc7d0 msgr2=0x7f0b0c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 shutdown_connections 2024-09-15T22:36:11.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:11.271+0000 7f0b11eef640 1 -- 172.21.15.16:0/998065436 wait complete. 2024-09-15T22:36:11.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:11 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] up:active 2024-09-15T22:36:11.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:11 smithi016 ceph-mon[31492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:11.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:11 smithi016 ceph-mon[31492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:11.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:11 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/998065436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:36:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:11 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] up:active 2024-09-15T22:36:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:11 smithi130 ceph-mon[35742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:11 smithi130 ceph-mon[35742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:11.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:11 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/998065436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:36:11.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:11 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] up:active 2024-09-15T22:36:11.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:11 smithi059 ceph-mon[34872]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:11.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:11 smithi059 ceph-mon[34872]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:11.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:11 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/998065436' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:36:11.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T22:36:12.177 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:12.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:12 smithi130 ceph-mon[35742]: pgmap v279: 97 pgs: 1 creating+activating, 15 creating+peering, 20 unknown, 61 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-09-15T22:36:12.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:12 smithi130 ceph-mon[35742]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:36:12.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:12 smithi130 ceph-mon[35742]: Cluster is now healthy 2024-09-15T22:36:12.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:12 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/998065436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:36:12.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:12 smithi130 ceph-mon[35742]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:36:12.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:12 smithi016 ceph-mon[31492]: pgmap v279: 97 pgs: 1 creating+activating, 15 creating+peering, 20 unknown, 61 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-09-15T22:36:12.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:12 smithi016 ceph-mon[31492]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:36:12.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:12 smithi016 ceph-mon[31492]: Cluster is now healthy 2024-09-15T22:36:12.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:12 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/998065436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:36:12.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:12 smithi016 ceph-mon[31492]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:36:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 -- 172.21.15.16:0/3198200475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44101de0 msgr2=0x7fcb441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 --2- 172.21.15.16:0/3198200475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44101de0 0x7fcb441021c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009900 tx=0x7fcb3402a410 comp rx=0 tx=0).stop 2024-09-15T22:36:12.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 -- 172.21.15.16:0/3198200475 shutdown_connections 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 --2- 172.21.15.16:0/3198200475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb44106f10 0x7fcb4410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 --2- 172.21.15.16:0/3198200475 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcb44102790 0x7fcb441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 --2- 172.21.15.16:0/3198200475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44101de0 0x7fcb441021c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 -- 172.21.15.16:0/3198200475 >> 172.21.15.16:0/3198200475 conn(0x7fcb440fd8c0 msgr2=0x7fcb440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 -- 172.21.15.16:0/3198200475 shutdown_connections 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.660+0000 7fcb4980d640 1 -- 172.21.15.16:0/3198200475 wait complete. 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 Processor -- start 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 -- start start 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcb44101de0 0x7fcb440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb44102790 0x7fcb44074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44106f10 0x7fcb44074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:12.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44112010 con 0x7fcb44106f10 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44111e90 con 0x7fcb44102790 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb4980d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44112190 con 0x7fcb44101de0 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44106f10 0x7fcb44074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44106f10 0x7fcb44074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50722/0 (socket says 172.21.15.16:50722) 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb437fe640 1 -- 172.21.15.16:0/669672875 learned_addr learned my addr 172.21.15.16:0/669672875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb3bfff640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb44102790 0x7fcb44074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.661+0000 7fcb42ffd640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcb44101de0 0x7fcb440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb437fe640 1 -- 172.21.15.16:0/669672875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcb44101de0 msgr2=0x7fcb440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb437fe640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcb44101de0 0x7fcb440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb437fe640 1 -- 172.21.15.16:0/669672875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb44102790 msgr2=0x7fcb44074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb437fe640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb44102790 0x7fcb44074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb437fe640 1 -- 172.21.15.16:0/669672875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb441a6a70 con 0x7fcb44106f10 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb437fe640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44106f10 0x7fcb44074f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b970 tx=0x7fcb3c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:12.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb4880b640 1 -- 172.21.15.16:0/669672875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c019070 con 0x7fcb44106f10 2024-09-15T22:36:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb4880b640 1 -- 172.21.15.16:0/669672875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb3c014440 con 0x7fcb44106f10 2024-09-15T22:36:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb441a6d60 con 0x7fcb44106f10 2024-09-15T22:36:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb4880b640 1 -- 172.21.15.16:0/669672875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0075d0 con 0x7fcb44106f10 2024-09-15T22:36:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.662+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb441a7250 con 0x7fcb44106f10 2024-09-15T22:36:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.663+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb44102240 con 0x7fcb44106f10 2024-09-15T22:36:12.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.664+0000 7fcb4880b640 1 -- 172.21.15.16:0/669672875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fcb3c01c070 con 0x7fcb44106f10 2024-09-15T22:36:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.665+0000 7fcb4880b640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcb300a3240 0x7fcb300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.665+0000 7fcb42ffd640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcb300a3240 0x7fcb300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.665+0000 7fcb4880b640 1 -- 172.21.15.16:0/669672875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7fcb3c0cefd0 con 0x7fcb44106f10 2024-09-15T22:36:12.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.666+0000 7fcb42ffd640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcb300a3240 0x7fcb300a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb340096c0 tx=0x7fcb34031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:12.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.669+0000 7fcb4880b640 1 -- 172.21.15.16:0/669672875 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb3c09bc60 con 0x7fcb44106f10 2024-09-15T22:36:12.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:12 smithi059 ceph-mon[34872]: pgmap v279: 97 pgs: 1 creating+activating, 15 creating+peering, 20 unknown, 61 active+clean; 577 KiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 0 op/s 2024-09-15T22:36:12.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:12 smithi059 ceph-mon[34872]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:36:12.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:12 smithi059 ceph-mon[34872]: Cluster is now healthy 2024-09-15T22:36:12.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:12 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/998065436' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:36:12.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:12 smithi059 ceph-mon[34872]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:36:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:12.849+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcb4410a0f0 con 0x7fcb44106f10 2024-09-15T22:36:13.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:13 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/669672875' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:36:13.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:13 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/669672875' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:36:13.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:13 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/669672875' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:36:14.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.271+0000 7fcb4880b640 1 -- 172.21.15.16:0/669672875 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7fcb3c09b560 con 0x7fcb44106f10 2024-09-15T22:36:14.272 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2024-09-15T22:36:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcb300a3240 msgr2=0x7fcb300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcb300a3240 0x7fcb300a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcb340096c0 tx=0x7fcb34031490 comp rx=0 tx=0).stop 2024-09-15T22:36:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44106f10 msgr2=0x7fcb44074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44106f10 0x7fcb44074f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00b970 tx=0x7fcb3c00bd70 comp rx=0 tx=0).stop 2024-09-15T22:36:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 shutdown_connections 2024-09-15T22:36:14.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcb300a3240 0x7fcb300a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb44106f10 0x7fcb44074f70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb44102790 0x7fcb44074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 --2- 172.21.15.16:0/669672875 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcb44101de0 0x7fcb440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.274+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 >> 172.21.15.16:0/669672875 conn(0x7fcb440fd8c0 msgr2=0x7fcb440fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.275+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 shutdown_connections 2024-09-15T22:36:14.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:14.275+0000 7fcb4980d640 1 -- 172.21.15.16:0/669672875 wait complete. 2024-09-15T22:36:14.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:14 smithi059 ceph-mon[34872]: pgmap v281: 97 pgs: 6 creating+peering, 91 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-15T22:36:14.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:14 smithi059 ceph-mon[34872]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:36:14.444 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:36:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: 2024-09-15T22:36:14.079+0000 7f0047508640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T22:36:14.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:14 smithi130 ceph-mon[35742]: pgmap v281: 97 pgs: 6 creating+peering, 91 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-15T22:36:14.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:14 smithi130 ceph-mon[35742]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:36:14.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:14 smithi016 ceph-mon[31492]: pgmap v281: 97 pgs: 6 creating+peering, 91 active+clean; 577 KiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-15T22:36:14.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:14 smithi016 ceph-mon[31492]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:36:15.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-15T22:36:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:15.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:15 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/669672875' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:36:15.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:15 smithi130 ceph-mon[35742]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:36:15.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:15 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/669672875' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:36:15.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:15 smithi016 ceph-mon[31492]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:36:15.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:15 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/669672875' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:36:15.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:15 smithi059 ceph-mon[34872]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:36:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 -- 172.21.15.16:0/3468623112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 msgr2=0x7f04c0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 --2- 172.21.15.16:0/3468623112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c0110810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009ca0 tx=0x7f04b802a950 comp rx=0 tx=0).stop 2024-09-15T22:36:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 -- 172.21.15.16:0/3468623112 shutdown_connections 2024-09-15T22:36:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 --2- 172.21.15.16:0/3468623112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c0110810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 --2- 172.21.15.16:0/3468623112 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04c0074f40 0x7f04c010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:15.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 --2- 172.21.15.16:0/3468623112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c0074620 0x7f04c0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 -- 172.21.15.16:0/3468623112 >> 172.21.15.16:0/3468623112 conn(0x7f04c00fd8c0 msgr2=0x7f04c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 -- 172.21.15.16:0/3468623112 shutdown_connections 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 -- 172.21.15.16:0/3468623112 wait complete. 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.789+0000 7f04c75a1640 1 Processor -- start 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c75a1640 1 -- start start 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c75a1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c0074620 0x7f04c019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c75a1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04c0074f40 0x7f04c019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c75a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c75a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04c01141b0 con 0x7f04c010e3c0 2024-09-15T22:36:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c75a1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0114030 con 0x7f04c0074620 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c75a1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0114330 con 0x7f04c0074f40 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50738/0 (socket says 172.21.15.16:50738) 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 -- 172.21.15.16:0/152592127 learned_addr learned my addr 172.21.15.16:0/152592127 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c533d640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c0074620 0x7f04c019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c4b3c640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04c0074f40 0x7f04c019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 -- 172.21.15.16:0/152592127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04c0074f40 msgr2=0x7f04c019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04c0074f40 0x7f04c019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 -- 172.21.15.16:0/152592127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c0074620 msgr2=0x7f04c019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c0074620 0x7f04c019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c5b3e640 1 -- 172.21.15.16:0/152592127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c019f700 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.790+0000 7f04c4b3c640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04c0074f40 0x7f04c019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.791+0000 7f04c5b3e640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c019f020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04b802ae70 tx=0x7f04b802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.791+0000 7f04b67fc640 1 -- 172.21.15.16:0/152592127 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8034040 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.791+0000 7f04b67fc640 1 -- 172.21.15.16:0/152592127 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b8030920 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.791+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c019f990 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.791+0000 7f04b67fc640 1 -- 172.21.15.16:0/152592127 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8045480 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.791+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c01a72d0 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.792+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c0074a00 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.793+0000 7f04b67fc640 1 -- 172.21.15.16:0/152592127 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f04b8037020 con 0x7f04c010e3c0 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.794+0000 7f04b67fc640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f04ac0a32c0 0x7f04ac0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.794+0000 7f04c533d640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f04ac0a32c0 0x7f04ac0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.794+0000 7f04b67fc640 1 -- 172.21.15.16:0/152592127 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f04b80f0bc0 con 0x7f04c010e3c0 2024-09-15T22:36:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.794+0000 7f04c533d640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f04ac0a32c0 0x7f04ac0a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f04b000bbe0 tx=0x7f04b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.797+0000 7f04b67fc640 1 -- 172.21.15.16:0/152592127 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04b80811b0 con 0x7f04c010e3c0 2024-09-15T22:36:15.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:15.971+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f04c0109540 con 0x7f04c010e3c0 2024-09-15T22:36:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.282+0000 7f04b67fc640 1 -- 172.21.15.16:0/152592127 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f04b80bd6c0 con 0x7f04c010e3c0 2024-09-15T22:36:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2024-09-15T22:36:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f04ac0a32c0 msgr2=0x7f04ac0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f04ac0a32c0 0x7f04ac0a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f04b000bbe0 tx=0x7f04b000b390 comp rx=0 tx=0).stop 2024-09-15T22:36:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 msgr2=0x7f04c019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c019f020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04b802ae70 tx=0x7f04b802f5b0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 shutdown_connections 2024-09-15T22:36:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f04ac0a32c0 0x7f04ac0a5780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04c010e3c0 0x7f04c019f020 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.284+0000 7f04c75a1640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04c0074f40 0x7f04c019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.285+0000 7f04c75a1640 1 --2- 172.21.15.16:0/152592127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04c0074620 0x7f04c019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.285+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 >> 172.21.15.16:0/152592127 conn(0x7f04c00fd8c0 msgr2=0x7f04c00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.285+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 shutdown_connections 2024-09-15T22:36:16.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:16.285+0000 7f04c75a1640 1 -- 172.21.15.16:0/152592127 wait complete. 2024-09-15T22:36:16.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:16 smithi016 ceph-mon[31492]: pgmap v284: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-15T22:36:16.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:16 smithi016 ceph-mon[31492]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:36:16.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:16 smithi016 ceph-mon[31492]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:36:16.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:16 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/152592127' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:36:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:16 smithi059 ceph-mon[34872]: pgmap v284: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-15T22:36:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:16 smithi059 ceph-mon[34872]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:36:16.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:16 smithi059 ceph-mon[34872]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:36:16.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:16 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/152592127' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:36:16.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:16 smithi130 ceph-mon[35742]: pgmap v284: 129 pgs: 5 creating+peering, 27 unknown, 97 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-15T22:36:16.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:16 smithi130 ceph-mon[35742]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:36:16.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:16 smithi130 ceph-mon[35742]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:36:16.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:16 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/152592127' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:36:16.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-15T22:36:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:17.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:17 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/152592127' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:36:17.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:17 smithi016 ceph-mon[31492]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:36:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 -- 172.21.15.16:0/2704801939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc101420 msgr2=0x7f27bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/2704801939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc101420 0x7f27bc10d950 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009cd0 tx=0x7f27b402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 -- 172.21.15.16:0/2704801939 shutdown_connections 2024-09-15T22:36:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/2704801939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc101420 0x7f27bc10d950 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/2704801939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc100a80 0x7f27bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/2704801939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27bc106a80 0x7f27bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 -- 172.21.15.16:0/2704801939 >> 172.21.15.16:0/2704801939 conn(0x7f27bc0fc7a0 msgr2=0x7f27bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 -- 172.21.15.16:0/2704801939 shutdown_connections 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.662+0000 7f27c0ac3640 1 -- 172.21.15.16:0/2704801939 wait complete. 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 Processor -- start 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 -- start start 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc100a80 0x7f27bc1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27bc101420 0x7f27bc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc106a80 0x7f27bc1a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:17.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc110e80 con 0x7f27bc100a80 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc110d00 con 0x7f27bc106a80 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27c0ac3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc111000 con 0x7f27bc101420 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc100a80 0x7f27bc1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc100a80 0x7f27bc1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50756/0 (socket says 172.21.15.16:50756) 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27ba59c640 1 -- 172.21.15.16:0/76863579 learned_addr learned my addr 172.21.15.16:0/76863579 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.663+0000 7f27bad9d640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc106a80 0x7f27bc1a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27b9d9b640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27bc101420 0x7f27bc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ba59c640 1 -- 172.21.15.16:0/76863579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27bc101420 msgr2=0x7f27bc199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ba59c640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27bc101420 0x7f27bc199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ba59c640 1 -- 172.21.15.16:0/76863579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc106a80 msgr2=0x7f27bc1a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ba59c640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc106a80 0x7f27bc1a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ba59c640 1 -- 172.21.15.16:0/76863579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc1a47b0 con 0x7f27bc100a80 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ba59c640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc100a80 0x7f27bc1991f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c920 tx=0x7f27ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:17.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ab7fe640 1 -- 172.21.15.16:0/76863579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac007c70 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc1a4980 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ab7fe640 1 -- 172.21.15.16:0/76863579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac007e10 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27ab7fe640 1 -- 172.21.15.16:0/76863579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac018890 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.664+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a4ec0 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.665+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc100ee0 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.666+0000 7f27ab7fe640 1 -- 172.21.15.16:0/76863579 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f27ac018a50 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.667+0000 7f27ab7fe640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f27a40a3240 0x7f27a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.667+0000 7f27b9d9b640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f27a40a3240 0x7f27a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.667+0000 7f27ab7fe640 1 -- 172.21.15.16:0/76863579 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f27ac0cfee0 con 0x7f27bc100a80 2024-09-15T22:36:17.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.667+0000 7f27b9d9b640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f27a40a3240 0x7f27a40a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005fd0 tx=0x7f27b0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:17.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.670+0000 7f27ab7fe640 1 -- 172.21.15.16:0/76863579 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27ac014030 con 0x7f27bc100a80 2024-09-15T22:36:17.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:17 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/152592127' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:36:17.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:17 smithi059 ceph-mon[34872]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:36:17.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:17 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/152592127' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:36:17.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:17 smithi130 ceph-mon[35742]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:36:17.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:17.884+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f27bc1a51b0 con 0x7f27bc100a80 2024-09-15T22:36:18.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.289+0000 7f27ab7fe640 1 -- 172.21.15.16:0/76863579 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f27ac09ca10 con 0x7f27bc100a80 2024-09-15T22:36:18.290 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2024-09-15T22:36:18.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f27a40a3240 msgr2=0x7f27a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:18.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f27a40a3240 0x7f27a40a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f27b0005fd0 tx=0x7f27b0005850 comp rx=0 tx=0).stop 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc100a80 msgr2=0x7f27bc1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc100a80 0x7f27bc1991f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c920 tx=0x7f27ac00cde0 comp rx=0 tx=0).stop 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 shutdown_connections 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f27a40a3240 0x7f27a40a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27bc106a80 0x7f27bc1a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f27bc101420 0x7f27bc199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 --2- 172.21.15.16:0/76863579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27bc100a80 0x7f27bc1991f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 >> 172.21.15.16:0/76863579 conn(0x7f27bc0fc7a0 msgr2=0x7f27bc0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 shutdown_connections 2024-09-15T22:36:18.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:18.292+0000 7f27c0ac3640 1 -- 172.21.15.16:0/76863579 wait complete. 2024-09-15T22:36:18.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:18 smithi016 ceph-mon[31492]: pgmap v287: 129 pgs: 9 creating+peering, 12 unknown, 108 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-15T22:36:18.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:18 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/76863579' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:36:18.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:18 smithi059 ceph-mon[34872]: pgmap v287: 129 pgs: 9 creating+peering, 12 unknown, 108 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-15T22:36:18.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:18 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/76863579' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:36:18.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:18 smithi130 ceph-mon[35742]: pgmap v287: 129 pgs: 9 creating+peering, 12 unknown, 108 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 3 op/s 2024-09-15T22:36:18.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:18 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/76863579' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:36:19.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:36:19.264 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:19.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:19 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/76863579' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:36:19.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:19 smithi016 ceph-mon[31492]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:36:19.554 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:19 smithi016 ceph-mon[31492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:19.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:19 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/76863579' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:36:19.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:19 smithi059 ceph-mon[34872]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:36:19.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:19 smithi059 ceph-mon[34872]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 -- 172.21.15.16:0/1354759767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818106a80 msgr2=0x7f1818106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:19.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 --2- 172.21.15.16:0/1354759767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818106a80 0x7f1818106e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1814009930 tx=0x7f181402a420 comp rx=0 tx=0).stop 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 -- 172.21.15.16:0/1354759767 shutdown_connections 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 --2- 172.21.15.16:0/1354759767 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1818101420 0x7f181810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 --2- 172.21.15.16:0/1354759767 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1818100a80 0x7f1818100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 --2- 172.21.15.16:0/1354759767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818106a80 0x7f1818106e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 -- 172.21.15.16:0/1354759767 >> 172.21.15.16:0/1354759767 conn(0x7f18180fc7a0 msgr2=0x7f18180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 -- 172.21.15.16:0/1354759767 shutdown_connections 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 -- 172.21.15.16:0/1354759767 wait complete. 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.737+0000 7f182049a640 1 Processor -- start 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f182049a640 1 -- start start 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f182049a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818100a80 0x7f18181983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f182049a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1818101420 0x7f18181988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:19.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f182049a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1818106a80 0x7f181819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f182049a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1818110e40 con 0x7f1818100a80 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f182049a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1818110cc0 con 0x7f1818101420 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f182049a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1818110fc0 con 0x7f1818106a80 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818100a80 0x7f18181983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818100a80 0x7f18181983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50768/0 (socket says 172.21.15.16:50768) 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 -- 172.21.15.16:0/4094871893 learned_addr learned my addr 172.21.15.16:0/4094871893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181ea37640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1818106a80 0x7f181819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181da35640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1818101420 0x7f18181988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 -- 172.21.15.16:0/4094871893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1818106a80 msgr2=0x7f181819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1818106a80 0x7f181819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 -- 172.21.15.16:0/4094871893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1818101420 msgr2=0x7f18181988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1818101420 0x7f18181988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 -- 172.21.15.16:0/4094871893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181819d350 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.738+0000 7f181e236640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818100a80 0x7f18181983a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1814009230 tx=0x7f1814004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.739+0000 7f180f7fe640 1 -- 172.21.15.16:0/4094871893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181403f030 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.739+0000 7f180f7fe640 1 -- 172.21.15.16:0/4094871893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1814003ad0 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.739+0000 7f180f7fe640 1 -- 172.21.15.16:0/4094871893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814035070 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.739+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181819d5e0 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.739+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181a4f20 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.740+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18181021c0 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.741+0000 7f180f7fe640 1 -- 172.21.15.16:0/4094871893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1814042030 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.742+0000 7f180f7fe640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18040a32c0 0x7f18040a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.742+0000 7f181da35640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18040a32c0 0x7f18040a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.742+0000 7f180f7fe640 1 -- 172.21.15.16:0/4094871893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f18140f04d0 con 0x7f1818100a80 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.742+0000 7f181da35640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18040a32c0 0x7f18040a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18180ff210 tx=0x7f1808005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:19.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.745+0000 7f180f7fe640 1 -- 172.21.15.16:0/4094871893 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18140bcf50 con 0x7f1818100a80 2024-09-15T22:36:19.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:19 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/76863579' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:36:19.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:19 smithi130 ceph-mon[35742]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:36:19.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:19 smithi130 ceph-mon[35742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.955+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1818100ee0 con 0x7f1818100a80 2024-09-15T22:36:19.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.956+0000 7f180f7fe640 1 -- 172.21.15.16:0/4094871893 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7f1818100ee0 con 0x7f1818100a80 2024-09-15T22:36:19.958 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:19.958 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-15T22:36:18:288147+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:17.885942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24500,"qdb_cluster":[24500]},"id":1}]} 2024-09-15T22:36:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18040a32c0 msgr2=0x7f18040a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18040a32c0 0x7f18040a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f18180ff210 tx=0x7f1808005cc0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818100a80 msgr2=0x7f18181983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818100a80 0x7f18181983a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1814009230 tx=0x7f1814004270 comp rx=0 tx=0).stop 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 shutdown_connections 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18040a32c0 0x7f18040a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1818106a80 0x7f181819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1818101420 0x7f18181988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 --2- 172.21.15.16:0/4094871893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1818100a80 0x7f18181983a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.959+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 >> 172.21.15.16:0/4094871893 conn(0x7f18180fc7a0 msgr2=0x7f181806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.960+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 shutdown_connections 2024-09-15T22:36:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:19.960+0000 7f182049a640 1 -- 172.21.15.16:0/4094871893 wait complete. 2024-09-15T22:36:20.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:20 smithi016 ceph-mon[31492]: pgmap v289: 129 pgs: 9 creating+peering, 7 unknown, 113 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:20.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:20 smithi016 ceph-mon[31492]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:36:20.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:20 smithi016 ceph-mon[31492]: Cluster is now healthy 2024-09-15T22:36:20.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:20 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4094871893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:20 smithi059 ceph-mon[34872]: pgmap v289: 129 pgs: 9 creating+peering, 7 unknown, 113 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:20 smithi059 ceph-mon[34872]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:36:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:20 smithi059 ceph-mon[34872]: Cluster is now healthy 2024-09-15T22:36:20.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:20 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4094871893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:20.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:36:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:20 smithi130 ceph-mon[35742]: pgmap v289: 129 pgs: 9 creating+peering, 7 unknown, 113 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:20 smithi130 ceph-mon[35742]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:36:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:20 smithi130 ceph-mon[35742]: Cluster is now healthy 2024-09-15T22:36:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:20 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4094871893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:21.040 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 -- 172.21.15.16:0/721652125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51001009d0 msgr2=0x7f5100100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 --2- 172.21.15.16:0/721652125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51001009d0 0x7f5100100db0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009900 tx=0x7f50f802a410 comp rx=0 tx=0).stop 2024-09-15T22:36:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 -- 172.21.15.16:0/721652125 shutdown_connections 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 --2- 172.21.15.16:0/721652125 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f510010e4f0 0x7f5100110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 --2- 172.21.15.16:0/721652125 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5100101380 0x7f510010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 --2- 172.21.15.16:0/721652125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51001009d0 0x7f5100100db0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 -- 172.21.15.16:0/721652125 >> 172.21.15.16:0/721652125 conn(0x7f51000fc7d0 msgr2=0x7f51000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 -- 172.21.15.16:0/721652125 shutdown_connections 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 -- 172.21.15.16:0/721652125 wait complete. 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 Processor -- start 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.519+0000 7f5107a36640 1 -- start start 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5107a36640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51001009d0 0x7f51001a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5107a36640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5100101380 0x7f51001a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5107a36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f510010e4f0 0x7f510019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5107a36640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5100114270 con 0x7f510010e4f0 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5107a36640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51001140f0 con 0x7f5100101380 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5107a36640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f51001143f0 con 0x7f51001009d0 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f510010e4f0 0x7f510019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f510010e4f0 0x7f510019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50886/0 (socket says 172.21.15.16:50886) 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 -- 172.21.15.16:0/1958815938 learned_addr learned my addr 172.21.15.16:0/1958815938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5104fd1640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5100101380 0x7f51001a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f51057d2640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51001009d0 0x7f51001a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 -- 172.21.15.16:0/1958815938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51001009d0 msgr2=0x7f51001a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51001009d0 0x7f51001a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 -- 172.21.15.16:0/1958815938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5100101380 msgr2=0x7f51001a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5100101380 0x7f51001a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 -- 172.21.15.16:0/1958815938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510019aef0 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f5105fd3640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f510010e4f0 0x7f510019a6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f50f000e9d0 tx=0x7f50f000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.520+0000 7f50fe7fc640 1 -- 172.21.15.16:0/1958815938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f000cd80 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.521+0000 7f50fe7fc640 1 -- 172.21.15.16:0/1958815938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0005020 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.521+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510019b1e0 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.521+0000 7f50fe7fc640 1 -- 172.21.15.16:0/1958815938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f00116d0 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.521+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51001a7380 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.521+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510006a7d0 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.523+0000 7f50fe7fc640 1 -- 172.21.15.16:0/1958815938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f50f00051d0 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.523+0000 7f50fe7fc640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50e80a3240 0x7f50e80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.523+0000 7f50fe7fc640 1 -- 172.21.15.16:0/1958815938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f50f00d0030 con 0x7f510010e4f0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.523+0000 7f51057d2640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50e80a3240 0x7f50e80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.524+0000 7f51057d2640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50e80a3240 0x7f50e80a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009230 tx=0x7f50f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.528+0000 7f50fe7fc640 1 -- 172.21.15.16:0/1958815938 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50f0016030 con 0x7f510010e4f0 2024-09-15T22:36:21.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.702+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5100109680 con 0x7f510010e4f0 2024-09-15T22:36:21.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.705+0000 7f50fe7fc640 1 -- 172.21.15.16:0/1958815938 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7f50f009c4c0 con 0x7f510010e4f0 2024-09-15T22:36:21.706 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:21.707 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:36:21.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50e80a3240 msgr2=0x7f50e80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:21.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50e80a3240 0x7f50e80a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009230 tx=0x7f50f80058d0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f510010e4f0 msgr2=0x7f510019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f510010e4f0 0x7f510019a6c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f50f000e9d0 tx=0x7f50f000ee90 comp rx=0 tx=0).stop 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 shutdown_connections 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50e80a3240 0x7f50e80a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f510010e4f0 0x7f510019a6c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5100101380 0x7f51001a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 --2- 172.21.15.16:0/1958815938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f51001009d0 0x7f51001a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 >> 172.21.15.16:0/1958815938 conn(0x7f51000fc7d0 msgr2=0x7f51000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 shutdown_connections 2024-09-15T22:36:21.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:21.708+0000 7f5107a36640 1 -- 172.21.15.16:0/1958815938 wait complete. 2024-09-15T22:36:22.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:22 smithi016 ceph-mon[31492]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:22.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:22 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1958815938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:22.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:36:22.581 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:22 smithi130 ceph-mon[35742]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:22.581 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:22 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1958815938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:22.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:22 smithi059 ceph-mon[34872]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:22 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1958815938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:23.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.117+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1922746575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 msgr2=0x7fb8a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:23.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.117+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1922746575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a81067e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009900 tx=0x7fb89c02a410 comp rx=0 tx=0).stop 2024-09-15T22:36:23.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.117+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1922746575 shutdown_connections 2024-09-15T22:36:23.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.117+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1922746575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8a8106f10 0x7fb8a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.117+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1922746575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a81067e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.117+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1922746575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8a8101de0 0x7fb8a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.117+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1922746575 >> 172.21.15.16:0/1922746575 conn(0x7fb8a80fd8c0 msgr2=0x7fb8a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.118+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1922746575 shutdown_connections 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.118+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1922746575 wait complete. 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.118+0000 7fb8aea7f640 1 Processor -- start 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.118+0000 7fb8aea7f640 1 -- start start 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8aea7f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8a8101de0 0x7fb8a8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8aea7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8aea7f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8a8106f10 0x7fb8a819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8aea7f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a8111f30 con 0x7fb8a8102790 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8aea7f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a8111db0 con 0x7fb8a8106f10 2024-09-15T22:36:23.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8aea7f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a81120b0 con 0x7fb8a8101de0 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8ac81b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8a8101de0 0x7fb8a8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50910/0 (socket says 172.21.15.16:50910) 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 -- 172.21.15.16:0/1197133701 learned_addr learned my addr 172.21.15.16:0/1197133701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8ad01c640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8a8106f10 0x7fb8a819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 -- 172.21.15.16:0/1197133701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8a8101de0 msgr2=0x7fb8a8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8a8101de0 0x7fb8a8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 -- 172.21.15.16:0/1197133701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8a8106f10 msgr2=0x7fb8a819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8a8106f10 0x7fb8a819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 -- 172.21.15.16:0/1197133701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a819d3c0 con 0x7fb8a8102790 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8ad01c640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8a8106f10 0x7fb8a819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8ac81b640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8a8101de0 0x7fb8a8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:23.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a7fff640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a8198950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb89c006010 tx=0x7fb89c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.119+0000 7fb8a5ffb640 1 -- 172.21.15.16:0/1197133701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c03f030 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.120+0000 7fb8a5ffb640 1 -- 172.21.15.16:0/1197133701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89c003ce0 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.120+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a819d650 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.120+0000 7fb8a5ffb640 1 -- 172.21.15.16:0/1197133701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c035070 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.120+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a81a4f90 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.121+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a81032c0 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.122+0000 7fb8a5ffb640 1 -- 172.21.15.16:0/1197133701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb89c042030 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.122+0000 7fb8a5ffb640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8900a3240 0x7fb8900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.123+0000 7fb8ac81b640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8900a3240 0x7fb8900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.123+0000 7fb8a5ffb640 1 -- 172.21.15.16:0/1197133701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fb89c0effd0 con 0x7fb8a8102790 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.123+0000 7fb8ac81b640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8900a3240 0x7fb8900a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb89800afb0 tx=0x7fb89800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:23.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.127+0000 7fb8a5ffb640 1 -- 172.21.15.16:0/1197133701 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb89c0bca50 con 0x7fb8a8102790 2024-09-15T22:36:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.306+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8a810a0f0 con 0x7fb8a8102790 2024-09-15T22:36:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.307+0000 7fb8a5ffb640 1 -- 172.21.15.16:0/1197133701 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7fb89c0bc350 con 0x7fb8a8102790 2024-09-15T22:36:23.309 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:23.309 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8900a3240 msgr2=0x7fb8900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8900a3240 0x7fb8900a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb89800afb0 tx=0x7fb89800a2d0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 msgr2=0x7fb8a8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a8198950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb89c006010 tx=0x7fb89c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 shutdown_connections 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fb8900a3240 0x7fb8900a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8a8106f10 0x7fb8a819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8a8102790 0x7fb8a8198950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 --2- 172.21.15.16:0/1197133701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb8a8101de0 0x7fb8a8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 >> 172.21.15.16:0/1197133701 conn(0x7fb8a80fd8c0 msgr2=0x7fb8a80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 shutdown_connections 2024-09-15T22:36:23.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:23.309+0000 7fb8aea7f640 1 -- 172.21.15.16:0/1197133701 wait complete. 2024-09-15T22:36:24.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:36:24.293 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:24.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:24 smithi016 ceph-mon[31492]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:24.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:24 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1197133701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:24.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:24 smithi059 ceph-mon[34872]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:24.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:24 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1197133701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:24.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:24 smithi130 ceph-mon[35742]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:24.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:24 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1197133701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 -- 172.21.15.16:0/3059504284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 msgr2=0x7f291c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 --2- 172.21.15.16:0/3059504284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c10e610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2914009ca0 tx=0x7f291402a950 comp rx=0 tx=0).stop 2024-09-15T22:36:24.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 -- 172.21.15.16:0/3059504284 shutdown_connections 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 --2- 172.21.15.16:0/3059504284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c10e610 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 --2- 172.21.15.16:0/3059504284 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291c103660 0x7f291c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 --2- 172.21.15.16:0/3059504284 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f291c102d40 0x7f291c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 -- 172.21.15.16:0/3059504284 >> 172.21.15.16:0/3059504284 conn(0x7f291c0fc7a0 msgr2=0x7f291c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 -- 172.21.15.16:0/3059504284 shutdown_connections 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 -- 172.21.15.16:0/3059504284 wait complete. 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.765+0000 7f2923f91640 1 Processor -- start 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2923f91640 1 -- start start 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2923f91640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291c102d40 0x7f291c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2923f91640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f291c103660 0x7f291c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2923f91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2923f91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f291c111f70 con 0x7f291c10c170 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2923f91640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f291c111df0 con 0x7f291c102d40 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2923f91640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1120f0 con 0x7f291c103660 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50924/0 (socket says 172.21.15.16:50924) 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f2921d2d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291c102d40 0x7f291c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 -- 172.21.15.16:0/1812841176 learned_addr learned my addr 172.21.15.16:0/1812841176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292152c640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f291c103660 0x7f291c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 -- 172.21.15.16:0/1812841176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f291c103660 msgr2=0x7f291c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f291c103660 0x7f291c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 -- 172.21.15.16:0/1812841176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291c102d40 msgr2=0x7f291c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291c102d40 0x7f291c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 -- 172.21.15.16:0/1812841176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c1a68d0 con 0x7f291c10c170 2024-09-15T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.766+0000 7f292252e640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c101420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f291402ae70 tx=0x7f291402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.767+0000 7f2912ffd640 1 -- 172.21.15.16:0/1812841176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914034040 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.767+0000 7f2912ffd640 1 -- 172.21.15.16:0/1812841176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914030920 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.767+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c1a6b60 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.767+0000 7f2912ffd640 1 -- 172.21.15.16:0/1812841176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914045480 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.767+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1a7090 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.768+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c103120 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.769+0000 7f2912ffd640 1 -- 172.21.15.16:0/1812841176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2914037020 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.770+0000 7f2912ffd640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f29080a32c0 0x7f29080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.770+0000 7f2921d2d640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f29080a32c0 0x7f29080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.770+0000 7f2912ffd640 1 -- 172.21.15.16:0/1812841176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f29140f09a0 con 0x7f291c10c170 2024-09-15T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.770+0000 7f2921d2d640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f29080a32c0 0x7f29080a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2918005fd0 tx=0x7f291800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.774+0000 7f2912ffd640 1 -- 172.21.15.16:0/1812841176 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29140bd480 con 0x7f291c10c170 2024-09-15T22:36:24.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.983+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f291c1a72c0 con 0x7f291c10c170 2024-09-15T22:36:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.984+0000 7f2912ffd640 1 -- 172.21.15.16:0/1812841176 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11839 (secure 0 0 0) 0x7f29140bcd80 con 0x7f291c10c170 2024-09-15T22:36:24.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:24.986 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"btime":"2024-09-15T22:36:18:288147+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24469,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24479,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:17.885942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24500},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24500,"qdb_cluster":[24500]},"id":1}]} 2024-09-15T22:36:24.986 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2024-09-15T22:36:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.986+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f29080a32c0 msgr2=0x7f29080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.986+0000 7f2923f91640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f29080a32c0 0x7f29080a5780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2918005fd0 tx=0x7f291800b390 comp rx=0 tx=0).stop 2024-09-15T22:36:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.986+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 msgr2=0x7f291c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.986+0000 7f2923f91640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c101420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f291402ae70 tx=0x7f291402f5b0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 shutdown_connections 2024-09-15T22:36:24.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f29080a32c0 0x7f29080a5780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f291c10c170 0x7f291c101420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f291c103660 0x7f291c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 --2- 172.21.15.16:0/1812841176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291c102d40 0x7f291c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 >> 172.21.15.16:0/1812841176 conn(0x7f291c0fc7a0 msgr2=0x7f291c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 shutdown_connections 2024-09-15T22:36:24.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:24.987+0000 7f2923f91640 1 -- 172.21.15.16:0/1812841176 wait complete. 2024-09-15T22:36:25.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:25 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1812841176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:25.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:25 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1812841176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:25.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:36:25.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:25 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1812841176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:26.391 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:26 smithi016 ceph-mon[31492]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 -- 172.21.15.16:0/3753228692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 msgr2=0x7fa51410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 --2- 172.21.15.16:0/3753228692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51410ea90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009900 tx=0x7fa50c02a410 comp rx=0 tx=0).stop 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 -- 172.21.15.16:0/3753228692 shutdown_connections 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 --2- 172.21.15.16:0/3753228692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51410ea90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 --2- 172.21.15.16:0/3753228692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa514102790 0x7fa5141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 --2- 172.21.15.16:0/3753228692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514101de0 0x7fa5141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 -- 172.21.15.16:0/3753228692 >> 172.21.15.16:0/3753228692 conn(0x7fa5140fd8c0 msgr2=0x7fa5140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 -- 172.21.15.16:0/3753228692 shutdown_connections 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 -- 172.21.15.16:0/3753228692 wait complete. 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 Processor -- start 2024-09-15T22:36:26.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.474+0000 7fa519a42640 1 -- start start 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa519a42640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa514101de0 0x7fa5141983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa519a42640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514102790 0x7fa5141988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa519a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa519a42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa514111f70 con 0x7fa514106f10 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa519a42640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa514111df0 con 0x7fa514102790 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa519a42640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa5141120f0 con 0x7fa514101de0 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50946/0 (socket says 172.21.15.16:50946) 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 -- 172.21.15.16:0/4044911481 learned_addr learned my addr 172.21.15.16:0/4044911481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa512ffd640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa514101de0 0x7fa5141983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5127fc640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514102790 0x7fa5141988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 -- 172.21.15.16:0/4044911481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa514101de0 msgr2=0x7fa5141983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa514101de0 0x7fa5141983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 -- 172.21.15.16:0/4044911481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514102790 msgr2=0x7fa5141988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514102790 0x7fa5141988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 -- 172.21.15.16:0/4044911481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51419d360 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa5137fe640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51419cc80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005ab0 tx=0x7fa50c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa518a40640 1 -- 172.21.15.16:0/4044911481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c03f030 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.475+0000 7fa518a40640 1 -- 172.21.15.16:0/4044911481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa50c003ce0 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.476+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51419d5f0 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.476+0000 7fa518a40640 1 -- 172.21.15.16:0/4044911481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c035070 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.476+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141a4f30 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.477+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5141021c0 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.479+0000 7fa518a40640 1 -- 172.21.15.16:0/4044911481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa50c042030 con 0x7fa514106f10 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.479+0000 7fa518a40640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5000a32c0 0x7fa5000a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.479+0000 7fa512ffd640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5000a32c0 0x7fa5000a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.480+0000 7fa518a40640 1 -- 172.21.15.16:0/4044911481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fa50c0f0550 con 0x7fa514106f10 2024-09-15T22:36:26.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.480+0000 7fa512ffd640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5000a32c0 0x7fa5000a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa504005c40 tx=0x7fa50400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:26.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.482+0000 7fa518a40640 1 -- 172.21.15.16:0/4044911481 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa50c0bd030 con 0x7fa514106f10 2024-09-15T22:36:26.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.661+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa51410a120 con 0x7fa514106f10 2024-09-15T22:36:26.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.662+0000 7fa518a40640 1 -- 172.21.15.16:0/4044911481 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7fa50c0bc930 con 0x7fa514106f10 2024-09-15T22:36:26.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:26.664 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:36:26.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5000a32c0 msgr2=0x7fa5000a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5000a32c0 0x7fa5000a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa504005c40 tx=0x7fa50400a590 comp rx=0 tx=0).stop 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 msgr2=0x7fa51419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51419cc80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005ab0 tx=0x7fa50c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 shutdown_connections 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa5000a32c0 0x7fa5000a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa514106f10 0x7fa51419cc80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa514102790 0x7fa5141988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 --2- 172.21.15.16:0/4044911481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa514101de0 0x7fa5141983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 >> 172.21.15.16:0/4044911481 conn(0x7fa5140fd8c0 msgr2=0x7fa5140fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 shutdown_connections 2024-09-15T22:36:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:26.665+0000 7fa519a42640 1 -- 172.21.15.16:0/4044911481 wait complete. 2024-09-15T22:36:26.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:26 smithi059 ceph-mon[34872]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:26.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:26 smithi130 ceph-mon[35742]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:27.473 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:27 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4044911481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:27.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T22:36:08.871593+0000', 'modified': '2024-09-15T22:36:17.885942+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24500}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6839/384228279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 384228279}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 384228279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24500, 'qdb_cluster': [24500]} 2024-09-15T22:36:27.508 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-15T22:36:27.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:36:27.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:27 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4044911481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:27.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:27 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4044911481' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:28.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 -- 172.21.15.16:0/752119939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34101de0 msgr2=0x7f9d341021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 --2- 172.21.15.16:0/752119939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34101de0 0x7f9d341021c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d2402a410 comp rx=0 tx=0).stop 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 -- 172.21.15.16:0/752119939 shutdown_connections 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 --2- 172.21.15.16:0/752119939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d34106f10 0x7f9d3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 --2- 172.21.15.16:0/752119939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d34102790 0x7f9d341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 --2- 172.21.15.16:0/752119939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34101de0 0x7f9d341021c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 -- 172.21.15.16:0/752119939 >> 172.21.15.16:0/752119939 conn(0x7f9d340fd8c0 msgr2=0x7f9d340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 -- 172.21.15.16:0/752119939 shutdown_connections 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.282+0000 7f9d39940640 1 -- 172.21.15.16:0/752119939 wait complete. 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 Processor -- start 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 -- start start 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d34101de0 0x7f9d341982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34102790 0x7f9d34198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d34106f10 0x7f9d3419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34111f70 con 0x7f9d34102790 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34111df0 con 0x7f9d34101de0 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d39940640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341120f0 con 0x7f9d34106f10 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34102790 0x7f9d34198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34102790 0x7f9d34198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50954/0 (socket says 172.21.15.16:50954) 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d327fc640 1 -- 172.21.15.16:0/3532391373 learned_addr learned my addr 172.21.15.16:0/3532391373 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.283+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d34101de0 0x7f9d341982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d327fc640 1 -- 172.21.15.16:0/3532391373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d34106f10 msgr2=0x7f9d3419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d327fc640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d34106f10 0x7f9d3419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d327fc640 1 -- 172.21.15.16:0/3532391373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d34101de0 msgr2=0x7f9d341982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d327fc640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d34101de0 0x7f9d341982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d327fc640 1 -- 172.21.15.16:0/3532391373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3419d2a0 con 0x7f9d34102790 2024-09-15T22:36:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d327fc640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34102790 0x7f9d34198830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c920 tx=0x7f9d2800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d3893e640 1 -- 172.21.15.16:0/3532391373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28007c70 con 0x7f9d34102790 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d3893e640 1 -- 172.21.15.16:0/3532391373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d28007e10 con 0x7f9d34102790 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3419d590 con 0x7f9d34102790 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d3893e640 1 -- 172.21.15.16:0/3532391373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28018890 con 0x7f9d34102790 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.284+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d341a4e70 con 0x7f9d34102790 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.286+0000 7f9d3893e640 1 -- 172.21.15.16:0/3532391373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9d28018a70 con 0x7f9d34102790 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 Processor -- start 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 -- start start 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab25bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab266670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab268fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555eab1c2090 con 0x555eab266bb0 2024-09-15T22:36:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eab169a70 con 0x555eab25c2f0 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eab1be050 con 0x555eab25b9d0 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab268fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d327fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab268fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50960/0 (socket says 172.21.15.16:50960) 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d327fc640 1 -- 172.21.15.16:0/1605893833 learned_addr learned my addr 172.21.15.16:0/1605893833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d3893e640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9d1c0a32c0 0x7f9d1c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab266670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.287+0000 7f9d337fe640 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab25bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d337fe640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9d1c0a32c0 0x7f9d1c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d327fc640 1 -- 172.21.15.16:0/1605893833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 msgr2=0x555eab25bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d327fc640 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab25bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d327fc640 1 -- 172.21.15.16:0/1605893833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 msgr2=0x555eab266670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d327fc640 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab266670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d327fc640 1 -- 172.21.15.16:0/1605893833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eab176130 con 0x555eab266bb0 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab266670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d3893e640 1 -- 172.21.15.16:0/3532391373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f9d280cfca0 con 0x7f9d34102790 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d327fc640 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab268fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d28098480 tx=0x7f9d28020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de97e2f91110bd70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d337fe640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9d1c0a32c0 0x7f9d1c0a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c013a20 tx=0x7f9d2c005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d19ffb640 1 -- 172.21.15.16:0/1605893833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28014910 con 0x555eab266bb0 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d19ffb640 1 -- 172.21.15.16:0/1605893833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d280bc110 con 0x555eab266bb0 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.288+0000 7f9d19ffb640 1 -- 172.21.15.16:0/1605893833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d28008230 con 0x555eab266bb0 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1605893833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 msgr2=0x555eab268fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab268fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d28098480 tx=0x7f9d28020000 comp rx=0 tx=0).stop 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1605893833 shutdown_connections 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab268fa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab266670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1605893833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab25bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1605893833 >> 172.21.15.16:0/1605893833 conn(0x555eab257f60 msgr2=0x555eab258340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1605893833 shutdown_connections 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1605893833 wait complete. 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.289+0000 7f9d3bd19740 1 -- start start 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 1 Processor -- start 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab266400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25c2f0 0x555eab263750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab266bb0 0x555eab263c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555eab269780 con 0x555eab25b9d0 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eab17cf20 con 0x555eab266bb0 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eab269900 con 0x555eab25c2f0 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d3bd19740 10 client.0 _ll_register_callbacks cb 0x555eab146030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab266400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab266400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50972/0 (socket says 172.21.15.16:50972) 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 -- 172.21.15.16:0/1109191141 learned_addr learned my addr 172.21.15.16:0/1109191141 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d327fc640 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25c2f0 0x555eab263750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d337fe640 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab266bb0 0x555eab263c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 -- 172.21.15.16:0/1109191141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25c2f0 msgr2=0x555eab263750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25c2f0 0x555eab263750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 -- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab266bb0 msgr2=0x555eab263c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab266bb0 0x555eab263c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 -- 172.21.15.16:0/1109191141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eaaf332b0 con 0x555eab25b9d0 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab266400 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d24031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59c4d43dabb9dab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.290+0000 7f9d12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:36:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d12ffd640 1 -- 172.21.15.16:0/1109191141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24035040 con 0x555eab25b9d0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d12ffd640 1 -- 172.21.15.16:0/1109191141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f9d240042c0 con 0x555eab25b9d0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d280bc110 con 0x555eab25b9d0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d12ffd640 1 -- 172.21.15.16:0/1109191141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9d240044c0 con 0x555eab25b9d0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 10 client.14709 fetch_fsmap learned FSMap version 17 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555eab275050 con 0x555eab25b9d0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d12ffd640 1 -- 172.21.15.16:0/1109191141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d280bc110 con 0x555eab25b9d0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d12ffd640 1 -- 172.21.15.16:0/1109191141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f9d2403d8a0 con 0x555eab25b9d0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 10 client.14709 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 10 client.14709 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 20 client.14709 populate_metadata read hostname 'smithi016' 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab278b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 4 client.14709 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eab1cf600 con 0x555eab2776a0 2024-09-15T22:36:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.291+0000 7f9d327fc640 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab278b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.292+0000 7f9d327fc640 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab278b10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157304a30e9f17c5 server_cookie=d759b071253f63 in_seq=0 out_seq=0 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.292+0000 7f9d12ffd640 10 client.14709 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d12ffd640 1 -- 172.21.15.16:0/1109191141 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555eab1cf600 con 0x555eab2776a0 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d12ffd640 10 client.14709 handle_command_reply: tid=0 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 client.14709 shutdown 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 msgr2=0x555eab266400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab266400 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d24031a60 comp rx=0 tx=0).stop 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 shutdown_connections 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab278b10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d12ffd640 0 client.14709 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab266bb0 0x555eab263c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25c2f0 0x555eab263750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/1109191141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab266400 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 >> 172.21.15.16:0/1109191141 conn(0x555eab257f60 msgr2=0x555eab25dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 shutdown_connections 2024-09-15T22:36:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 1 -- 172.21.15.16:0/1109191141 wait complete. 2024-09-15T22:36:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.293+0000 7f9d3bd19740 20 client.14709 trim_cache size 0 max 16384 2024-09-15T22:36:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 Processor -- start 2024-09-15T22:36:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 -- start start 2024-09-15T22:36:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab277d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab25ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab266bb0 0x555eab2609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555eab276e50 con 0x555eab25b9d0 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eab277520 con 0x555eab25c2f0 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eab277bf0 con 0x555eab266bb0 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab277d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d32ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab277d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50976/0 (socket says 172.21.15.16:50976) 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.311+0000 7f9d32ffd640 1 -- 172.21.15.16:0/2013372430 learned_addr learned my addr 172.21.15.16:0/2013372430 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d32ffd640 1 -- 172.21.15.16:0/2013372430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab266bb0 msgr2=0x555eab2609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/2013372430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab266bb0 0x555eab2609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d32ffd640 1 -- 172.21.15.16:0/2013372430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 msgr2=0x555eab25ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/2013372430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab25ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d32ffd640 1 -- 172.21.15.16:0/2013372430 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eab266020 con 0x555eab25b9d0 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/2013372430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab277d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d24000c00 tx=0x7f9d240316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f91ee440d54579 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d197fa640 1 -- 172.21.15.16:0/2013372430 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24035030 con 0x555eab25b9d0 2024-09-15T22:36:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d197fa640 1 -- 172.21.15.16:0/2013372430 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2403c450 con 0x555eab25b9d0 2024-09-15T22:36:28.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.312+0000 7f9d197fa640 1 -- 172.21.15.16:0/2013372430 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24046480 con 0x555eab25b9d0 2024-09-15T22:36:28.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- 172.21.15.16:0/2013372430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 msgr2=0x555eab277d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/2013372430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab277d70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d24000c00 tx=0x7f9d240316d0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- 172.21.15.16:0/2013372430 shutdown_connections 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/2013372430 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab266bb0 0x555eab2609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/2013372430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab25ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/2013372430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab25b9d0 0x555eab277d70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- 172.21.15.16:0/2013372430 >> 172.21.15.16:0/2013372430 conn(0x555eab257f60 msgr2=0x555eab25e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- 172.21.15.16:0/2013372430 shutdown_connections 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- 172.21.15.16:0/2013372430 wait complete. 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- start start 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 Processor -- start 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab2813d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab2604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab260a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555eab278990 con 0x555eab266bb0 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eab2782c0 con 0x555eab25c2f0 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.313+0000 7f9d3bd19740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eab279060 con 0x555eab25b9d0 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d3bd19740 10 client.0 _ll_register_callbacks cb 0x7f9d24009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab260a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab260a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50992/0 (socket says 172.21.15.16:50992) 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 -- 172.21.15.16:0/863140289 learned_addr learned my addr 172.21.15.16:0/863140289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab2813d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d327fc640 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab2604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 -- 172.21.15.16:0/863140289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 msgr2=0x555eab2813d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab2813d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 -- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 msgr2=0x555eab2604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab2604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 -- 172.21.15.16:0/863140289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eab261070 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d337fe640 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab260a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c011ad0 tx=0x7f9d2c009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd2f8cb4a4c6c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d12ffd640 1 -- 172.21.15.16:0/863140289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c01ce00 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d12ffd640 1 -- 172.21.15.16:0/863140289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f9d2c003970 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d2403c450 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d12ffd640 1 -- 172.21.15.16:0/863140289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9d2c003b70 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d3bd19740 10 client.14715 fetch_fsmap learned FSMap version 17 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.314+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555eab2e1db0 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d12ffd640 1 -- 172.21.15.16:0/863140289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2403c450 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d12ffd640 1 -- 172.21.15.16:0/863140289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f9d2c00f040 con 0x555eab266bb0 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d3bd19740 10 client.14715 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T22:36:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d3bd19740 10 client.14715 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d3bd19740 20 client.14715 populate_metadata read hostname 'smithi016' 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab2e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d3bd19740 4 client.14715 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "status"}) -- 0x555eab1c9b10 con 0x555eab2776a0 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.315+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab2e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d32ffd640 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab2e54f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d5d6270a689eda server_cookie=749bd4e272fd30d4 in_seq=0 out_seq=0 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d12ffd640 10 client.14715 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d12ffd640 1 -- 172.21.15.16:0/863140289 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555eab1c9b10 con 0x555eab2776a0 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d12ffd640 10 client.14715 handle_command_reply: tid=0 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 client.14715 shutdown 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 msgr2=0x555eab260a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab260a00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c011ad0 tx=0x7f9d2c009760 comp rx=0 tx=0).stop 2024-09-15T22:36:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 shutdown_connections 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eab2776a0 0x555eab2e54f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eab266bb0 0x555eab260a00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eab25c2f0 0x555eab2604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d12ffd640 0 client.14715 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 --2- 172.21.15.16:0/863140289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eab25b9d0 0x555eab2813d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 >> 172.21.15.16:0/863140289 conn(0x555eab257f60 msgr2=0x555eab25ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 shutdown_connections 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 1 -- 172.21.15.16:0/863140289 wait complete. 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.316+0000 7f9d3bd19740 20 client.14715 trim_cache size 0 max 16384 2024-09-15T22:36:28.323 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 19.065398761000001, 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 17, 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 85, 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 84, 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 29.916563023999998 2024-09-15T22:36:28.324 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:36:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9d1c0a32c0 msgr2=0x7f9d1c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9d1c0a32c0 0x7f9d1c0a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c013a20 tx=0x7f9d2c005e90 comp rx=0 tx=0).stop 2024-09-15T22:36:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34102790 msgr2=0x7f9d34198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34102790 0x7f9d34198830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c920 tx=0x7f9d2800cde0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 shutdown_connections 2024-09-15T22:36:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9d1c0a32c0 0x7f9d1c0a5780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9d34106f10 0x7f9d3419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d34102790 0x7f9d34198830 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 --2- 172.21.15.16:0/3532391373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d34101de0 0x7f9d341982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 >> 172.21.15.16:0/3532391373 conn(0x7f9d340fd8c0 msgr2=0x7f9d340ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 shutdown_connections 2024-09-15T22:36:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:28.317+0000 7f9d39940640 1 -- 172.21.15.16:0/3532391373 wait complete. 2024-09-15T22:36:28.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:28 smithi016 ceph-mon[31492]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:28.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:28 smithi059 ceph-mon[34872]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:28.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:28 smithi130 ceph-mon[35742]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:29.038 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T22:36:29.038 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:29.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-15T22:36:29.297 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 -- 172.21.15.16:0/1184538803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc101420 msgr2=0x7efddc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1184538803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc101420 0x7efddc10d950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009cd0 tx=0x7efdd402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 -- 172.21.15.16:0/1184538803 shutdown_connections 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1184538803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc101420 0x7efddc10d950 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1184538803 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efddc100a80 0x7efddc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1184538803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efddc106a80 0x7efddc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 -- 172.21.15.16:0/1184538803 >> 172.21.15.16:0/1184538803 conn(0x7efddc0fc7a0 msgr2=0x7efddc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 -- 172.21.15.16:0/1184538803 shutdown_connections 2024-09-15T22:36:29.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.767+0000 7efde2cb7640 1 -- 172.21.15.16:0/1184538803 wait complete. 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 Processor -- start 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 -- start start 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc100a80 0x7efddc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efddc101420 0x7efddc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efddc106a80 0x7efddc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efddc110e80 con 0x7efddc100a80 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efddc110d00 con 0x7efddc101420 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde2cb7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efddc111000 con 0x7efddc106a80 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc100a80 0x7efddc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc100a80 0x7efddc19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51022/0 (socket says 172.21.15.16:51022) 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 -- 172.21.15.16:0/1297287876 learned_addr learned my addr 172.21.15.16:0/1297287876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde1254640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efddc106a80 0x7efddc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efddbfff640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efddc101420 0x7efddc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 -- 172.21.15.16:0/1297287876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efddc106a80 msgr2=0x7efddc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efddc106a80 0x7efddc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 -- 172.21.15.16:0/1297287876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efddc101420 msgr2=0x7efddc19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efddc101420 0x7efddc19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.768+0000 7efde0a53640 1 -- 172.21.15.16:0/1297287876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc19f610 con 0x7efddc100a80 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efde1254640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efddc106a80 0x7efddc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efddbfff640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efddc101420 0x7efddc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efde0a53640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc100a80 0x7efddc19a660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c890 tx=0x7efdcc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efdd9ffb640 1 -- 172.21.15.16:0/1297287876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc004220 con 0x7efddc100a80 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efdd9ffb640 1 -- 172.21.15.16:0/1297287876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc0043c0 con 0x7efddc100a80 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efdd9ffb640 1 -- 172.21.15.16:0/1297287876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0049e0 con 0x7efddc100a80 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc19f900 con 0x7efddc100a80 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.769+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc1a71e0 con 0x7efddc100a80 2024-09-15T22:36:29.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.770+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efddc100ee0 con 0x7efddc100a80 2024-09-15T22:36:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.771+0000 7efdd9ffb640 1 -- 172.21.15.16:0/1297287876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efdcc0079b0 con 0x7efddc100a80 2024-09-15T22:36:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.772+0000 7efdd9ffb640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efdc80a32c0 0x7efdc80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.772+0000 7efddbfff640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efdc80a32c0 0x7efdc80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.772+0000 7efdd9ffb640 1 -- 172.21.15.16:0/1297287876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7efdcc0cfc00 con 0x7efddc100a80 2024-09-15T22:36:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.773+0000 7efddbfff640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efdc80a32c0 0x7efdc80a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efdc4005fd0 tx=0x7efdc4005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.777+0000 7efdd9ffb640 1 -- 172.21.15.16:0/1297287876 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdcc014030 con 0x7efddc100a80 2024-09-15T22:36:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.989+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efddc1a7510 con 0x7efddc100a80 2024-09-15T22:36:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.989+0000 7efdd9ffb640 1 -- 172.21.15.16:0/1297287876 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7efdcc09c680 con 0x7efddc100a80 2024-09-15T22:36:29.991 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T22:36:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.991+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efdc80a32c0 msgr2=0x7efdc80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.991+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efdc80a32c0 0x7efdc80a5780 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7efdc4005fd0 tx=0x7efdc4005ea0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.991+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc100a80 msgr2=0x7efddc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.991+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc100a80 0x7efddc19a660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efdcc00c890 tx=0x7efdcc00cd50 comp rx=0 tx=0).stop 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.991+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 shutdown_connections 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.991+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efdc80a32c0 0x7efdc80a5780 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.992+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efddc106a80 0x7efddc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.992+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efddc101420 0x7efddc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.992+0000 7efde2cb7640 1 --2- 172.21.15.16:0/1297287876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efddc100a80 0x7efddc19a660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.992+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 >> 172.21.15.16:0/1297287876 conn(0x7efddc0fc7a0 msgr2=0x7efddc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.992+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 shutdown_connections 2024-09-15T22:36:29.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:29.992+0000 7efde2cb7640 1 -- 172.21.15.16:0/1297287876 wait complete. 2024-09-15T22:36:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:30 smithi059 ceph-mon[34872]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:30 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1297287876' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:36:30.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:30 smithi130 ceph-mon[35742]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:30.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:30 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1297287876' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:36:30.784 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:30 smithi016 ceph-mon[31492]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:30.784 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:30 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1297287876' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T22:36:30.815 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T22:36:30.816 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T22:36:30.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:30.816 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-15T22:36:30.833 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T22:36:30.833 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T22:36:30.833 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T22:36:30.833 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T22:36:30.834 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-09-15T22:36:30.835 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-15T22:36:30.835 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:36:30.835 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:36:30.835 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:36:30.835 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-15T22:36:30.835 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T22:36:30.836 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:36:30.836 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:36:30.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-15T22:36:30.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-15T22:36:30.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-15T22:36:30.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T22:36:30.836 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:36:30.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:36:31.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:36:31.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:31.059 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T22:36:31.084 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-15T22:36:31.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:31.086 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-15T22:36:31.156 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-15T22:36:31.156 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T22:36:31.156 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T22:36:31.157 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T22:36:31.157 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T22:36:31.157 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:36:31.157 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:36:31.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T22:36:31.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T22:36:31.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T22:36:31.157 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:36:31.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:36:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:36:31.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:31.594 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-15T22:36:31.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:31.611 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-15T22:36:31.667 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:36:31.667 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:36:31.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:36:31.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-15T22:36:31.668 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:36:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:36:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:36:31.788 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-15T22:36:31.788 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:36:31.788 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:36:31.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-15T22:36:31.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T22:36:31.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-15T22:36:31.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-15T22:36:31.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-15T22:36:31.789 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:36:31.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:36:31.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:36:31.919 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:36:31.919 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:36:31.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-15T22:36:31.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T22:36:31.919 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:36:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:36:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:36:31.998 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-15T22:36:31.998 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:36:31.998 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:32.028 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T22:36:32.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:36:32.028 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:32.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-15T22:36:32.151 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T22:36:32.208 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-15T22:36:32.208 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-15T22:36:32.208 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-15T22:36:32.208 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-15T22:36:32.208 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-15T22:36:32.209 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/03f26c07a4769ceafb18b234e14ab504daa659dfc00638284bea13cb03b88efc/merged 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/d281fcb9635e478489e425f2e4b30b3f5e1139f73dd5acb09eba3cc17edf6aec/merged 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/c2363512fbccd61b7f3f14c14aa5328b1f4b523398f9d407bd1115287befbd51/merged 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/29bb0ba3894111d305c7ef09764b70ae1db0118fc060d7f864f1e26751571bab/merged 2024-09-15T22:36:32.210 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/76b482d6a98f415487649f17cc34e90fd8bcab3ea42bf7bc30124448856cdbcb/merged 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/25c2bec205c500b2b56a0830f1de1019de5689ed065b5ae98ac2f5628f54c98f/merged 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/2e0a436ba067f2b8a5d6956d15e5736e17f7822b75bcd6006ac32d41753ac840/merged 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/cdb58e9f71dfff48632f360bb5e4a9a6ea2e345e9cd744b6f45f900379d9f8f2/merged 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e0350b20fa92a87f7c96d7f23ae3f2aaaf78385d30db21fa433e9255b3d07bc9/merged 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/d0347694e7e9b9d99f00cc230f14bb636e75b55f8e2519fa93780c4912d3206c/merged 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:36:32.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:32.212 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-15T22:36:32.266 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-15T22:36:32.266 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-15T22:36:32.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2024-09-15T22:36:32.334 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2024-09-15T22:36:32.393 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T22:36:32.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/03f26c07a4769ceafb18b234e14ab504daa659dfc00638284bea13cb03b88efc/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/d281fcb9635e478489e425f2e4b30b3f5e1139f73dd5acb09eba3cc17edf6aec/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/0 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/c2363512fbccd61b7f3f14c14aa5328b1f4b523398f9d407bd1115287befbd51/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/29bb0ba3894111d305c7ef09764b70ae1db0118fc060d7f864f1e26751571bab/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/76b482d6a98f415487649f17cc34e90fd8bcab3ea42bf7bc30124448856cdbcb/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/25c2bec205c500b2b56a0830f1de1019de5689ed065b5ae98ac2f5628f54c98f/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/2e0a436ba067f2b8a5d6956d15e5736e17f7822b75bcd6006ac32d41753ac840/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/cdb58e9f71dfff48632f360bb5e4a9a6ea2e345e9cd744b6f45f900379d9f8f2/merged 2024-09-15T22:36:32.395 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/e0350b20fa92a87f7c96d7f23ae3f2aaaf78385d30db21fa433e9255b3d07bc9/merged 2024-09-15T22:36:32.396 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/containers/storage/overlay/d0347694e7e9b9d99f00cc230f14bb636e75b55f8e2519fa93780c4912d3206c/merged 2024-09-15T22:36:32.396 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2024-09-15T22:36:32.396 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:36:32.396 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:36:32.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:32.396 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2024-09-15T22:36:32.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2024-09-15T22:36:32.432+0000 7fc24d71b480 -1 init, newargv = 0x55679e253b20 newargc=15 2024-09-15T22:36:32.434 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[75244]: starting ceph client 2024-09-15T22:36:32.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[75244]: starting fuse 2024-09-15T22:36:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:120 2024-09-15T22:36:32.450 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-15T22:36:32.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2024-09-15T22:36:32.450 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> import re 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> import os 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2024-09-15T22:36:32.451 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> return f 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2024-09-15T22:36:32.452 DEBUG:teuthology.orchestra.run.smithi016:> ' 2024-09-15T22:36:32.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:32 smithi016 ceph-mon[31492]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:32.571 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.75244.asok 2024-09-15T22:36:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:36:32.582 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75244.asok 2024-09-15T22:36:32.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:32.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75244.asok status 2024-09-15T22:36:32.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:32 smithi059 ceph-mon[34872]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:32.720 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:36:32.720 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2024-09-15T22:36:32.720 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "75244", 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 14727, 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2024-09-15T22:36:32.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 14727 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 382641996 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 382641996 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:36:32.722 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.14727 192.168.0.1:0/382641996", 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/382641996", 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 17, 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 85, 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2024-09-15T22:36:32.723 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:36:32.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:32.729 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2024-09-15T22:36:32.729 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:32.745 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2024-09-15T22:36:32.745 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:32.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:32.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:32.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:32 smithi130 ceph-mon[35742]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:32.812 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-15T22:36:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:36:32.926 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2024-09-15T22:36:32.927 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-15T22:36:32.927 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:36:32.933 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:36:32.933 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:36:32.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:36:33.247 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 -- 172.21.15.16:0/3134272633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 msgr2=0x7fc12410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 --2- 172.21.15.16:0/3134272633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12410e610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009ca0 tx=0x7fc11c02a950 comp rx=0 tx=0).stop 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 -- 172.21.15.16:0/3134272633 shutdown_connections 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 --2- 172.21.15.16:0/3134272633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12410e610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 --2- 172.21.15.16:0/3134272633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc124103660 0x7fc12410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 --2- 172.21.15.16:0/3134272633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc124102d40 0x7fc124103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 -- 172.21.15.16:0/3134272633 >> 172.21.15.16:0/3134272633 conn(0x7fc1240fc7a0 msgr2=0x7fc1240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 -- 172.21.15.16:0/3134272633 shutdown_connections 2024-09-15T22:36:33.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.707+0000 7fc129c3e640 1 -- 172.21.15.16:0/3134272633 wait complete. 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 Processor -- start 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 -- start start 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc124102d40 0x7fc124198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc124103660 0x7fc124198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc124111f80 con 0x7fc12410c170 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc124111e00 con 0x7fc124103660 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc129c3e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc124112100 con 0x7fc124102d40 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc123fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc123fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37370/0 (socket says 172.21.15.16:37370) 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc123fff640 1 -- 172.21.15.16:0/844780537 learned_addr learned my addr 172.21.15.16:0/844780537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc1237fe640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc124102d40 0x7fc124198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:33.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.708+0000 7fc122ffd640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc124103660 0x7fc124198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc123fff640 1 -- 172.21.15.16:0/844780537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc124102d40 msgr2=0x7fc124198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc123fff640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc124102d40 0x7fc124198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc123fff640 1 -- 172.21.15.16:0/844780537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc124103660 msgr2=0x7fc124198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc123fff640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc124103660 0x7fc124198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc123fff640 1 -- 172.21.15.16:0/844780537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12419d2f0 con 0x7fc12410c170 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc123fff640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12419cc10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009dc0 tx=0x7fc11c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc120ff9640 1 -- 172.21.15.16:0/844780537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c034040 con 0x7fc12410c170 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc120ff9640 1 -- 172.21.15.16:0/844780537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc11c02fe20 con 0x7fc12410c170 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12419d580 con 0x7fc12410c170 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc120ff9640 1 -- 172.21.15.16:0/844780537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c0047f0 con 0x7fc12410c170 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.709+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1241a4ec0 con 0x7fc12410c170 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.710+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124103120 con 0x7fc12410c170 2024-09-15T22:36:33.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.712+0000 7fc120ff9640 1 -- 172.21.15.16:0/844780537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc11c037020 con 0x7fc12410c170 2024-09-15T22:36:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.713+0000 7fc120ff9640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc10c0a32c0 0x7fc10c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.713+0000 7fc1237fe640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc10c0a32c0 0x7fc10c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.713+0000 7fc120ff9640 1 -- 172.21.15.16:0/844780537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fc11c0f0c00 con 0x7fc12410c170 2024-09-15T22:36:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.713+0000 7fc1237fe640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc10c0a32c0 0x7fc10c0a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc114005d00 tx=0x7fc11400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:33.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.716+0000 7fc120ff9640 1 -- 172.21.15.16:0/844780537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc11c0811b0 con 0x7fc12410c170 2024-09-15T22:36:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.902+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc1241a5090 con 0x7fc12410c170 2024-09-15T22:36:33.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.902+0000 7fc120ff9640 1 -- 172.21.15.16:0/844780537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+592 (secure 0 0 0) 0x7fc11c0bd680 con 0x7fc12410c170 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1740394357 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3905659611 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3985544937 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3381483588 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2652317700 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3860212428 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2652317700 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1563124617 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3949807891 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:33.905 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3949807891 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-15T22:36:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc10c0a32c0 msgr2=0x7fc10c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc10c0a32c0 0x7fc10c0a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc114005d00 tx=0x7fc11400b410 comp rx=0 tx=0).stop 2024-09-15T22:36:33.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 msgr2=0x7fc12419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12419cc10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc11c009dc0 tx=0x7fc11c003f10 comp rx=0 tx=0).stop 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 shutdown_connections 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc10c0a32c0 0x7fc10c0a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12410c170 0x7fc12419cc10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc124103660 0x7fc124198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 --2- 172.21.15.16:0/844780537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc124102d40 0x7fc124198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 >> 172.21.15.16:0/844780537 conn(0x7fc1240fc7a0 msgr2=0x7fc1240fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 shutdown_connections 2024-09-15T22:36:33.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:33.904+0000 7fc129c3e640 1 -- 172.21.15.16:0/844780537 wait complete. 2024-09-15T22:36:34.585 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:36:34.585 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:36:34.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:36:34.668 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:34 smithi016 ceph-mon[31492]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:34.668 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:34 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/844780537' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:36:34.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:34 smithi059 ceph-mon[34872]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:34.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:34 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/844780537' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:36:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:34 smithi130 ceph-mon[35742]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:36:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:34 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/844780537' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.404+0000 7f6c48bd9640 1 -- 172.21.15.16:0/1737072706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44106f10 msgr2=0x7f6c4410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.404+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/1737072706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44106f10 0x7f6c4410ea90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009ca0 tx=0x7f6c3c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 -- 172.21.15.16:0/1737072706 shutdown_connections 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/1737072706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44106f10 0x7f6c4410ea90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/1737072706 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44102790 0x7f6c441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/1737072706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c44101de0 0x7f6c441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 -- 172.21.15.16:0/1737072706 >> 172.21.15.16:0/1737072706 conn(0x7f6c440fd8c0 msgr2=0x7f6c440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 -- 172.21.15.16:0/1737072706 shutdown_connections 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 -- 172.21.15.16:0/1737072706 wait complete. 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 Processor -- start 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.405+0000 7f6c48bd9640 1 -- start start 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c48bd9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c44101de0 0x7f6c44198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c48bd9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44102790 0x7f6c44198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c48bd9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44106f10 0x7f6c4419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c48bd9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111f70 con 0x7f6c44102790 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c48bd9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44111df0 con 0x7f6c44101de0 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c48bd9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441120f0 con 0x7f6c44106f10 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44102790 0x7f6c44198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44102790 0x7f6c44198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37388/0 (socket says 172.21.15.16:37388) 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 -- 172.21.15.16:0/3063411761 learned_addr learned my addr 172.21.15.16:0/3063411761 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c42d9d640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44106f10 0x7f6c4419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c4259c640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c44101de0 0x7f6c44198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 -- 172.21.15.16:0/3063411761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44106f10 msgr2=0x7f6c4419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44106f10 0x7f6c4419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 -- 172.21.15.16:0/3063411761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c44101de0 msgr2=0x7f6c44198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c44101de0 0x7f6c44198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 -- 172.21.15.16:0/3063411761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4419d2e0 con 0x7f6c44102790 2024-09-15T22:36:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c42d9d640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44106f10 0x7f6c4419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c41d9b640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44102790 0x7f6c44198870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c920 tx=0x7f6c3800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c337fe640 1 -- 172.21.15.16:0/3063411761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38007c70 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4419d5d0 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c337fe640 1 -- 172.21.15.16:0/3063411761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c38007e10 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.406+0000 7f6c337fe640 1 -- 172.21.15.16:0/3063411761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38018890 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.407+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a4eb0 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.408+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c441021c0 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.409+0000 7f6c337fe640 1 -- 172.21.15.16:0/3063411761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6c38018a50 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.410+0000 7f6c337fe640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6c2c0a3240 0x7f6c2c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.410+0000 7f6c4259c640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6c2c0a3240 0x7f6c2c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.410+0000 7f6c337fe640 1 -- 172.21.15.16:0/3063411761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f6c380cff70 con 0x7f6c44102790 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.410+0000 7f6c4259c640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6c2c0a3240 0x7f6c2c0a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c340058c0 tx=0x7f6c3400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:35.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.413+0000 7f6c337fe640 1 -- 172.21.15.16:0/3063411761 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c38014030 con 0x7f6c44102790 2024-09-15T22:36:35.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.600+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6c441a5170 con 0x7f6c44102790 2024-09-15T22:36:35.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.601+0000 7f6c337fe640 1 -- 172.21.15.16:0/3063411761 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+592 (secure 0 0 0) 0x7f6c3800de20 con 0x7f6c44102790 2024-09-15T22:36:35.603 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1740394357 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:35.603 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3905659611 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:35.603 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3985544937 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:35.603 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3381483588 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:35.604 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2652317700 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:35.604 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3860212428 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:35.604 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2652317700 2024-09-16T22:29:10.483605+0000 2024-09-15T22:36:35.604 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1563124617 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:35.604 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3949807891 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:35.604 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3949807891 2024-09-16T22:28:46.507397+0000 2024-09-15T22:36:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:listed 10 entries 2024-09-15T22:36:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6c2c0a3240 msgr2=0x7f6c2c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6c2c0a3240 0x7f6c2c0a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6c340058c0 tx=0x7f6c3400a4c0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44102790 msgr2=0x7f6c44198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44102790 0x7f6c44198870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c920 tx=0x7f6c3800cde0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 shutdown_connections 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6c2c0a3240 0x7f6c2c0a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6c44106f10 0x7f6c4419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c44102790 0x7f6c44198870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 --2- 172.21.15.16:0/3063411761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6c44101de0 0x7f6c44198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 >> 172.21.15.16:0/3063411761 conn(0x7f6c440fd8c0 msgr2=0x7f6c440ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 shutdown_connections 2024-09-15T22:36:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:35.603+0000 7f6c48bd9640 1 -- 172.21.15.16:0/3063411761 wait complete. 2024-09-15T22:36:36.332 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2024-09-15T22:36:36.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:36.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:36:36.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[75244]: fuse finished with error 0 and tester_r 0 2024-09-15T22:36:36.404 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T22:36:36.423 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:36:36.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:36:36.423 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-15T22:36:36.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:36:36.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:36 smithi016 ceph-mon[31492]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:36:36.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:36 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3063411761' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:36:36.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:36 smithi059 ceph-mon[34872]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:36:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:36 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3063411761' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:36:36.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:36 smithi130 ceph-mon[35742]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:36:36.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:36 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3063411761' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:36:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:37.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 -- 172.21.15.16:0/1288430704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 msgr2=0x7f8f6c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 --2- 172.21.15.16:0/1288430704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c100560 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009900 tx=0x7f8f5402a410 comp rx=0 tx=0).stop 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 -- 172.21.15.16:0/1288430704 shutdown_connections 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 --2- 172.21.15.16:0/1288430704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6c100aa0 0x7f8f6c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 --2- 172.21.15.16:0/1288430704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c100560 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 --2- 172.21.15.16:0/1288430704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f6c106b80 0x7f8f6c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 -- 172.21.15.16:0/1288430704 >> 172.21.15.16:0/1288430704 conn(0x7f8f6c077230 msgr2=0x7f8f6c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 -- 172.21.15.16:0/1288430704 shutdown_connections 2024-09-15T22:36:37.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.377+0000 7f8f70dde640 1 -- 172.21.15.16:0/1288430704 wait complete. 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 Processor -- start 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 -- start start 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f6c100aa0 0x7f8f6c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c112010 con 0x7f8f6c100100 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c111e90 con 0x7f8f6c106b80 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f70dde640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c112190 con 0x7f8f6c100aa0 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37400/0 (socket says 172.21.15.16:37400) 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 -- 172.21.15.16:0/2228498906 learned_addr learned my addr 172.21.15.16:0/2228498906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f63fff640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f6c100aa0 0x7f8f6c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6ad9d640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 -- 172.21.15.16:0/2228498906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f6c100aa0 msgr2=0x7f8f6c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f6c100aa0 0x7f8f6c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 -- 172.21.15.16:0/2228498906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6c106b80 msgr2=0x7f8f6c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6a59c640 1 -- 172.21.15.16:0/2228498906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c19d3d0 con 0x7f8f6c100100 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.378+0000 7f8f6ad9d640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.379+0000 7f8f6a59c640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c198420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00c920 tx=0x7f8f5c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.379+0000 7f8f637fe640 1 -- 172.21.15.16:0/2228498906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c007c70 con 0x7f8f6c100100 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.379+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c19d6c0 con 0x7f8f6c100100 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.379+0000 7f8f637fe640 1 -- 172.21.15.16:0/2228498906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f5c007e10 con 0x7f8f6c100100 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.379+0000 7f8f637fe640 1 -- 172.21.15.16:0/2228498906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c018890 con 0x7f8f6c100100 2024-09-15T22:36:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.379+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1a4fa0 con 0x7f8f6c100100 2024-09-15T22:36:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.380+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c100560 con 0x7f8f6c100100 2024-09-15T22:36:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.381+0000 7f8f637fe640 1 -- 172.21.15.16:0/2228498906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8f5c018a50 con 0x7f8f6c100100 2024-09-15T22:36:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.382+0000 7f8f637fe640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f580a3240 0x7f8f580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.382+0000 7f8f637fe640 1 -- 172.21.15.16:0/2228498906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f8f5c0cff30 con 0x7f8f6c100100 2024-09-15T22:36:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.382+0000 7f8f63fff640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f580a3240 0x7f8f580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.382+0000 7f8f63fff640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f580a3240 0x7f8f580a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009230 tx=0x7f8f540058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:37.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.385+0000 7f8f637fe640 1 -- 172.21.15.16:0/2228498906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f5c014030 con 0x7f8f6c100100 2024-09-15T22:36:37.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.565+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f6c199780 con 0x7f8f6c100100 2024-09-15T22:36:37.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.566+0000 7f8f637fe640 1 -- 172.21.15.16:0/2228498906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7f8f5c09caa0 con 0x7f8f6c100100 2024-09-15T22:36:37.569 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:37.569 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f580a3240 msgr2=0x7f8f580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f580a3240 0x7f8f580a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8f54009230 tx=0x7f8f540058d0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 msgr2=0x7f8f6c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c198420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00c920 tx=0x7f8f5c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 shutdown_connections 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8f580a3240 0x7f8f580a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f6c106b80 0x7f8f6c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f6c100aa0 0x7f8f6c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 --2- 172.21.15.16:0/2228498906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f6c100100 0x7f8f6c198420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.568+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 >> 172.21.15.16:0/2228498906 conn(0x7f8f6c077230 msgr2=0x7f8f6c104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.569+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 shutdown_connections 2024-09-15T22:36:37.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:37.569+0000 7f8f70dde640 1 -- 172.21.15.16:0/2228498906 wait complete. 2024-09-15T22:36:38.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:36:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:38.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:38 smithi059 ceph-mon[34872]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:36:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:38 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2228498906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:38.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:38 smithi016 ceph-mon[31492]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:36:38.764 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:38 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2228498906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:38.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:38 smithi130 ceph-mon[35742]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-15T22:36:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:38 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2228498906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 -- 172.21.15.16:0/3558281661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258102d40 msgr2=0x7f9258103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 --2- 172.21.15.16:0/3558281661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258102d40 0x7f9258103120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9248009900 tx=0x7f924802a480 comp rx=0 tx=0).stop 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 -- 172.21.15.16:0/3558281661 shutdown_connections 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 --2- 172.21.15.16:0/3558281661 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f925810c170 0x7f925810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 --2- 172.21.15.16:0/3558281661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9258103660 0x7f925810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 --2- 172.21.15.16:0/3558281661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258102d40 0x7f9258103120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 -- 172.21.15.16:0/3558281661 >> 172.21.15.16:0/3558281661 conn(0x7f92580fc7a0 msgr2=0x7f92580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 -- 172.21.15.16:0/3558281661 shutdown_connections 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 -- 172.21.15.16:0/3558281661 wait complete. 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 Processor -- start 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.165+0000 7f925e33b640 1 -- start start 2024-09-15T22:36:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f925e33b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9258102d40 0x7f92581009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f925e33b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258103660 0x7f9258100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f925e33b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f925810c170 0x7f9258101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f925e33b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9258111f70 con 0x7f9258103660 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f925e33b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9258111df0 con 0x7f925810c170 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f925e33b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f92581120f0 con 0x7f9258102d40 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258103660 0x7f9258100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258103660 0x7f9258100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37430/0 (socket says 172.21.15.16:37430) 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 -- 172.21.15.16:0/3446529214 learned_addr learned my addr 172.21.15.16:0/3446529214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f9257fff640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9258102d40 0x7f92581009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f925c8d8640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f925810c170 0x7f9258101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 -- 172.21.15.16:0/3446529214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9258102d40 msgr2=0x7f92581009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9258102d40 0x7f92581009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 -- 172.21.15.16:0/3446529214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f925810c170 msgr2=0x7f9258101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f925810c170 0x7f9258101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.166+0000 7f92577fe640 1 -- 172.21.15.16:0/3446529214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92581a68e0 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.167+0000 7f92577fe640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258103660 0x7f9258100ee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9f0 tx=0x7f924c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.167+0000 7f92557fa640 1 -- 172.21.15.16:0/3446529214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c00ccc0 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.167+0000 7f92557fa640 1 -- 172.21.15.16:0/3446529214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f924c010910 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.167+0000 7f92557fa640 1 -- 172.21.15.16:0/3446529214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c018920 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.167+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92581a6bd0 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.167+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92581a7160 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.168+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9258103120 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.169+0000 7f92557fa640 1 -- 172.21.15.16:0/3446529214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f924c010430 con 0x7f9258103660 2024-09-15T22:36:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.170+0000 7f92557fa640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f92400a32c0 0x7f92400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.170+0000 7f9257fff640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f92400a32c0 0x7f92400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.170+0000 7f92557fa640 1 -- 172.21.15.16:0/3446529214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f924c0cfcf0 con 0x7f9258103660 2024-09-15T22:36:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.170+0000 7f9257fff640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f92400a32c0 0x7f92400a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9248009230 tx=0x7f92480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:39.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.174+0000 7f92557fa640 1 -- 172.21.15.16:0/3446529214 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f924c0d4070 con 0x7f9258103660 2024-09-15T22:36:39.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.353+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9258107210 con 0x7f9258103660 2024-09-15T22:36:39.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.355+0000 7f92557fa640 1 -- 172.21.15.16:0/3446529214 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7f924c09c180 con 0x7f9258103660 2024-09-15T22:36:39.357 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:36:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f92400a32c0 msgr2=0x7f92400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f92400a32c0 0x7f92400a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9248009230 tx=0x7f92480058d0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258103660 msgr2=0x7f9258100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258103660 0x7f9258100ee0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9f0 tx=0x7f924c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 shutdown_connections 2024-09-15T22:36:39.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f92400a32c0 0x7f92400a5780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f925810c170 0x7f9258101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9258103660 0x7f9258100ee0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.357+0000 7f925e33b640 1 --2- 172.21.15.16:0/3446529214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9258102d40 0x7f92581009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.358+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 >> 172.21.15.16:0/3446529214 conn(0x7f92580fc7a0 msgr2=0x7f92580fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.358+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 shutdown_connections 2024-09-15T22:36:39.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:39.358+0000 7f925e33b640 1 -- 172.21.15.16:0/3446529214 wait complete. 2024-09-15T22:36:40.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-15T22:36:40.415 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:40.537 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:40 smithi016 ceph-mon[31492]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:40.537 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:40 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3446529214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:40.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:40 smithi059 ceph-mon[34872]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:40.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:40 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3446529214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:40.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:40 smithi130 ceph-mon[35742]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:40.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:40 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3446529214' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 -- 172.21.15.16:0/4115399652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420106f10 msgr2=0x7f942010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 --2- 172.21.15.16:0/4115399652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420106f10 0x7f942010ea90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9418009ca0 tx=0x7f941802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 -- 172.21.15.16:0/4115399652 shutdown_connections 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 --2- 172.21.15.16:0/4115399652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420106f10 0x7f942010ea90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 --2- 172.21.15.16:0/4115399652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9420102790 0x7f94201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 --2- 172.21.15.16:0/4115399652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9420101de0 0x7f94201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 -- 172.21.15.16:0/4115399652 >> 172.21.15.16:0/4115399652 conn(0x7f94200fd8c0 msgr2=0x7f94200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:40.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 -- 172.21.15.16:0/4115399652 shutdown_connections 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.876+0000 7f94256c8640 1 -- 172.21.15.16:0/4115399652 wait complete. 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 Processor -- start 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 -- start start 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9420101de0 0x7f9420198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420102790 0x7f94201989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9420106f10 0x7f942019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9420111f90 con 0x7f9420102790 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9420111e10 con 0x7f9420106f10 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f94256c8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9420112110 con 0x7f9420101de0 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.877+0000 7f941e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420102790 0x7f94201989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420102790 0x7f94201989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37458/0 (socket says 172.21.15.16:37458) 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 -- 172.21.15.16:0/4221834839 learned_addr learned my addr 172.21.15.16:0/4221834839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941f7fe640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9420106f10 0x7f942019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941effd640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9420101de0 0x7f9420198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 -- 172.21.15.16:0/4221834839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9420101de0 msgr2=0x7f9420198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9420101de0 0x7f9420198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 -- 172.21.15.16:0/4221834839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9420106f10 msgr2=0x7f942019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9420106f10 0x7f942019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 -- 172.21.15.16:0/4221834839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942019d420 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f941e7fc640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420102790 0x7f94201989b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f941400b4a0 tx=0x7f941400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f9407fff640 1 -- 172.21.15.16:0/4221834839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414014070 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f9407fff640 1 -- 172.21.15.16:0/4221834839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9414010af0 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942019d710 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f9407fff640 1 -- 172.21.15.16:0/4221834839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414017d10 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.878+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94201a4ff0 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.879+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94201021c0 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.880+0000 7f9407fff640 1 -- 172.21.15.16:0/4221834839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9414017070 con 0x7f9420102790 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.881+0000 7f9407fff640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94080a32c0 0x7f94080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.881+0000 7f941effd640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94080a32c0 0x7f94080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.881+0000 7f9407fff640 1 -- 172.21.15.16:0/4221834839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f94140cee00 con 0x7f9420102790 2024-09-15T22:36:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.882+0000 7f941effd640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94080a32c0 0x7f94080a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f941000bbe0 tx=0x7f941000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:40.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:40.886+0000 7f9407fff640 1 -- 172.21.15.16:0/4221834839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f941409b290 con 0x7f9420102790 2024-09-15T22:36:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.100+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f942002d7c0 con 0x7f9420102790 2024-09-15T22:36:41.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.353+0000 7f9407fff640 1 -- 172.21.15.16:0/4221834839 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f94140b9e20 con 0x7f9420102790 2024-09-15T22:36:41.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:41 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4221834839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:36:41.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94080a32c0 msgr2=0x7f94080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:41.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94080a32c0 0x7f94080a5780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f941000bbe0 tx=0x7f941000b390 comp rx=0 tx=0).stop 2024-09-15T22:36:41.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420102790 msgr2=0x7f94201989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:41.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420102790 0x7f94201989b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f941400b4a0 tx=0x7f941400b960 comp rx=0 tx=0).stop 2024-09-15T22:36:41.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 shutdown_connections 2024-09-15T22:36:41.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f94080a32c0 0x7f94080a5780 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9420106f10 0x7f942019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9420102790 0x7f94201989b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 --2- 172.21.15.16:0/4221834839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9420101de0 0x7f9420198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 >> 172.21.15.16:0/4221834839 conn(0x7f94200fd8c0 msgr2=0x7f94200fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:41.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 shutdown_connections 2024-09-15T22:36:41.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:41.356+0000 7f94256c8640 1 -- 172.21.15.16:0/4221834839 wait complete. 2024-09-15T22:36:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:41 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4221834839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:36:41.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:41 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4221834839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:36:42.227 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-15T22:36:42.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-15T22:36:42.489 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:42.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:42 smithi016 ceph-mon[31492]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:42 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4221834839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:36:42.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:42 smithi016 ceph-mon[31492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:42.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:42 smithi059 ceph-mon[34872]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:42 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4221834839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:36:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:42 smithi059 ceph-mon[34872]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:42 smithi130 ceph-mon[35742]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:42 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4221834839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:36:42.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:42 smithi130 ceph-mon[35742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 -- 172.21.15.16:0/786265919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 msgr2=0x7f75cc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 --2- 172.21.15.16:0/786265919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc106e60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009930 tx=0x7f75bc02a420 comp rx=0 tx=0).stop 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 -- 172.21.15.16:0/786265919 shutdown_connections 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 --2- 172.21.15.16:0/786265919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75cc101420 0x7f75cc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 --2- 172.21.15.16:0/786265919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc100a80 0x7f75cc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 --2- 172.21.15.16:0/786265919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc106e60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 -- 172.21.15.16:0/786265919 >> 172.21.15.16:0/786265919 conn(0x7f75cc0fc7a0 msgr2=0x7f75cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 -- 172.21.15.16:0/786265919 shutdown_connections 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.974+0000 7f75d1f87640 1 -- 172.21.15.16:0/786265919 wait complete. 2024-09-15T22:36:42.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 Processor -- start 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 -- start start 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc100a80 0x7f75cc1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75cc101420 0x7f75cc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc1a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc110e80 con 0x7f75cc106a80 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc110d00 con 0x7f75cc100a80 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75d1f87640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc111000 con 0x7f75cc101420 2024-09-15T22:36:42.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.975+0000 7f75cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc1a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cb7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc100a80 0x7f75cc1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cb7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc100a80 0x7f75cc1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:43708/0 (socket says 172.21.15.16:43708) 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cb7fe640 1 -- 172.21.15.16:0/1123857575 learned_addr learned my addr 172.21.15.16:0/1123857575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc1a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60838/0 (socket says 172.21.15.16:60838) 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75caffd640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75cc101420 0x7f75cc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cbfff640 1 -- 172.21.15.16:0/1123857575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75cc101420 msgr2=0x7f75cc199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cbfff640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75cc101420 0x7f75cc199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cbfff640 1 -- 172.21.15.16:0/1123857575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc100a80 msgr2=0x7f75cc1992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cbfff640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc100a80 0x7f75cc1992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cbfff640 1 -- 172.21.15.16:0/1123857575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc1a4890 con 0x7f75cc106a80 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75cbfff640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc1a4350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009fd0 tx=0x7f75c400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:42.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75c8ff9640 1 -- 172.21.15.16:0/1123857575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c400ce20 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75c8ff9640 1 -- 172.21.15.16:0/1123857575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75c4005020 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc1a4a60 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75c8ff9640 1 -- 172.21.15.16:0/1123857575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4010720 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.976+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc1a4f70 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.977+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc100ee0 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.979+0000 7f75c8ff9640 1 -- 172.21.15.16:0/1123857575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f75c4005540 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.979+0000 7f75c8ff9640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f75b40a3240 0x7f75b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.979+0000 7f75cb7fe640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f75b40a3240 0x7f75b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.979+0000 7f75c8ff9640 1 -- 172.21.15.16:0/1123857575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f75c40d01f0 con 0x7f75cc106a80 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.980+0000 7f75cb7fe640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f75b40a3240 0x7f75b40a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75bc000c00 tx=0x7f75bc009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:42.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:42.984+0000 7f75c8ff9640 1 -- 172.21.15.16:0/1123857575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75c409ccd0 con 0x7f75cc106a80 2024-09-15T22:36:43.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.199+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f75cc108ff0 con 0x7f75cc106a80 2024-09-15T22:36:43.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.362+0000 7f75c8ff9640 1 -- 172.21.15.16:0/1123857575 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f75c409c5d0 con 0x7f75cc106a80 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f75b40a3240 msgr2=0x7f75b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f75b40a3240 0x7f75b40a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f75bc000c00 tx=0x7f75bc009560 comp rx=0 tx=0).stop 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 msgr2=0x7f75cc1a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc1a4350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009fd0 tx=0x7f75c400ef10 comp rx=0 tx=0).stop 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 shutdown_connections 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f75b40a3240 0x7f75b40a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75cc106a80 0x7f75cc1a4350 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f75cc101420 0x7f75cc199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 --2- 172.21.15.16:0/1123857575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc100a80 0x7f75cc1992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.365+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 >> 172.21.15.16:0/1123857575 conn(0x7f75cc0fc7a0 msgr2=0x7f75cc0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.366+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 shutdown_connections 2024-09-15T22:36:43.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:43.366+0000 7f75d1f87640 1 -- 172.21.15.16:0/1123857575 wait complete. 2024-09-15T22:36:43.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:43 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1123857575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:36:43.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:43 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1123857575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:36:43.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:43 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1123857575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:36:44.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-15T22:36:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:44.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:44 smithi016 ceph-mon[31492]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:44.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:44 smithi016 ceph-mon[31492]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:36:44.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:44 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1123857575' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:36:44.625 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:44 smithi016 ceph-mon[31492]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:36:44.625 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:44 smithi016 ceph-mon[31492]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:44.625 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:44 smithi016 ceph-mon[31492]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:36:44.625 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:44 smithi016 ceph-mon[31492]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:36:44.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:44 smithi059 ceph-mon[34872]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:44 smithi059 ceph-mon[34872]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:36:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:44 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1123857575' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:36:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:44 smithi059 ceph-mon[34872]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:36:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:44 smithi059 ceph-mon[34872]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:44 smithi059 ceph-mon[34872]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:36:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:44 smithi059 ceph-mon[34872]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:36:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:44 smithi130 ceph-mon[35742]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:36:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:44 smithi130 ceph-mon[35742]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:36:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:44 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1123857575' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:36:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:44 smithi130 ceph-mon[35742]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:36:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:44 smithi130 ceph-mon[35742]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:36:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:44 smithi130 ceph-mon[35742]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:36:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:44 smithi130 ceph-mon[35742]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:36:44.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 -- 172.21.15.16:0/2742999025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc10c180 msgr2=0x7fe6dc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/2742999025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc10c180 0x7fe6dc10e610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009ca0 tx=0x7fe6d402a950 comp rx=0 tx=0).stop 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 -- 172.21.15.16:0/2742999025 shutdown_connections 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/2742999025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc10c180 0x7fe6dc10e610 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/2742999025 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6dc103700 0x7fe6dc10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/2742999025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6dc102de0 0x7fe6dc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 -- 172.21.15.16:0/2742999025 >> 172.21.15.16:0/2742999025 conn(0x7fe6dc0fc7a0 msgr2=0x7fe6dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 -- 172.21.15.16:0/2742999025 shutdown_connections 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.831+0000 7fe6e17d4640 1 -- 172.21.15.16:0/2742999025 wait complete. 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 Processor -- start 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 -- start start 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc102de0 0x7fe6dc100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6dc103700 0x7fe6dc100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6dc10c180 0x7fe6dc1014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc112030 con 0x7fe6dc102de0 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc111eb0 con 0x7fe6dc103700 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6e17d4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc1121b0 con 0x7fe6dc10c180 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc102de0 0x7fe6dc100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc102de0 0x7fe6dc100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60864/0 (socket says 172.21.15.16:60864) 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6daffd640 1 -- 172.21.15.16:0/1679364718 learned_addr learned my addr 172.21.15.16:0/1679364718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6db7fe640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6dc10c180 0x7fe6dc1014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6d27fc640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6dc103700 0x7fe6dc100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6daffd640 1 -- 172.21.15.16:0/1679364718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6dc10c180 msgr2=0x7fe6dc1014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.832+0000 7fe6daffd640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6dc10c180 0x7fe6dc1014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6daffd640 1 -- 172.21.15.16:0/1679364718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6dc103700 msgr2=0x7fe6dc100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6daffd640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6dc103700 0x7fe6dc100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6daffd640 1 -- 172.21.15.16:0/1679364718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc1a6a00 con 0x7fe6dc102de0 2024-09-15T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6daffd640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc102de0 0x7fe6dc100a40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00b4a0 tx=0x7fe6cc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6d8ff9640 1 -- 172.21.15.16:0/1679364718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc015020 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc1a6cf0 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6d8ff9640 1 -- 172.21.15.16:0/1679364718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6cc011d10 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6d8ff9640 1 -- 172.21.15.16:0/1679364718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc00f2e0 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.833+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1a71d0 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.834+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc103240 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.835+0000 7fe6d8ff9640 1 -- 172.21.15.16:0/1679364718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe6cc018020 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.836+0000 7fe6d8ff9640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6c80a3240 0x7fe6c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.837+0000 7fe6d27fc640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6c80a3240 0x7fe6c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.837+0000 7fe6d8ff9640 1 -- 172.21.15.16:0/1679364718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fe6cc0cf7c0 con 0x7fe6dc102de0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.837+0000 7fe6d27fc640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6c80a3240 0x7fe6c80a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc1024d0 tx=0x7fe6c4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:44.840+0000 7fe6d8ff9640 1 -- 172.21.15.16:0/1679364718 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6cc01f020 con 0x7fe6dc102de0 2024-09-15T22:36:45.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.054+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe6dc101f70 con 0x7fe6dc102de0 2024-09-15T22:36:45.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.378+0000 7fe6d8ff9640 1 -- 172.21.15.16:0/1679364718 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fe6cc09c240 con 0x7fe6dc102de0 2024-09-15T22:36:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6c80a3240 msgr2=0x7fe6c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6c80a3240 0x7fe6c80a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc1024d0 tx=0x7fe6c4009110 comp rx=0 tx=0).stop 2024-09-15T22:36:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc102de0 msgr2=0x7fe6dc100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc102de0 0x7fe6dc100a40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00b4a0 tx=0x7fe6cc00b8a0 comp rx=0 tx=0).stop 2024-09-15T22:36:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 shutdown_connections 2024-09-15T22:36:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6c80a3240 0x7fe6c80a5700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6dc10c180 0x7fe6dc1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6dc103700 0x7fe6dc100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 --2- 172.21.15.16:0/1679364718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6dc102de0 0x7fe6dc100a40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.380+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 >> 172.21.15.16:0/1679364718 conn(0x7fe6dc0fc7a0 msgr2=0x7fe6dc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.381+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 shutdown_connections 2024-09-15T22:36:45.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:45.381+0000 7fe6e17d4640 1 -- 172.21.15.16:0/1679364718 wait complete. 2024-09-15T22:36:45.693 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] up:active 2024-09-15T22:36:45.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:36:45.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:36:45.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: Cluster is now healthy 2024-09-15T22:36:45.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:36:45.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:36:45.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:36:45.694 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:45 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1679364718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] up:active 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: Cluster is now healthy 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:36:45.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:45 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1679364718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:36:45.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] up:active 2024-09-15T22:36:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:36:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:36:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: Cluster is now healthy 2024-09-15T22:36:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:36:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:36:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:36:45.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:45 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1679364718' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:36:46.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-15T22:36:46.398 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:46.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:46 smithi016 ceph-mon[31492]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:36:46.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:46 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1679364718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:36:46.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:46 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] up:active 2024-09-15T22:36:46.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:46 smithi016 ceph-mon[31492]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:36:46.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:46 smithi016 ceph-mon[31492]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:36:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:46 smithi059 ceph-mon[34872]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:36:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:46 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1679364718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:36:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:46 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] up:active 2024-09-15T22:36:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:46 smithi059 ceph-mon[34872]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:36:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:46 smithi059 ceph-mon[34872]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:36:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:46 smithi130 ceph-mon[35742]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:36:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:46 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1679364718' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:36:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:46 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] up:active 2024-09-15T22:36:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:46 smithi130 ceph-mon[35742]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:36:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:46 smithi130 ceph-mon[35742]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:36:46.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 -- 172.21.15.16:0/129183396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 msgr2=0x7f379810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:46.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 --2- 172.21.15.16:0/129183396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379810e610 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3790009ca0 tx=0x7f379002a950 comp rx=0 tx=0).stop 2024-09-15T22:36:46.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 -- 172.21.15.16:0/129183396 shutdown_connections 2024-09-15T22:36:46.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 --2- 172.21.15.16:0/129183396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379810e610 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 --2- 172.21.15.16:0/129183396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3798103660 0x7f379810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 --2- 172.21.15.16:0/129183396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3798102d40 0x7f3798103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 -- 172.21.15.16:0/129183396 >> 172.21.15.16:0/129183396 conn(0x7f37980fc7a0 msgr2=0x7f37980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:46.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 -- 172.21.15.16:0/129183396 shutdown_connections 2024-09-15T22:36:46.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.881+0000 7f379cf07640 1 -- 172.21.15.16:0/129183396 wait complete. 2024-09-15T22:36:46.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.882+0000 7f379cf07640 1 Processor -- start 2024-09-15T22:36:46.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.882+0000 7f379cf07640 1 -- start start 2024-09-15T22:36:46.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.882+0000 7f379cf07640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3798102d40 0x7f37981983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:46.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.882+0000 7f379cf07640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3798103660 0x7f3798198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f379cf07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f379cf07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3798111f70 con 0x7f379810c170 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f379cf07640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3798111df0 con 0x7f3798102d40 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f379cf07640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37981120f0 con 0x7f3798103660 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60882/0 (socket says 172.21.15.16:60882) 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 -- 172.21.15.16:0/2625832913 learned_addr learned my addr 172.21.15.16:0/2625832913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3795d9b640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3798103660 0x7f3798198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f379659c640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3798102d40 0x7f37981983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 -- 172.21.15.16:0/2625832913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3798103660 msgr2=0x7f3798198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:46.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3798103660 0x7f3798198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 -- 172.21.15.16:0/2625832913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3798102d40 msgr2=0x7f37981983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3798102d40 0x7f37981983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 -- 172.21.15.16:0/2625832913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379819d3a0 con 0x7f379810c170 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f3796d9d640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379819ccc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f379002ae70 tx=0x7f379002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f37877fe640 1 -- 172.21.15.16:0/2625832913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790034040 con 0x7f379810c170 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f37877fe640 1 -- 172.21.15.16:0/2625832913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3790030920 con 0x7f379810c170 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f379819d630 con 0x7f379810c170 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.883+0000 7f37877fe640 1 -- 172.21.15.16:0/2625832913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3790045480 con 0x7f379810c170 2024-09-15T22:36:46.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.884+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37981a4f70 con 0x7f379810c170 2024-09-15T22:36:46.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.885+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f379806b070 con 0x7f379810c170 2024-09-15T22:36:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.886+0000 7f37877fe640 1 -- 172.21.15.16:0/2625832913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3790037020 con 0x7f379810c170 2024-09-15T22:36:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.887+0000 7f37877fe640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f37800a3240 0x7f37800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.887+0000 7f379659c640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f37800a3240 0x7f37800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.887+0000 7f37877fe640 1 -- 172.21.15.16:0/2625832913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f37900f0c30 con 0x7f379810c170 2024-09-15T22:36:46.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.887+0000 7f379659c640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f37800a3240 0x7f37800a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37880058c0 tx=0x7f378800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:46.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:46.890+0000 7f37877fe640 1 -- 172.21.15.16:0/2625832913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37900811b0 con 0x7f379810c170 2024-09-15T22:36:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.101+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f37981071e0 con 0x7f379810c170 2024-09-15T22:36:47.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.385+0000 7f37877fe640 1 -- 172.21.15.16:0/2625832913 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f37900bd6f0 con 0x7f379810c170 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f37800a3240 msgr2=0x7f37800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f37800a3240 0x7f37800a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f37880058c0 tx=0x7f378800a4c0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 msgr2=0x7f379819ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379819ccc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f379002ae70 tx=0x7f379002f5b0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 shutdown_connections 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f37800a3240 0x7f37800a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f379810c170 0x7f379819ccc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3798103660 0x7f3798198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 --2- 172.21.15.16:0/2625832913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3798102d40 0x7f37981983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.387+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 >> 172.21.15.16:0/2625832913 conn(0x7f37980fc7a0 msgr2=0x7f37980fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.388+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 shutdown_connections 2024-09-15T22:36:47.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:47.388+0000 7f379cf07640 1 -- 172.21.15.16:0/2625832913 wait complete. 2024-09-15T22:36:47.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:47 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2625832913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:36:47.701 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:47 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2625832913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:36:47.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:47 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2625832913' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:36:48.139 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-15T22:36:48.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-15T22:36:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:48.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:48 smithi016 ceph-mon[31492]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-15T22:36:48.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:48 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2625832913' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:36:48.665 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:48 smithi016 ceph-mon[31492]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:36:48.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:48 smithi059 ceph-mon[34872]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-15T22:36:48.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:48 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2625832913' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:36:48.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:48 smithi059 ceph-mon[34872]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:36:48.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:48 smithi130 ceph-mon[35742]: pgmap v303: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-15T22:36:48.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:48 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2625832913' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:36:48.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:48 smithi130 ceph-mon[35742]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:36:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 -- 172.21.15.16:0/1768133452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 msgr2=0x7f8ab4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 --2- 172.21.15.16:0/1768133452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab4100ee0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009930 tx=0x7f8aa802a420 comp rx=0 tx=0).stop 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 -- 172.21.15.16:0/1768133452 shutdown_connections 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 --2- 172.21.15.16:0/1768133452 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ab4101420 0x7f8ab410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 --2- 172.21.15.16:0/1768133452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab4100ee0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 --2- 172.21.15.16:0/1768133452 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4106a80 0x7f8ab4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 -- 172.21.15.16:0/1768133452 >> 172.21.15.16:0/1768133452 conn(0x7f8ab40fc7a0 msgr2=0x7f8ab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 -- 172.21.15.16:0/1768133452 shutdown_connections 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.890+0000 7f8abb02c640 1 -- 172.21.15.16:0/1768133452 wait complete. 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 Processor -- start 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 -- start start 2024-09-15T22:36:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4101420 0x7f8ab419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ab4106a80 0x7f8ab419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4110e80 con 0x7f8ab4100a80 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4110d00 con 0x7f8ab4101420 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8abb02c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111000 con 0x7f8ab4106a80 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60902/0 (socket says 172.21.15.16:60902) 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 -- 172.21.15.16:0/892194322 learned_addr learned my addr 172.21.15.16:0/892194322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab3fff640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4101420 0x7f8ab419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab95c9640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ab4106a80 0x7f8ab419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 -- 172.21.15.16:0/892194322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ab4106a80 msgr2=0x7f8ab419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ab4106a80 0x7f8ab419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 -- 172.21.15.16:0/892194322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4101420 msgr2=0x7f8ab419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4101420 0x7f8ab419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.891+0000 7f8ab8dc8640 1 -- 172.21.15.16:0/892194322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab419f5f0 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.892+0000 7f8ab8dc8640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab419a640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400c890 tx=0x7f8aa400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.892+0000 7f8ab1ffb640 1 -- 172.21.15.16:0/892194322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4004220 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.892+0000 7f8ab1ffb640 1 -- 172.21.15.16:0/892194322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa40043c0 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.892+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab419f8e0 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.892+0000 7f8ab1ffb640 1 -- 172.21.15.16:0/892194322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40049e0 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.892+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a71c0 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.893+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab4100ee0 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.894+0000 7f8ab1ffb640 1 -- 172.21.15.16:0/892194322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8aa40079b0 con 0x7f8ab4100a80 2024-09-15T22:36:48.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.895+0000 7f8ab1ffb640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.895+0000 7f8ab3fff640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.895+0000 7f8ab1ffb640 1 -- 172.21.15.16:0/892194322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f8aa40cfaa0 con 0x7f8ab4100a80 2024-09-15T22:36:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.896+0000 7f8ab3fff640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:48.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:48.900+0000 7f8ab1ffb640 1 -- 172.21.15.16:0/892194322 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8aa4014030 con 0x7f8ab4100a80 2024-09-15T22:36:49.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.100+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8ab41039e0 con 0x7f8a9c0a3240 2024-09-15T22:36:49.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.124+0000 7f8ab1ffb640 1 -- 172.21.15.16:0/892194322 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ab41039e0 con 0x7f8a9c0a3240 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a9c0a3240 msgr2=0x7f8a9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa80058d0 comp rx=0 tx=0).stop 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 msgr2=0x7f8ab419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab419a640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400c890 tx=0x7f8aa400cd50 comp rx=0 tx=0).stop 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 shutdown_connections 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8ab4106a80 0x7f8ab419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ab4101420 0x7f8ab419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 --2- 172.21.15.16:0/892194322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ab4100a80 0x7f8ab419a640 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 >> 172.21.15.16:0/892194322 conn(0x7f8ab40fc7a0 msgr2=0x7f8ab406a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.126+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 shutdown_connections 2024-09-15T22:36:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:49.127+0000 7f8abb02c640 1 -- 172.21.15.16:0/892194322 wait complete. 2024-09-15T22:36:49.444 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:36:49 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: 2024-09-15T22:36:49.081+0000 7f0047508640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-15T22:36:49.445 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:36:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:36:49.105+0000 7f81c36f0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:36:49.445 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:36:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:36:49.105+0000 7f81c36f0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:36:49.445 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:36:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:36:49.105+0000 7f81c36f0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:36:49.446 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:36:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:36:49.105+0000 7f81c36f0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:36:49.446 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:36:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:36:49.105+0000 7f81c36f0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:36:49.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-15T22:36:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:50.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:50 smithi016 ceph-mon[31492]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-15T22:36:50.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:50 smithi016 ceph-mon[31492]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:50.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 -- 172.21.15.16:0/2981803294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90100d00 msgr2=0x7fed90106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 --2- 172.21.15.16:0/2981803294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90100d00 0x7fed90106840 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fed78009930 tx=0x7fed7802a420 comp rx=0 tx=0).stop 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 -- 172.21.15.16:0/2981803294 shutdown_connections 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 --2- 172.21.15.16:0/2981803294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed90106f50 0x7fed9010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 --2- 172.21.15.16:0/2981803294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90100d00 0x7fed90106840 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 --2- 172.21.15.16:0/2981803294 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed90100350 0x7fed90100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 -- 172.21.15.16:0/2981803294 >> 172.21.15.16:0/2981803294 conn(0x7fed90077210 msgr2=0x7fed900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 -- 172.21.15.16:0/2981803294 shutdown_connections 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.680+0000 7fed96971640 1 -- 172.21.15.16:0/2981803294 wait complete. 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 Processor -- start 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 -- start start 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed90100350 0x7fed90198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:50.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed90100d00 0x7fed90198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90106f50 0x7fed9019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fed90111fa0 con 0x7fed90106f50 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed90111e20 con 0x7fed90100d00 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.681+0000 7fed96971640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fed90112120 con 0x7fed90100350 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90106f50 0x7fed9019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90106f50 0x7fed9019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60924/0 (socket says 172.21.15.16:60924) 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 -- 172.21.15.16:0/2997552462 learned_addr learned my addr 172.21.15.16:0/2997552462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed8f7fe640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed90100d00 0x7fed90198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed8ffff640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed90100350 0x7fed90198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 -- 172.21.15.16:0/2997552462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed90100350 msgr2=0x7fed90198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:50.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed90100350 0x7fed90198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 -- 172.21.15.16:0/2997552462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed90100d00 msgr2=0x7fed90198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed90100d00 0x7fed90198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 -- 172.21.15.16:0/2997552462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9019d4b0 con 0x7fed90106f50 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed8ffff640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed90100350 0x7fed90198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed94f0e640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90106f50 0x7fed9019cdd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed88009fd0 tx=0x7fed8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed8d7fa640 1 -- 172.21.15.16:0/2997552462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88017070 con 0x7fed90106f50 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9019d7a0 con 0x7fed90106f50 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed8d7fa640 1 -- 172.21.15.16:0/2997552462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed88003e50 con 0x7fed90106f50 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed8d7fa640 1 -- 172.21.15.16:0/2997552462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed880106c0 con 0x7fed90106f50 2024-09-15T22:36:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.682+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed901a5080 con 0x7fed90106f50 2024-09-15T22:36:50.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.683+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed90103410 con 0x7fed90106f50 2024-09-15T22:36:50.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.685+0000 7fed8d7fa640 1 -- 172.21.15.16:0/2997552462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fed88003970 con 0x7fed90106f50 2024-09-15T22:36:50.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.685+0000 7fed8d7fa640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fed7c0a3240 0x7fed7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:50.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.685+0000 7fed8d7fa640 1 -- 172.21.15.16:0/2997552462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fed880cf8e0 con 0x7fed90106f50 2024-09-15T22:36:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.685+0000 7fed8ffff640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fed7c0a3240 0x7fed7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.686+0000 7fed8ffff640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fed7c0a3240 0x7fed7c0a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fed800058c0 tx=0x7fed80005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.690+0000 7fed8d7fa640 1 -- 172.21.15.16:0/2997552462 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed8809c3c0 con 0x7fed90106f50 2024-09-15T22:36:50.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:50 smithi059 ceph-mon[34872]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-15T22:36:50.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:50 smithi059 ceph-mon[34872]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:50.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:50 smithi130 ceph-mon[35742]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-15T22:36:50.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:50 smithi130 ceph-mon[35742]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.896+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fed9010a190 con 0x7fed7c0a3240 2024-09-15T22:36:50.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.920+0000 7fed8d7fa640 1 -- 172.21.15.16:0/2997552462 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fed9010a190 con 0x7fed7c0a3240 2024-09-15T22:36:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fed7c0a3240 msgr2=0x7fed7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fed7c0a3240 0x7fed7c0a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fed800058c0 tx=0x7fed80005ec0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90106f50 msgr2=0x7fed9019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90106f50 0x7fed9019cdd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fed88009fd0 tx=0x7fed8800b260 comp rx=0 tx=0).stop 2024-09-15T22:36:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 shutdown_connections 2024-09-15T22:36:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fed7c0a3240 0x7fed7c0a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed90106f50 0x7fed9019cdd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed90100d00 0x7fed90198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 --2- 172.21.15.16:0/2997552462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fed90100350 0x7fed90198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 >> 172.21.15.16:0/2997552462 conn(0x7fed90077210 msgr2=0x7fed900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.922+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 shutdown_connections 2024-09-15T22:36:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:50.923+0000 7fed96971640 1 -- 172.21.15.16:0/2997552462 wait complete. 2024-09-15T22:36:51.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-15T22:36:51.986 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 -- 172.21.15.16:0/3028923712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 msgr2=0x7fec6c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 --2- 172.21.15.16:0/3028923712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c1021c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec68009900 tx=0x7fec6802a410 comp rx=0 tx=0).stop 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 -- 172.21.15.16:0/3028923712 shutdown_connections 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 --2- 172.21.15.16:0/3028923712 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec6c106f10 0x7fec6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 --2- 172.21.15.16:0/3028923712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c102790 0x7fec6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 --2- 172.21.15.16:0/3028923712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c1021c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 -- 172.21.15.16:0/3028923712 >> 172.21.15.16:0/3028923712 conn(0x7fec6c0fd8c0 msgr2=0x7fec6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:52.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 -- 172.21.15.16:0/3028923712 shutdown_connections 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 -- 172.21.15.16:0/3028923712 wait complete. 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 Processor -- start 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.481+0000 7fec736a8640 1 -- start start 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec736a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec736a8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c102790 0x7fec6c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec736a8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec6c106f10 0x7fec6c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec736a8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c111fb0 con 0x7fec6c101de0 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec736a8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c111e30 con 0x7fec6c102790 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec736a8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fec6c112130 con 0x7fec6c106f10 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49812/0 (socket says 172.21.15.16:49812) 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 -- 172.21.15.16:0/3814584005 learned_addr learned my addr 172.21.15.16:0/3814584005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:52.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec70c43640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c102790 0x7fec6c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 -- 172.21.15.16:0/3814584005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec6c106f10 msgr2=0x7fec6c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71c45640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec6c106f10 0x7fec6c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec6c106f10 0x7fec6c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 -- 172.21.15.16:0/3814584005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c102790 msgr2=0x7fec6c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c102790 0x7fec6c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 -- 172.21.15.16:0/3814584005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6c19d4c0 con 0x7fec6c101de0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.482+0000 7fec71444640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c198510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec68006010 tx=0x7fec680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.483+0000 7fec627fc640 1 -- 172.21.15.16:0/3814584005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6803f030 con 0x7fec6c101de0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.483+0000 7fec627fc640 1 -- 172.21.15.16:0/3814584005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec68003ce0 con 0x7fec6c101de0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.483+0000 7fec627fc640 1 -- 172.21.15.16:0/3814584005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68035070 con 0x7fec6c101de0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.483+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec6c19d750 con 0x7fec6c101de0 2024-09-15T22:36:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.483+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6c1a5090 con 0x7fec6c101de0 2024-09-15T22:36:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.484+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6c1032c0 con 0x7fec6c101de0 2024-09-15T22:36:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.485+0000 7fec627fc640 1 -- 172.21.15.16:0/3814584005 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fec68042030 con 0x7fec6c101de0 2024-09-15T22:36:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.486+0000 7fec627fc640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fec580a32d0 0x7fec580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.486+0000 7fec70c43640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fec580a32d0 0x7fec580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.486+0000 7fec627fc640 1 -- 172.21.15.16:0/3814584005 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fec680f0790 con 0x7fec6c101de0 2024-09-15T22:36:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.486+0000 7fec70c43640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fec580a32d0 0x7fec580a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0fdd00 tx=0x7fec5c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.489+0000 7fec627fc640 1 -- 172.21.15.16:0/3814584005 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec680bd170 con 0x7fec6c101de0 2024-09-15T22:36:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:52 smithi059 ceph-mon[34872]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-15T22:36:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:52 smithi059 ceph-mon[34872]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:52.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:52 smithi059 ceph-mon[34872]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:36:52.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.694+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fec6c062630 con 0x7fec580a32d0 2024-09-15T22:36:52.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:52 smithi016 ceph-mon[31492]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-15T22:36:52.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:52 smithi016 ceph-mon[31492]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:52.695 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:52 smithi016 ceph-mon[31492]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:36:52.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.698+0000 7fec627fc640 1 -- 172.21.15.16:0/3814584005 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fec6c062630 con 0x7fec580a32d0 2024-09-15T22:36:52.700 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/e20bd8a2-729f-4bb4-864f-885a1bf35332 2024-09-15T22:36:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fec580a32d0 msgr2=0x7fec580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fec580a32d0 0x7fec580a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0fdd00 tx=0x7fec5c008040 comp rx=0 tx=0).stop 2024-09-15T22:36:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 msgr2=0x7fec6c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c198510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec68006010 tx=0x7fec680042f0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 shutdown_connections 2024-09-15T22:36:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fec580a32d0 0x7fec580a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fec6c106f10 0x7fec6c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec6c102790 0x7fec6c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 --2- 172.21.15.16:0/3814584005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec6c101de0 0x7fec6c198510 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.700+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 >> 172.21.15.16:0/3814584005 conn(0x7fec6c0fd8c0 msgr2=0x7fec6c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.701+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 shutdown_connections 2024-09-15T22:36:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:52.701+0000 7fec736a8640 1 -- 172.21.15.16:0/3814584005 wait complete. 2024-09-15T22:36:52.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:52 smithi130 ceph-mon[35742]: pgmap v305: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-15T22:36:52.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:52 smithi130 ceph-mon[35742]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:52.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:52 smithi130 ceph-mon[35742]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:36:53.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-15T22:36:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 -- 172.21.15.16:0/2023983653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102ef0 msgr2=0x7faf6410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 --2- 172.21.15.16:0/2023983653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102ef0 0x7faf6410da40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009ca0 tx=0x7faf5c02a950 comp rx=0 tx=0).stop 2024-09-15T22:36:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 -- 172.21.15.16:0/2023983653 shutdown_connections 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 --2- 172.21.15.16:0/2023983653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102ef0 0x7faf6410da40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 --2- 172.21.15.16:0/2023983653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf64102570 0x7faf641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 --2- 172.21.15.16:0/2023983653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf64100e20 0x7faf64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 -- 172.21.15.16:0/2023983653 >> 172.21.15.16:0/2023983653 conn(0x7faf640fc7d0 msgr2=0x7faf640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 -- 172.21.15.16:0/2023983653 shutdown_connections 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 -- 172.21.15.16:0/2023983653 wait complete. 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 Processor -- start 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.390+0000 7faf6a286640 1 -- start start 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf6a286640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf64100e20 0x7faf64199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf6a286640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102570 0x7faf64199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf6a286640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf64102ef0 0x7faf641a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf6a286640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf64110f80 con 0x7faf64102570 2024-09-15T22:36:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf6a286640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf64110e00 con 0x7faf64100e20 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf6a286640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111100 con 0x7faf64102ef0 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102570 0x7faf64199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102570 0x7faf64199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49834/0 (socket says 172.21.15.16:49834) 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 -- 172.21.15.16:0/2095362866 learned_addr learned my addr 172.21.15.16:0/2095362866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf68823640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf64102ef0 0x7faf641a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf63fff640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf64100e20 0x7faf64199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 -- 172.21.15.16:0/2095362866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf64102ef0 msgr2=0x7faf641a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf64102ef0 0x7faf641a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 -- 172.21.15.16:0/2095362866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf64100e20 msgr2=0x7faf64199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf64100e20 0x7faf64199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.391+0000 7faf637fe640 1 -- 172.21.15.16:0/2095362866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf641a47f0 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.392+0000 7faf637fe640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102570 0x7faf64199770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c920 tx=0x7faf5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.392+0000 7faf617fa640 1 -- 172.21.15.16:0/2095362866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58007c70 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.392+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf641a49c0 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.392+0000 7faf617fa640 1 -- 172.21.15.16:0/2095362866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf58007e10 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.392+0000 7faf617fa640 1 -- 172.21.15.16:0/2095362866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58018890 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.392+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a4f50 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.393+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf64101200 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.394+0000 7faf617fa640 1 -- 172.21.15.16:0/2095362866 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7faf58018a70 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.395+0000 7faf617fa640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faf4c0a32d0 0x7faf4c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.395+0000 7faf63fff640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faf4c0a32d0 0x7faf4c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.395+0000 7faf617fa640 1 -- 172.21.15.16:0/2095362866 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7faf580d01a0 con 0x7faf64102570 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.395+0000 7faf63fff640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faf4c0a32d0 0x7faf4c0a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf5400bbe0 tx=0x7faf5400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.398+0000 7faf617fa640 1 -- 172.21.15.16:0/2095362866 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf58014030 con 0x7faf64102570 2024-09-15T22:36:54.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:54 smithi016 ceph-mon[31492]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-15T22:36:54.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:54 smithi016 ceph-mon[31492]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.602+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faf64106ed0 con 0x7faf4c0a32d0 2024-09-15T22:36:54.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.626+0000 7faf617fa640 1 -- 172.21.15.16:0/2095362866 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faf64106ed0 con 0x7faf4c0a32d0 2024-09-15T22:36:54.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faf4c0a32d0 msgr2=0x7faf4c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faf4c0a32d0 0x7faf4c0a5790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf5400bbe0 tx=0x7faf5400b390 comp rx=0 tx=0).stop 2024-09-15T22:36:54.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102570 msgr2=0x7faf64199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102570 0x7faf64199770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c920 tx=0x7faf5800cde0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 shutdown_connections 2024-09-15T22:36:54.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7faf4c0a32d0 0x7faf4c0a5790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf64102ef0 0x7faf641a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64102570 0x7faf64199770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 --2- 172.21.15.16:0/2095362866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf64100e20 0x7faf64199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 >> 172.21.15.16:0/2095362866 conn(0x7faf640fc7d0 msgr2=0x7faf641067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:54.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 shutdown_connections 2024-09-15T22:36:54.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:54.628+0000 7faf6a286640 1 -- 172.21.15.16:0/2095362866 wait complete. 2024-09-15T22:36:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:54 smithi059 ceph-mon[34872]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-15T22:36:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:54 smithi059 ceph-mon[34872]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:54.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:54 smithi130 ceph-mon[35742]: pgmap v306: 129 pgs: 129 active+clean; 584 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-15T22:36:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:54 smithi130 ceph-mon[35742]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:55.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-15T22:36:55.600 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:56.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 -- 172.21.15.16:0/3059041460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258102790 msgr2=0x7f42581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 --2- 172.21.15.16:0/3059041460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258102790 0x7f42581067e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f424c009900 tx=0x7f424c02a410 comp rx=0 tx=0).stop 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 -- 172.21.15.16:0/3059041460 shutdown_connections 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 --2- 172.21.15.16:0/3059041460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4258106f10 0x7f425810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 --2- 172.21.15.16:0/3059041460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258102790 0x7f42581067e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 --2- 172.21.15.16:0/3059041460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4258101de0 0x7f42581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 -- 172.21.15.16:0/3059041460 >> 172.21.15.16:0/3059041460 conn(0x7f42580fd8c0 msgr2=0x7f42580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 -- 172.21.15.16:0/3059041460 shutdown_connections 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.065+0000 7f425cfcd640 1 -- 172.21.15.16:0/3059041460 wait complete. 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.066+0000 7f425cfcd640 1 Processor -- start 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.066+0000 7f425cfcd640 1 -- start start 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425cfcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258101de0 0x7f42581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425cfcd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4258102790 0x7f4258198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425cfcd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4258106f10 0x7f425819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425cfcd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4258111f70 con 0x7f4258101de0 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425cfcd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4258111df0 con 0x7f4258102790 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425cfcd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f42581120f0 con 0x7f4258106f10 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258101de0 0x7f42581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258101de0 0x7f42581983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49856/0 (socket says 172.21.15.16:49856) 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 -- 172.21.15.16:0/535180750 learned_addr learned my addr 172.21.15.16:0/535180750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f4255d9b640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4258102790 0x7f4258198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f4256d9d640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4258106f10 0x7f425819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 -- 172.21.15.16:0/535180750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4258106f10 msgr2=0x7f425819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4258106f10 0x7f425819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 -- 172.21.15.16:0/535180750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4258102790 msgr2=0x7f4258198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4258102790 0x7f4258198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 -- 172.21.15.16:0/535180750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425819d380 con 0x7f4258101de0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f4256d9d640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4258106f10 0x7f425819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f4255d9b640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4258102790 0x7f4258198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425659c640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258101de0 0x7f42581983d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f424800d770 tx=0x7f424800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f42477fe640 1 -- 172.21.15.16:0/535180750 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248017070 con 0x7f4258101de0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f42477fe640 1 -- 172.21.15.16:0/535180750 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4248005500 con 0x7f4258101de0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.067+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425819d670 con 0x7f4258101de0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.068+0000 7f42477fe640 1 -- 172.21.15.16:0/535180750 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4248004e00 con 0x7f4258101de0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.068+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42581a4f50 con 0x7f4258101de0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.069+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42581021c0 con 0x7f4258101de0 2024-09-15T22:36:56.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.070+0000 7f42477fe640 1 -- 172.21.15.16:0/535180750 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4248004fa0 con 0x7f4258101de0 2024-09-15T22:36:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.071+0000 7f42477fe640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f42400a3350 0x7f42400a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.071+0000 7f4255d9b640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f42400a3350 0x7f42400a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.071+0000 7f42477fe640 1 -- 172.21.15.16:0/535180750 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f42480cf7a0 con 0x7f4258101de0 2024-09-15T22:36:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.072+0000 7f4255d9b640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f42400a3350 0x7f42400a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f42581088e0 tx=0x7f424c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:56.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.075+0000 7f42477fe640 1 -- 172.21.15.16:0/535180750 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f424809c120 con 0x7f4258101de0 2024-09-15T22:36:56.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.281+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f425810a1c0 con 0x7f42400a3350 2024-09-15T22:36:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.286+0000 7f42477fe640 1 -- 172.21.15.16:0/535180750 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f424c0316a0 con 0x7f42400a3350 2024-09-15T22:36:56.287 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/8f003b02-afc3-47c1-8722-deab403be128 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f42400a3350 msgr2=0x7f42400a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f42400a3350 0x7f42400a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f42581088e0 tx=0x7f424c005a20 comp rx=0 tx=0).stop 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258101de0 msgr2=0x7f42581983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258101de0 0x7f42581983d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f424800d770 tx=0x7f424800dc30 comp rx=0 tx=0).stop 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 shutdown_connections 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f42400a3350 0x7f42400a5810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4258106f10 0x7f425819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4258102790 0x7f4258198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 --2- 172.21.15.16:0/535180750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4258101de0 0x7f42581983d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.288+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 >> 172.21.15.16:0/535180750 conn(0x7f42580fd8c0 msgr2=0x7f42580fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.289+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 shutdown_connections 2024-09-15T22:36:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:56.289+0000 7f425cfcd640 1 -- 172.21.15.16:0/535180750 wait complete. 2024-09-15T22:36:56.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:56 smithi016 ceph-mon[31492]: pgmap v307: 129 pgs: 129 active+clean; 605 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-15T22:36:56.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:56 smithi016 ceph-mon[31492]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:56.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:56 smithi059 ceph-mon[34872]: pgmap v307: 129 pgs: 129 active+clean; 605 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-15T22:36:56.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:56 smithi059 ceph-mon[34872]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:56.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:56 smithi130 ceph-mon[35742]: pgmap v307: 129 pgs: 129 active+clean; 605 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-15T22:36:56.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:56 smithi130 ceph-mon[35742]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:56.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:36:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:57.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:57 smithi016 ceph-mon[31492]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:57.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:57 smithi059 ceph-mon[34872]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:57.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 -- 172.21.15.16:0/3939035820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8101480 msgr2=0x7fc5c8101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3939035820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8101480 0x7fc5c8101860 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009ca0 tx=0x7fc5b802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 -- 172.21.15.16:0/3939035820 shutdown_connections 2024-09-15T22:36:57.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3939035820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5c8105e80 0x7fc5c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3939035820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c8101e30 0x7fc5c8105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3939035820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8101480 0x7fc5c8101860 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 -- 172.21.15.16:0/3939035820 >> 172.21.15.16:0/3939035820 conn(0x7fc5c8077210 msgr2=0x7fc5c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 -- 172.21.15.16:0/3939035820 shutdown_connections 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 -- 172.21.15.16:0/3939035820 wait complete. 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 Processor -- start 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.755+0000 7fc5d0489640 1 -- start start 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5d0489640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5c8101480 0x7fc5c8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5d0489640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c8101e30 0x7fc5c8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5d0489640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8105e80 0x7fc5c819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5d0489640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8110f20 con 0x7fc5c8105e80 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5d0489640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8110da0 con 0x7fc5c8101e30 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5d0489640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c81110a0 con 0x7fc5c8101480 2024-09-15T22:36:57.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8105e80 0x7fc5c819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8105e80 0x7fc5c819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49876/0 (socket says 172.21.15.16:49876) 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 -- 172.21.15.16:0/3526299031 learned_addr learned my addr 172.21.15.16:0/3526299031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5ce225640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5c8101480 0x7fc5c8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cda24640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c8101e30 0x7fc5c8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 -- 172.21.15.16:0/3526299031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5c8101480 msgr2=0x7fc5c8198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5c8101480 0x7fc5c8198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 -- 172.21.15.16:0/3526299031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c8101e30 msgr2=0x7fc5c8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c8101e30 0x7fc5c8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 -- 172.21.15.16:0/3526299031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c819d4f0 con 0x7fc5c8105e80 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cda24640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c8101e30 0x7fc5c8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:36:57.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.756+0000 7fc5cea26640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8105e80 0x7fc5c819ce10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000d750 tx=0x7fc5c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.757+0000 7fc5c77fe640 1 -- 172.21.15.16:0/3526299031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0017070 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.757+0000 7fc5c77fe640 1 -- 172.21.15.16:0/3526299031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c0005500 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.757+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c819d7e0 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.757+0000 7fc5c77fe640 1 -- 172.21.15.16:0/3526299031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0004d90 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.757+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c81a50c0 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.758+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8101860 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.759+0000 7fc5c77fe640 1 -- 172.21.15.16:0/3526299031 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc5c0004f30 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.760+0000 7fc5c77fe640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc5b00a3350 0x7fc5b00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.760+0000 7fc5ce225640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc5b00a3350 0x7fc5b00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.760+0000 7fc5c77fe640 1 -- 172.21.15.16:0/3526299031 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fc5c00cf640 con 0x7fc5c8105e80 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.760+0000 7fc5ce225640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc5b00a3350 0x7fc5b00a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000f80 tx=0x7fc5b80093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:57.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.764+0000 7fc5c77fe640 1 -- 172.21.15.16:0/3526299031 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5c009bfc0 con 0x7fc5c8105e80 2024-09-15T22:36:57.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:57 smithi130 ceph-mon[35742]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.980+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5c81a5320 con 0x7fc5c8105e80 2024-09-15T22:36:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.981+0000 7fc5c77fe640 1 -- 172.21.15.16:0/3526299031 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fc5c009b8c0 con 0x7fc5c8105e80 2024-09-15T22:36:57.983 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:57.983 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-15T22:36:47:380763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:47.103123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24469,24479,24500]},"id":1}]} 2024-09-15T22:36:57.983 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc5b00a3350 msgr2=0x7fc5b00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc5b00a3350 0x7fc5b00a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000f80 tx=0x7fc5b80093f0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8105e80 msgr2=0x7fc5c819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8105e80 0x7fc5c819ce10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000d750 tx=0x7fc5c000dc10 comp rx=0 tx=0).stop 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 shutdown_connections 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc5b00a3350 0x7fc5b00a5810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5c8105e80 0x7fc5c819ce10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5c8101e30 0x7fc5c8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 --2- 172.21.15.16:0/3526299031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc5c8101480 0x7fc5c8198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:57.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 >> 172.21.15.16:0/3526299031 conn(0x7fc5c8077210 msgr2=0x7fc5c8104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 shutdown_connections 2024-09-15T22:36:57.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:57.983+0000 7fc5d0489640 1 -- 172.21.15.16:0/3526299031 wait complete. 2024-09-15T22:36:58.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:58 smithi059 ceph-mon[34872]: pgmap v308: 129 pgs: 129 active+clean; 605 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-15T22:36:58.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:36:58 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3526299031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:58.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:58 smithi016 ceph-mon[31492]: pgmap v308: 129 pgs: 129 active+clean; 605 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-15T22:36:58.753 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:36:58 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3526299031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:58.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:58 smithi130 ceph-mon[35742]: pgmap v308: 129 pgs: 129 active+clean; 605 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-15T22:36:58.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:36:58 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3526299031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:36:58.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:36:59.046 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.554+0000 7fc86fc2a640 1 -- 172.21.15.16:0/1939099696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc868101280 msgr2=0x7fc86810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.554+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/1939099696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc868101280 0x7fc86810df40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc858009900 tx=0x7fc85802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.554+0000 7fc86fc2a640 1 -- 172.21.15.16:0/1939099696 shutdown_connections 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.554+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/1939099696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc86810e480 0x7fc868110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.554+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/1939099696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc868101280 0x7fc86810df40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.554+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/1939099696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc8681008d0 0x7fc868100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.554+0000 7fc86fc2a640 1 -- 172.21.15.16:0/1939099696 >> 172.21.15.16:0/1939099696 conn(0x7fc8680fc7a0 msgr2=0x7fc8680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 -- 172.21.15.16:0/1939099696 shutdown_connections 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 -- 172.21.15.16:0/1939099696 wait complete. 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 Processor -- start 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 -- start start 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8681008d0 0x7fc86819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc868101280 0x7fc86819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86810e480 0x7fc86819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc868114190 con 0x7fc86810e480 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc868114010 con 0x7fc8681008d0 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.555+0000 7fc86fc2a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc868114310 con 0x7fc868101280 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86810e480 0x7fc86819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86810e480 0x7fc86819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49898/0 (socket says 172.21.15.16:49898) 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 -- 172.21.15.16:0/638994543 learned_addr learned my addr 172.21.15.16:0/638994543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:36:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86d9c6640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8681008d0 0x7fc86819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86d1c5640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc868101280 0x7fc86819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 -- 172.21.15.16:0/638994543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc868101280 msgr2=0x7fc86819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc868101280 0x7fc86819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 -- 172.21.15.16:0/638994543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8681008d0 msgr2=0x7fc86819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8681008d0 0x7fc86819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 -- 172.21.15.16:0/638994543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86819f6c0 con 0x7fc86810e480 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86e1c7640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86810e480 0x7fc86819efe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc86000cce0 tx=0x7fc86000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc85effd640 1 -- 172.21.15.16:0/638994543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000f450 con 0x7fc86810e480 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc85effd640 1 -- 172.21.15.16:0/638994543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86000fad0 con 0x7fc86810e480 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86819f9b0 con 0x7fc86810e480 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc85effd640 1 -- 172.21.15.16:0/638994543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860018bd0 con 0x7fc86810e480 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.556+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8681a7290 con 0x7fc86810e480 2024-09-15T22:36:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.557+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc868100cb0 con 0x7fc86810e480 2024-09-15T22:36:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.558+0000 7fc85effd640 1 -- 172.21.15.16:0/638994543 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc86000fc80 con 0x7fc86810e480 2024-09-15T22:36:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.559+0000 7fc85effd640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8540a32d0 0x7fc8540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.559+0000 7fc85effd640 1 -- 172.21.15.16:0/638994543 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fc8600d0210 con 0x7fc86810e480 2024-09-15T22:36:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.559+0000 7fc86d9c6640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8540a32d0 0x7fc8540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.560+0000 7fc86d9c6640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8540a32d0 0x7fc8540a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc86400bbe0 tx=0x7fc86400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.563+0000 7fc85effd640 1 -- 172.21.15.16:0/638994543 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc860014030 con 0x7fc86810e480 2024-09-15T22:36:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.740+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc868109080 con 0x7fc86810e480 2024-09-15T22:36:59.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.742+0000 7fc85effd640 1 -- 172.21.15.16:0/638994543 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7fc86009cc30 con 0x7fc86810e480 2024-09-15T22:36:59.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:36:59.744 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:36:59.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8540a32d0 msgr2=0x7fc8540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8540a32d0 0x7fc8540a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc86400bbe0 tx=0x7fc86400b390 comp rx=0 tx=0).stop 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86810e480 msgr2=0x7fc86819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86810e480 0x7fc86819efe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc86000cce0 tx=0x7fc86000a620 comp rx=0 tx=0).stop 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 shutdown_connections 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8540a32d0 0x7fc8540a5790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc86810e480 0x7fc86819efe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc868101280 0x7fc86819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 --2- 172.21.15.16:0/638994543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8681008d0 0x7fc86819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 >> 172.21.15.16:0/638994543 conn(0x7fc8680fc7a0 msgr2=0x7fc8680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 shutdown_connections 2024-09-15T22:36:59.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:36:59.744+0000 7fc86fc2a640 1 -- 172.21.15.16:0/638994543 wait complete. 2024-09-15T22:37:00.432 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:37:00.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:37:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:00 smithi059 ceph-mon[34872]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-15T22:37:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:00 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/638994543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:00.695 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:00.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:00 smithi130 ceph-mon[35742]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-15T22:37:00.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:00 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/638994543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:00.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:00 smithi016 ceph-mon[31492]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-15T22:37:00.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:00 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/638994543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 -- 172.21.15.16:0/1213147124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c1008d0 msgr2=0x7ff00c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 --2- 172.21.15.16:0/1213147124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c1008d0 0x7ff00c100cb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff008009900 tx=0x7ff00802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 -- 172.21.15.16:0/1213147124 shutdown_connections 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 --2- 172.21.15.16:0/1213147124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff00c10e480 0x7ff00c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 --2- 172.21.15.16:0/1213147124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff00c101280 0x7ff00c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 --2- 172.21.15.16:0/1213147124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c1008d0 0x7ff00c100cb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 -- 172.21.15.16:0/1213147124 >> 172.21.15.16:0/1213147124 conn(0x7ff00c0fc7a0 msgr2=0x7ff00c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 -- 172.21.15.16:0/1213147124 shutdown_connections 2024-09-15T22:37:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.171+0000 7ff013c2f640 1 -- 172.21.15.16:0/1213147124 wait complete. 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 Processor -- start 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 -- start start 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff00c1008d0 0x7ff00c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff00c101280 0x7ff00c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c10e480 0x7ff00c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c114190 con 0x7ff00c10e480 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c114010 con 0x7ff00c1008d0 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff013c2f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c114310 con 0x7ff00c101280 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff0121cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c10e480 0x7ff00c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff0121cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c10e480 0x7ff00c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49916/0 (socket says 172.21.15.16:49916) 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.172+0000 7ff0121cc640 1 -- 172.21.15.16:0/2123404529 learned_addr learned my addr 172.21.15.16:0/2123404529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0111ca640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff00c101280 0x7ff00c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:01.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0119cb640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff00c1008d0 0x7ff00c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0121cc640 1 -- 172.21.15.16:0/2123404529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff00c101280 msgr2=0x7ff00c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0121cc640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff00c101280 0x7ff00c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0121cc640 1 -- 172.21.15.16:0/2123404529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff00c1008d0 msgr2=0x7ff00c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0121cc640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff00c1008d0 0x7ff00c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0121cc640 1 -- 172.21.15.16:0/2123404529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c19f560 con 0x7ff00c10e480 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0111ca640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff00c101280 0x7ff00c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff0121cc640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c10e480 0x7ff00c19ee80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff00400cce0 tx=0x7ff00400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff002ffd640 1 -- 172.21.15.16:0/2123404529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00400f450 con 0x7ff00c10e480 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff002ffd640 1 -- 172.21.15.16:0/2123404529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff00400fad0 con 0x7ff00c10e480 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff002ffd640 1 -- 172.21.15.16:0/2123404529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004018bd0 con 0x7ff00c10e480 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c19f850 con 0x7ff00c10e480 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.173+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c1a7130 con 0x7ff00c10e480 2024-09-15T22:37:01.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.174+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c100cb0 con 0x7ff00c10e480 2024-09-15T22:37:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.175+0000 7ff002ffd640 1 -- 172.21.15.16:0/2123404529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff00400fc80 con 0x7ff00c10e480 2024-09-15T22:37:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.176+0000 7ff002ffd640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7feff80a32d0 0x7feff80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.176+0000 7ff002ffd640 1 -- 172.21.15.16:0/2123404529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7ff0040d0460 con 0x7ff00c10e480 2024-09-15T22:37:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.176+0000 7ff0119cb640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7feff80a32d0 0x7feff80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.177+0000 7ff0119cb640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7feff80a32d0 0x7feff80a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0080096c0 tx=0x7ff008031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.180+0000 7ff002ffd640 1 -- 172.21.15.16:0/2123404529 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff004014030 con 0x7ff00c10e480 2024-09-15T22:37:01.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.351+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff00c10f640 con 0x7feff80a32d0 2024-09-15T22:37:01.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.360+0000 7ff002ffd640 1 -- 172.21.15.16:0/2123404529 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340335 (secure 0 0 0) 0x7ff00c10f640 con 0x7feff80a32d0 2024-09-15T22:37:01.365 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:01.367 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:37:01.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.363+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7feff80a32d0 msgr2=0x7feff80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.363+0000 7ff013c2f640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7feff80a32d0 0x7feff80a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0080096c0 tx=0x7ff008031490 comp rx=0 tx=0).stop 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.363+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c10e480 msgr2=0x7ff00c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.363+0000 7ff013c2f640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c10e480 0x7ff00c19ee80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff00400cce0 tx=0x7ff00400a620 comp rx=0 tx=0).stop 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 shutdown_connections 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7feff80a32d0 0x7feff80a5790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00c10e480 0x7ff00c19ee80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff00c101280 0x7ff00c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 --2- 172.21.15.16:0/2123404529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff00c1008d0 0x7ff00c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 >> 172.21.15.16:0/2123404529 conn(0x7ff00c0fc7a0 msgr2=0x7ff00c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 shutdown_connections 2024-09-15T22:37:01.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:01.364+0000 7ff013c2f640 1 -- 172.21.15.16:0/2123404529 wait complete. 2024-09-15T22:37:02.082 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-15T22:37:00.587649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637950,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":236,"num_read_kb":220,"num_write":336,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412468,"kb_used_data":13300,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124529356,"statfs":{"total":1151940427776,"available":1151518060544,"internally_reserved":0,"allocated":13619200,"data_stored":7590844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40128,"num_objects":3,"num_object_clones":0,"num_object_copies":11,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":77,"num_write":27,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.386523+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:36:44.386523+0000","last_peered":"2024-09-15T22:36:44.386523+0000","last_clean":"2024-09-15T22:36:44.386523+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:36:44.386523+0000","last_undegraded":"2024-09-15T22:36:44.386523+0000","last_fullsized":"2024-09-15T22:36:44.386523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591308+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:36:18.591308+0000","last_peered":"2024-09-15T22:36:18.591308+0000","last_clean":"2024-09-15T22:36:18.591308+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:36:18.591308+0000","last_undegraded":"2024-09-15T22:36:18.591308+0000","last_fullsized":"2024-09-15T22:36:18.591308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295068+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:36:18.295068+0000","last_peered":"2024-09-15T22:36:18.295068+0000","last_clean":"2024-09-15T22:36:18.295068+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:36:18.295068+0000","last_undegraded":"2024-09-15T22:36:18.295068+0000","last_fullsized":"2024-09-15T22:36:18.295068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591852+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:36:18.591852+0000","last_peered":"2024-09-15T22:36:18.591852+0000","last_clean":"2024-09-15T22:36:18.591852+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:36:18.591852+0000","last_undegraded":"2024-09-15T22:36:18.591852+0000","last_fullsized":"2024-09-15T22:36:18.591852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592321+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:36:18.592321+0000","last_peered":"2024-09-15T22:36:18.592321+0000","last_clean":"2024-09-15T22:36:18.592321+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:36:18.592321+0000","last_undegraded":"2024-09-15T22:36:18.592321+0000","last_fullsized":"2024-09-15T22:36:18.592321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591003+0000","last_change":"2024-09-15T22:36:15.292609+0000","last_active":"2024-09-15T22:36:18.591003+0000","last_peered":"2024-09-15T22:36:18.591003+0000","last_clean":"2024-09-15T22:36:18.591003+0000","last_became_active":"2024-09-15T22:36:15.292169+0000","last_became_peered":"2024-09-15T22:36:15.292169+0000","last_unstale":"2024-09-15T22:36:18.591003+0000","last_undegraded":"2024-09-15T22:36:18.591003+0000","last_fullsized":"2024-09-15T22:36:18.591003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:40.449385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.384906+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:36:44.384906+0000","last_peered":"2024-09-15T22:36:44.384906+0000","last_clean":"2024-09-15T22:36:44.384906+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:36:44.384906+0000","last_undegraded":"2024-09-15T22:36:44.384906+0000","last_fullsized":"2024-09-15T22:36:44.384906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358520+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:36:18.358520+0000","last_peered":"2024-09-15T22:36:18.358520+0000","last_clean":"2024-09-15T22:36:18.358520+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:36:18.358520+0000","last_undegraded":"2024-09-15T22:36:18.358520+0000","last_fullsized":"2024-09-15T22:36:18.358520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591207+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:36:18.591207+0000","last_peered":"2024-09-15T22:36:18.591207+0000","last_clean":"2024-09-15T22:36:18.591207+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:36:18.591207+0000","last_undegraded":"2024-09-15T22:36:18.591207+0000","last_fullsized":"2024-09-15T22:36:18.591207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592123+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:36:18.592123+0000","last_peered":"2024-09-15T22:36:18.592123+0000","last_clean":"2024-09-15T22:36:18.592123+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:36:18.592123+0000","last_undegraded":"2024-09-15T22:36:18.592123+0000","last_fullsized":"2024-09-15T22:36:18.592123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592812+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:36:18.592812+0000","last_peered":"2024-09-15T22:36:18.592812+0000","last_clean":"2024-09-15T22:36:18.592812+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:36:18.592812+0000","last_undegraded":"2024-09-15T22:36:18.592812+0000","last_fullsized":"2024-09-15T22:36:18.592812+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591115+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:36:18.591115+0000","last_peered":"2024-09-15T22:36:18.591115+0000","last_clean":"2024-09-15T22:36:18.591115+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:36:18.591115+0000","last_undegraded":"2024-09-15T22:36:18.591115+0000","last_fullsized":"2024-09-15T22:36:18.591115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844691+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:36:19.844691+0000","last_peered":"2024-09-15T22:36:19.844691+0000","last_clean":"2024-09-15T22:36:19.844691+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:36:19.844691+0000","last_undegraded":"2024-09-15T22:36:19.844691+0000","last_fullsized":"2024-09-15T22:36:19.844691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.380970+0000","last_change":"2024-09-15T22:36:06.225236+0000","last_active":"2024-09-15T22:36:43.380970+0000","last_peered":"2024-09-15T22:36:43.380970+0000","last_clean":"2024-09-15T22:36:43.380970+0000","last_became_active":"2024-09-15T22:36:06.224898+0000","last_became_peered":"2024-09-15T22:36:06.224898+0000","last_unstale":"2024-09-15T22:36:43.380970+0000","last_undegraded":"2024-09-15T22:36:43.380970+0000","last_fullsized":"2024-09-15T22:36:43.380970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:06.254724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591604+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:36:18.591604+0000","last_peered":"2024-09-15T22:36:18.591604+0000","last_clean":"2024-09-15T22:36:18.591604+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:36:18.591604+0000","last_undegraded":"2024-09-15T22:36:18.591604+0000","last_fullsized":"2024-09-15T22:36:18.591604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591879+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:36:18.591879+0000","last_peered":"2024-09-15T22:36:18.591879+0000","last_clean":"2024-09-15T22:36:18.591879+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:36:18.591879+0000","last_undegraded":"2024-09-15T22:36:18.591879+0000","last_fullsized":"2024-09-15T22:36:18.591879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294796+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:36:18.294796+0000","last_peered":"2024-09-15T22:36:18.294796+0000","last_clean":"2024-09-15T22:36:18.294796+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:36:18.294796+0000","last_undegraded":"2024-09-15T22:36:18.294796+0000","last_fullsized":"2024-09-15T22:36:18.294796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591548+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:36:18.591548+0000","last_peered":"2024-09-15T22:36:18.591548+0000","last_clean":"2024-09-15T22:36:18.591548+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:36:18.591548+0000","last_undegraded":"2024-09-15T22:36:18.591548+0000","last_fullsized":"2024-09-15T22:36:18.591548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591158+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:36:18.591158+0000","last_peered":"2024-09-15T22:36:18.591158+0000","last_clean":"2024-09-15T22:36:18.591158+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:36:18.591158+0000","last_undegraded":"2024-09-15T22:36:18.591158+0000","last_fullsized":"2024-09-15T22:36:18.591158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.383370+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:36:43.383370+0000","last_peered":"2024-09-15T22:36:43.383370+0000","last_clean":"2024-09-15T22:36:43.383370+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:36:43.383370+0000","last_undegraded":"2024-09-15T22:36:43.383370+0000","last_fullsized":"2024-09-15T22:36:43.383370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593415+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:36:18.593415+0000","last_peered":"2024-09-15T22:36:18.593415+0000","last_clean":"2024-09-15T22:36:18.593415+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:36:18.593415+0000","last_undegraded":"2024-09-15T22:36:18.593415+0000","last_fullsized":"2024-09-15T22:36:18.593415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.590940+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:36:18.590940+0000","last_peered":"2024-09-15T22:36:18.590940+0000","last_clean":"2024-09-15T22:36:18.590940+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:36:18.590940+0000","last_undegraded":"2024-09-15T22:36:18.590940+0000","last_fullsized":"2024-09-15T22:36:18.590940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.379909+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:36:43.379909+0000","last_peered":"2024-09-15T22:36:43.379909+0000","last_clean":"2024-09-15T22:36:43.379909+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:36:43.379909+0000","last_undegraded":"2024-09-15T22:36:43.379909+0000","last_fullsized":"2024-09-15T22:36:43.379909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591655+0000","last_change":"2024-09-15T22:36:15.301702+0000","last_active":"2024-09-15T22:36:18.591655+0000","last_peered":"2024-09-15T22:36:18.591655+0000","last_clean":"2024-09-15T22:36:18.591655+0000","last_became_active":"2024-09-15T22:36:15.301097+0000","last_became_peered":"2024-09-15T22:36:15.301097+0000","last_unstale":"2024-09-15T22:36:18.591655+0000","last_undegraded":"2024-09-15T22:36:18.591655+0000","last_fullsized":"2024-09-15T22:36:18.591655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:46.779257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592525+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:36:18.592525+0000","last_peered":"2024-09-15T22:36:18.592525+0000","last_clean":"2024-09-15T22:36:18.592525+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:36:18.592525+0000","last_undegraded":"2024-09-15T22:36:18.592525+0000","last_fullsized":"2024-09-15T22:36:18.592525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.379604+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:36:43.379604+0000","last_peered":"2024-09-15T22:36:43.379604+0000","last_clean":"2024-09-15T22:36:43.379604+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:36:43.379604+0000","last_undegraded":"2024-09-15T22:36:43.379604+0000","last_fullsized":"2024-09-15T22:36:43.379604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294716+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:36:18.294716+0000","last_peered":"2024-09-15T22:36:18.294716+0000","last_clean":"2024-09-15T22:36:18.294716+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:36:18.294716+0000","last_undegraded":"2024-09-15T22:36:18.294716+0000","last_fullsized":"2024-09-15T22:36:18.294716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591522+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:36:18.591522+0000","last_peered":"2024-09-15T22:36:18.591522+0000","last_clean":"2024-09-15T22:36:18.591522+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:36:18.591522+0000","last_undegraded":"2024-09-15T22:36:18.591522+0000","last_fullsized":"2024-09-15T22:36:18.591522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:55.264774+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:36:55.264774+0000","last_peered":"2024-09-15T22:36:55.264774+0000","last_clean":"2024-09-15T22:36:55.264774+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:36:55.264774+0000","last_undegraded":"2024-09-15T22:36:55.264774+0000","last_fullsized":"2024-09-15T22:36:55.264774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.590975+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:36:18.590975+0000","last_peered":"2024-09-15T22:36:18.590975+0000","last_clean":"2024-09-15T22:36:18.590975+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:36:18.590975+0000","last_undegraded":"2024-09-15T22:36:18.590975+0000","last_fullsized":"2024-09-15T22:36:18.590975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293096+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:36:18.293096+0000","last_peered":"2024-09-15T22:36:18.293096+0000","last_clean":"2024-09-15T22:36:18.293096+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:36:18.293096+0000","last_undegraded":"2024-09-15T22:36:18.293096+0000","last_fullsized":"2024-09-15T22:36:18.293096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.378467+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:36:43.378467+0000","last_peered":"2024-09-15T22:36:43.378467+0000","last_clean":"2024-09-15T22:36:43.378467+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:36:43.378467+0000","last_undegraded":"2024-09-15T22:36:43.378467+0000","last_fullsized":"2024-09-15T22:36:43.378467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591612+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:36:18.591612+0000","last_peered":"2024-09-15T22:36:18.591612+0000","last_clean":"2024-09-15T22:36:18.591612+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:36:18.591612+0000","last_undegraded":"2024-09-15T22:36:18.591612+0000","last_fullsized":"2024-09-15T22:36:18.591612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293076+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:36:18.293076+0000","last_peered":"2024-09-15T22:36:18.293076+0000","last_clean":"2024-09-15T22:36:18.293076+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:36:18.293076+0000","last_undegraded":"2024-09-15T22:36:18.293076+0000","last_fullsized":"2024-09-15T22:36:18.293076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593209+0000","last_change":"2024-09-15T22:36:06.335992+0000","last_active":"2024-09-15T22:36:18.593209+0000","last_peered":"2024-09-15T22:36:18.593209+0000","last_clean":"2024-09-15T22:36:18.593209+0000","last_became_active":"2024-09-15T22:36:06.335660+0000","last_became_peered":"2024-09-15T22:36:06.335660+0000","last_unstale":"2024-09-15T22:36:18.593209+0000","last_undegraded":"2024-09-15T22:36:18.593209+0000","last_fullsized":"2024-09-15T22:36:18.593209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:10:12.935878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591364+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:36:18.591364+0000","last_peered":"2024-09-15T22:36:18.591364+0000","last_clean":"2024-09-15T22:36:18.591364+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:36:18.591364+0000","last_undegraded":"2024-09-15T22:36:18.591364+0000","last_fullsized":"2024-09-15T22:36:18.591364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292881+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:36:18.292881+0000","last_peered":"2024-09-15T22:36:18.292881+0000","last_clean":"2024-09-15T22:36:18.292881+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:36:18.292881+0000","last_undegraded":"2024-09-15T22:36:18.292881+0000","last_fullsized":"2024-09-15T22:36:18.292881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844941+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:36:19.844941+0000","last_peered":"2024-09-15T22:36:19.844941+0000","last_clean":"2024-09-15T22:36:19.844941+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:36:19.844941+0000","last_undegraded":"2024-09-15T22:36:19.844941+0000","last_fullsized":"2024-09-15T22:36:19.844941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292899+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:36:18.292899+0000","last_peered":"2024-09-15T22:36:18.292899+0000","last_clean":"2024-09-15T22:36:18.292899+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:36:18.292899+0000","last_undegraded":"2024-09-15T22:36:18.292899+0000","last_fullsized":"2024-09-15T22:36:18.292899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592422+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:36:18.592422+0000","last_peered":"2024-09-15T22:36:18.592422+0000","last_clean":"2024-09-15T22:36:18.592422+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:36:18.592422+0000","last_undegraded":"2024-09-15T22:36:18.592422+0000","last_fullsized":"2024-09-15T22:36:18.592422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:54.404198+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:36:54.404198+0000","last_peered":"2024-09-15T22:36:54.404198+0000","last_clean":"2024-09-15T22:36:54.404198+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:36:54.404198+0000","last_undegraded":"2024-09-15T22:36:54.404198+0000","last_fullsized":"2024-09-15T22:36:54.404198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292925+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:36:18.292925+0000","last_peered":"2024-09-15T22:36:18.292925+0000","last_clean":"2024-09-15T22:36:18.292925+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:36:18.292925+0000","last_undegraded":"2024-09-15T22:36:18.292925+0000","last_fullsized":"2024-09-15T22:36:18.292925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292237+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:36:18.292237+0000","last_peered":"2024-09-15T22:36:18.292237+0000","last_clean":"2024-09-15T22:36:18.292237+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:36:18.292237+0000","last_undegraded":"2024-09-15T22:36:18.292237+0000","last_fullsized":"2024-09-15T22:36:18.292237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.386813+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:36:44.386813+0000","last_peered":"2024-09-15T22:36:44.386813+0000","last_clean":"2024-09-15T22:36:44.386813+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:36:44.386813+0000","last_undegraded":"2024-09-15T22:36:44.386813+0000","last_fullsized":"2024-09-15T22:36:44.386813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593023+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:36:18.593023+0000","last_peered":"2024-09-15T22:36:18.593023+0000","last_clean":"2024-09-15T22:36:18.593023+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:36:18.593023+0000","last_undegraded":"2024-09-15T22:36:18.593023+0000","last_fullsized":"2024-09-15T22:36:18.593023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591320+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:36:18.591320+0000","last_peered":"2024-09-15T22:36:18.591320+0000","last_clean":"2024-09-15T22:36:18.591320+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:36:18.591320+0000","last_undegraded":"2024-09-15T22:36:18.591320+0000","last_fullsized":"2024-09-15T22:36:18.591320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:45.387207+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:36:45.387207+0000","last_peered":"2024-09-15T22:36:45.387207+0000","last_clean":"2024-09-15T22:36:45.387207+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:36:45.387207+0000","last_undegraded":"2024-09-15T22:36:45.387207+0000","last_fullsized":"2024-09-15T22:36:45.387207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844838+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:36:19.844838+0000","last_peered":"2024-09-15T22:36:19.844838+0000","last_clean":"2024-09-15T22:36:19.844838+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:36:19.844838+0000","last_undegraded":"2024-09-15T22:36:19.844838+0000","last_fullsized":"2024-09-15T22:36:19.844838+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845128+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:36:19.845128+0000","last_peered":"2024-09-15T22:36:19.845128+0000","last_clean":"2024-09-15T22:36:19.845128+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:36:19.845128+0000","last_undegraded":"2024-09-15T22:36:19.845128+0000","last_fullsized":"2024-09-15T22:36:19.845128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845209+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:36:19.845209+0000","last_peered":"2024-09-15T22:36:19.845209+0000","last_clean":"2024-09-15T22:36:19.845209+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:36:19.845209+0000","last_undegraded":"2024-09-15T22:36:19.845209+0000","last_fullsized":"2024-09-15T22:36:19.845209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.385825+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:36:44.385825+0000","last_peered":"2024-09-15T22:36:44.385825+0000","last_clean":"2024-09-15T22:36:44.385825+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:36:44.385825+0000","last_undegraded":"2024-09-15T22:36:44.385825+0000","last_fullsized":"2024-09-15T22:36:44.385825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358561+0000","last_change":"2024-09-15T22:36:15.301170+0000","last_active":"2024-09-15T22:36:18.358561+0000","last_peered":"2024-09-15T22:36:18.358561+0000","last_clean":"2024-09-15T22:36:18.358561+0000","last_became_active":"2024-09-15T22:36:15.300721+0000","last_became_peered":"2024-09-15T22:36:15.300721+0000","last_unstale":"2024-09-15T22:36:18.358561+0000","last_undegraded":"2024-09-15T22:36:18.358561+0000","last_fullsized":"2024-09-15T22:36:18.358561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:25.955835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844985+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:36:19.844985+0000","last_peered":"2024-09-15T22:36:19.844985+0000","last_clean":"2024-09-15T22:36:19.844985+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:36:19.844985+0000","last_undegraded":"2024-09-15T22:36:19.844985+0000","last_fullsized":"2024-09-15T22:36:19.844985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593130+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:36:18.593130+0000","last_peered":"2024-09-15T22:36:18.593130+0000","last_clean":"2024-09-15T22:36:18.593130+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:36:18.593130+0000","last_undegraded":"2024-09-15T22:36:18.593130+0000","last_fullsized":"2024-09-15T22:36:18.593130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591670+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:36:18.591670+0000","last_peered":"2024-09-15T22:36:18.591670+0000","last_clean":"2024-09-15T22:36:18.591670+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:36:18.591670+0000","last_undegraded":"2024-09-15T22:36:18.591670+0000","last_fullsized":"2024-09-15T22:36:18.591670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845197+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:36:19.845197+0000","last_peered":"2024-09-15T22:36:19.845197+0000","last_clean":"2024-09-15T22:36:19.845197+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:36:19.845197+0000","last_undegraded":"2024-09-15T22:36:19.845197+0000","last_fullsized":"2024-09-15T22:36:19.845197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293518+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:36:18.293518+0000","last_peered":"2024-09-15T22:36:18.293518+0000","last_clean":"2024-09-15T22:36:18.293518+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:36:18.293518+0000","last_undegraded":"2024-09-15T22:36:18.293518+0000","last_fullsized":"2024-09-15T22:36:18.293518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592254+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:36:18.592254+0000","last_peered":"2024-09-15T22:36:18.592254+0000","last_clean":"2024-09-15T22:36:18.592254+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:36:18.592254+0000","last_undegraded":"2024-09-15T22:36:18.592254+0000","last_fullsized":"2024-09-15T22:36:18.592254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591981+0000","last_change":"2024-09-15T22:36:08.257268+0000","last_active":"2024-09-15T22:36:18.591981+0000","last_peered":"2024-09-15T22:36:18.591981+0000","last_clean":"2024-09-15T22:36:18.591981+0000","last_became_active":"2024-09-15T22:36:08.256970+0000","last_became_peered":"2024-09-15T22:36:08.256970+0000","last_unstale":"2024-09-15T22:36:18.591981+0000","last_undegraded":"2024-09-15T22:36:18.591981+0000","last_fullsized":"2024-09-15T22:36:18.591981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.481895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:45.388772+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:36:45.388772+0000","last_peered":"2024-09-15T22:36:45.388772+0000","last_clean":"2024-09-15T22:36:45.388772+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:36:45.388772+0000","last_undegraded":"2024-09-15T22:36:45.388772+0000","last_fullsized":"2024-09-15T22:36:45.388772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592457+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:36:18.592457+0000","last_peered":"2024-09-15T22:36:18.592457+0000","last_clean":"2024-09-15T22:36:18.592457+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:36:18.592457+0000","last_undegraded":"2024-09-15T22:36:18.592457+0000","last_fullsized":"2024-09-15T22:36:18.592457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592502+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:36:18.592502+0000","last_peered":"2024-09-15T22:36:18.592502+0000","last_clean":"2024-09-15T22:36:18.592502+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:36:18.592502+0000","last_undegraded":"2024-09-15T22:36:18.592502+0000","last_fullsized":"2024-09-15T22:36:18.592502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.387272+0000","last_change":"2024-09-15T22:36:06.337927+0000","last_active":"2024-09-15T22:36:44.387272+0000","last_peered":"2024-09-15T22:36:44.387272+0000","last_clean":"2024-09-15T22:36:44.387272+0000","last_became_active":"2024-09-15T22:36:06.337478+0000","last_became_peered":"2024-09-15T22:36:06.337478+0000","last_unstale":"2024-09-15T22:36:44.387272+0000","last_undegraded":"2024-09-15T22:36:44.387272+0000","last_fullsized":"2024-09-15T22:36:44.387272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:12.223022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293467+0000","last_change":"2024-09-15T22:36:15.297875+0000","last_active":"2024-09-15T22:36:18.293467+0000","last_peered":"2024-09-15T22:36:18.293467+0000","last_clean":"2024-09-15T22:36:18.293467+0000","last_became_active":"2024-09-15T22:36:15.297339+0000","last_became_peered":"2024-09-15T22:36:15.297339+0000","last_unstale":"2024-09-15T22:36:18.293467+0000","last_undegraded":"2024-09-15T22:36:18.293467+0000","last_fullsized":"2024-09-15T22:36:18.293467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:16.229871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844561+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:36:19.844561+0000","last_peered":"2024-09-15T22:36:19.844561+0000","last_clean":"2024-09-15T22:36:19.844561+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:36:19.844561+0000","last_undegraded":"2024-09-15T22:36:19.844561+0000","last_fullsized":"2024-09-15T22:36:19.844561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:54.404804+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:36:54.404804+0000","last_peered":"2024-09-15T22:36:54.404804+0000","last_clean":"2024-09-15T22:36:54.404804+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:36:54.404804+0000","last_undegraded":"2024-09-15T22:36:54.404804+0000","last_fullsized":"2024-09-15T22:36:54.404804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593462+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:36:18.593462+0000","last_peered":"2024-09-15T22:36:18.593462+0000","last_clean":"2024-09-15T22:36:18.593462+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:36:18.593462+0000","last_undegraded":"2024-09-15T22:36:18.593462+0000","last_fullsized":"2024-09-15T22:36:18.593462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844850+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:36:19.844850+0000","last_peered":"2024-09-15T22:36:19.844850+0000","last_clean":"2024-09-15T22:36:19.844850+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:36:19.844850+0000","last_undegraded":"2024-09-15T22:36:19.844850+0000","last_fullsized":"2024-09-15T22:36:19.844850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.383093+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:36:43.383093+0000","last_peered":"2024-09-15T22:36:43.383093+0000","last_clean":"2024-09-15T22:36:43.383093+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:36:43.383093+0000","last_undegraded":"2024-09-15T22:36:43.383093+0000","last_fullsized":"2024-09-15T22:36:43.383093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591369+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:36:18.591369+0000","last_peered":"2024-09-15T22:36:18.591369+0000","last_clean":"2024-09-15T22:36:18.591369+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:36:18.591369+0000","last_undegraded":"2024-09-15T22:36:18.591369+0000","last_fullsized":"2024-09-15T22:36:18.591369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292529+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:36:18.292529+0000","last_peered":"2024-09-15T22:36:18.292529+0000","last_clean":"2024-09-15T22:36:18.292529+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:36:18.292529+0000","last_undegraded":"2024-09-15T22:36:18.292529+0000","last_fullsized":"2024-09-15T22:36:18.292529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.384752+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:36:44.384752+0000","last_peered":"2024-09-15T22:36:44.384752+0000","last_clean":"2024-09-15T22:36:44.384752+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:36:44.384752+0000","last_undegraded":"2024-09-15T22:36:44.384752+0000","last_fullsized":"2024-09-15T22:36:44.384752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592960+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:36:18.592960+0000","last_peered":"2024-09-15T22:36:18.592960+0000","last_clean":"2024-09-15T22:36:18.592960+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:36:18.592960+0000","last_undegraded":"2024-09-15T22:36:18.592960+0000","last_fullsized":"2024-09-15T22:36:18.592960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591318+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:36:18.591318+0000","last_peered":"2024-09-15T22:36:18.591318+0000","last_clean":"2024-09-15T22:36:18.591318+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:36:18.591318+0000","last_undegraded":"2024-09-15T22:36:18.591318+0000","last_fullsized":"2024-09-15T22:36:18.591318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'25","reported_seq":87,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:57.412011+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:36:57.412011+0000","last_peered":"2024-09-15T22:36:57.412011+0000","last_clean":"2024-09-15T22:36:57.412011+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:36:57.412011+0000","last_undegraded":"2024-09-15T22:36:57.412011+0000","last_fullsized":"2024-09-15T22:36:57.412011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592950+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:36:18.592950+0000","last_peered":"2024-09-15T22:36:18.592950+0000","last_clean":"2024-09-15T22:36:18.592950+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:36:18.592950+0000","last_undegraded":"2024-09-15T22:36:18.592950+0000","last_fullsized":"2024-09-15T22:36:18.592950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358593+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:36:18.358593+0000","last_peered":"2024-09-15T22:36:18.358593+0000","last_clean":"2024-09-15T22:36:18.358593+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:36:18.358593+0000","last_undegraded":"2024-09-15T22:36:18.358593+0000","last_fullsized":"2024-09-15T22:36:18.358593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.381995+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:36:43.381995+0000","last_peered":"2024-09-15T22:36:43.381995+0000","last_clean":"2024-09-15T22:36:43.381995+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:36:43.381995+0000","last_undegraded":"2024-09-15T22:36:43.381995+0000","last_fullsized":"2024-09-15T22:36:43.381995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591582+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:36:18.591582+0000","last_peered":"2024-09-15T22:36:18.591582+0000","last_clean":"2024-09-15T22:36:18.591582+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:36:18.591582+0000","last_undegraded":"2024-09-15T22:36:18.591582+0000","last_fullsized":"2024-09-15T22:36:18.591582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592430+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:36:18.592430+0000","last_peered":"2024-09-15T22:36:18.592430+0000","last_clean":"2024-09-15T22:36:18.592430+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:36:18.592430+0000","last_undegraded":"2024-09-15T22:36:18.592430+0000","last_fullsized":"2024-09-15T22:36:18.592430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.383004+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:36:43.383004+0000","last_peered":"2024-09-15T22:36:43.383004+0000","last_clean":"2024-09-15T22:36:43.383004+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:36:43.383004+0000","last_undegraded":"2024-09-15T22:36:43.383004+0000","last_fullsized":"2024-09-15T22:36:43.383004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592339+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:36:18.592339+0000","last_peered":"2024-09-15T22:36:18.592339+0000","last_clean":"2024-09-15T22:36:18.592339+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:36:18.592339+0000","last_undegraded":"2024-09-15T22:36:18.592339+0000","last_fullsized":"2024-09-15T22:36:18.592339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358888+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:36:18.358888+0000","last_peered":"2024-09-15T22:36:18.358888+0000","last_clean":"2024-09-15T22:36:18.358888+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:36:18.358888+0000","last_undegraded":"2024-09-15T22:36:18.358888+0000","last_fullsized":"2024-09-15T22:36:18.358888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:56.405889+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:36:56.405889+0000","last_peered":"2024-09-15T22:36:56.405889+0000","last_clean":"2024-09-15T22:36:56.405889+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:36:56.405889+0000","last_undegraded":"2024-09-15T22:36:56.405889+0000","last_fullsized":"2024-09-15T22:36:56.405889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591308+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:36:18.591308+0000","last_peered":"2024-09-15T22:36:18.591308+0000","last_clean":"2024-09-15T22:36:18.591308+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:36:18.591308+0000","last_undegraded":"2024-09-15T22:36:18.591308+0000","last_fullsized":"2024-09-15T22:36:18.591308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592297+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:36:18.592297+0000","last_peered":"2024-09-15T22:36:18.592297+0000","last_clean":"2024-09-15T22:36:18.592297+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:36:18.592297+0000","last_undegraded":"2024-09-15T22:36:18.592297+0000","last_fullsized":"2024-09-15T22:36:18.592297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592495+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:36:18.592495+0000","last_peered":"2024-09-15T22:36:18.592495+0000","last_clean":"2024-09-15T22:36:18.592495+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:36:18.592495+0000","last_undegraded":"2024-09-15T22:36:18.592495+0000","last_fullsized":"2024-09-15T22:36:18.592495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592665+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:36:18.592665+0000","last_peered":"2024-09-15T22:36:18.592665+0000","last_clean":"2024-09-15T22:36:18.592665+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:36:18.592665+0000","last_undegraded":"2024-09-15T22:36:18.592665+0000","last_fullsized":"2024-09-15T22:36:18.592665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591525+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:36:18.591525+0000","last_peered":"2024-09-15T22:36:18.591525+0000","last_clean":"2024-09-15T22:36:18.591525+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:36:18.591525+0000","last_undegraded":"2024-09-15T22:36:18.591525+0000","last_fullsized":"2024-09-15T22:36:18.591525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591108+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:36:18.591108+0000","last_peered":"2024-09-15T22:36:18.591108+0000","last_clean":"2024-09-15T22:36:18.591108+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:36:18.591108+0000","last_undegraded":"2024-09-15T22:36:18.591108+0000","last_fullsized":"2024-09-15T22:36:18.591108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.385491+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:36:44.385491+0000","last_peered":"2024-09-15T22:36:44.385491+0000","last_clean":"2024-09-15T22:36:44.385491+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:36:44.385491+0000","last_undegraded":"2024-09-15T22:36:44.385491+0000","last_fullsized":"2024-09-15T22:36:44.385491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.384457+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:36:44.384457+0000","last_peered":"2024-09-15T22:36:44.384457+0000","last_clean":"2024-09-15T22:36:44.384457+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:36:44.384457+0000","last_undegraded":"2024-09-15T22:36:44.384457+0000","last_fullsized":"2024-09-15T22:36:44.384457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592640+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:36:18.592640+0000","last_peered":"2024-09-15T22:36:18.592640+0000","last_clean":"2024-09-15T22:36:18.592640+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:36:18.592640+0000","last_undegraded":"2024-09-15T22:36:18.592640+0000","last_fullsized":"2024-09-15T22:36:18.592640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294883+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:36:18.294883+0000","last_peered":"2024-09-15T22:36:18.294883+0000","last_clean":"2024-09-15T22:36:18.294883+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:36:18.294883+0000","last_undegraded":"2024-09-15T22:36:18.294883+0000","last_fullsized":"2024-09-15T22:36:18.294883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:57.407594+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:36:57.407594+0000","last_peered":"2024-09-15T22:36:57.407594+0000","last_clean":"2024-09-15T22:36:57.407594+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:36:57.407594+0000","last_undegraded":"2024-09-15T22:36:57.407594+0000","last_fullsized":"2024-09-15T22:36:57.407594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591540+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:36:18.591540+0000","last_peered":"2024-09-15T22:36:18.591540+0000","last_clean":"2024-09-15T22:36:18.591540+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:36:18.591540+0000","last_undegraded":"2024-09-15T22:36:18.591540+0000","last_fullsized":"2024-09-15T22:36:18.591540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295048+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:36:18.295048+0000","last_peered":"2024-09-15T22:36:18.295048+0000","last_clean":"2024-09-15T22:36:18.295048+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:36:18.295048+0000","last_undegraded":"2024-09-15T22:36:18.295048+0000","last_fullsized":"2024-09-15T22:36:18.295048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591113+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:36:18.591113+0000","last_peered":"2024-09-15T22:36:18.591113+0000","last_clean":"2024-09-15T22:36:18.591113+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:36:18.591113+0000","last_undegraded":"2024-09-15T22:36:18.591113+0000","last_fullsized":"2024-09-15T22:36:18.591113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295384+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:36:18.295384+0000","last_peered":"2024-09-15T22:36:18.295384+0000","last_clean":"2024-09-15T22:36:18.295384+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:36:18.295384+0000","last_undegraded":"2024-09-15T22:36:18.295384+0000","last_fullsized":"2024-09-15T22:36:18.295384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294676+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:36:18.294676+0000","last_peered":"2024-09-15T22:36:18.294676+0000","last_clean":"2024-09-15T22:36:18.294676+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:36:18.294676+0000","last_undegraded":"2024-09-15T22:36:18.294676+0000","last_fullsized":"2024-09-15T22:36:18.294676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591741+0000","last_change":"2024-09-15T22:36:08.250464+0000","last_active":"2024-09-15T22:36:18.591741+0000","last_peered":"2024-09-15T22:36:18.591741+0000","last_clean":"2024-09-15T22:36:18.591741+0000","last_became_active":"2024-09-15T22:36:08.250134+0000","last_became_peered":"2024-09-15T22:36:08.250134+0000","last_unstale":"2024-09-15T22:36:18.591741+0000","last_undegraded":"2024-09-15T22:36:18.591741+0000","last_fullsized":"2024-09-15T22:36:18.591741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:41.969151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293025+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:36:18.293025+0000","last_peered":"2024-09-15T22:36:18.293025+0000","last_clean":"2024-09-15T22:36:18.293025+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:36:18.293025+0000","last_undegraded":"2024-09-15T22:36:18.293025+0000","last_fullsized":"2024-09-15T22:36:18.293025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591637+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:36:18.591637+0000","last_peered":"2024-09-15T22:36:18.591637+0000","last_clean":"2024-09-15T22:36:18.591637+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:36:18.591637+0000","last_undegraded":"2024-09-15T22:36:18.591637+0000","last_fullsized":"2024-09-15T22:36:18.591637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295210+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:36:18.295210+0000","last_peered":"2024-09-15T22:36:18.295210+0000","last_clean":"2024-09-15T22:36:18.295210+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:36:18.295210+0000","last_undegraded":"2024-09-15T22:36:18.295210+0000","last_fullsized":"2024-09-15T22:36:18.295210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591394+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:36:18.591394+0000","last_peered":"2024-09-15T22:36:18.591394+0000","last_clean":"2024-09-15T22:36:18.591394+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:36:18.591394+0000","last_undegraded":"2024-09-15T22:36:18.591394+0000","last_fullsized":"2024-09-15T22:36:18.591394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592341+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:36:18.592341+0000","last_peered":"2024-09-15T22:36:18.592341+0000","last_clean":"2024-09-15T22:36:18.592341+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:36:18.592341+0000","last_undegraded":"2024-09-15T22:36:18.592341+0000","last_fullsized":"2024-09-15T22:36:18.592341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591538+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:36:18.591538+0000","last_peered":"2024-09-15T22:36:18.591538+0000","last_clean":"2024-09-15T22:36:18.591538+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:36:18.591538+0000","last_undegraded":"2024-09-15T22:36:18.591538+0000","last_fullsized":"2024-09-15T22:36:18.591538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358589+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:36:18.358589+0000","last_peered":"2024-09-15T22:36:18.358589+0000","last_clean":"2024-09-15T22:36:18.358589+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:36:18.358589+0000","last_undegraded":"2024-09-15T22:36:18.358589+0000","last_fullsized":"2024-09-15T22:36:18.358589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292689+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:36:18.292689+0000","last_peered":"2024-09-15T22:36:18.292689+0000","last_clean":"2024-09-15T22:36:18.292689+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:36:18.292689+0000","last_undegraded":"2024-09-15T22:36:18.292689+0000","last_fullsized":"2024-09-15T22:36:18.292689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591322+0000","last_change":"2024-09-15T22:36:08.262314+0000","last_active":"2024-09-15T22:36:18.591322+0000","last_peered":"2024-09-15T22:36:18.591322+0000","last_clean":"2024-09-15T22:36:18.591322+0000","last_became_active":"2024-09-15T22:36:08.261958+0000","last_became_peered":"2024-09-15T22:36:08.261958+0000","last_unstale":"2024-09-15T22:36:18.591322+0000","last_undegraded":"2024-09-15T22:36:18.591322+0000","last_fullsized":"2024-09-15T22:36:18.591322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:25.822697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295177+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:36:18.295177+0000","last_peered":"2024-09-15T22:36:18.295177+0000","last_clean":"2024-09-15T22:36:18.295177+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:36:18.295177+0000","last_undegraded":"2024-09-15T22:36:18.295177+0000","last_fullsized":"2024-09-15T22:36:18.295177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593326+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:36:18.593326+0000","last_peered":"2024-09-15T22:36:18.593326+0000","last_clean":"2024-09-15T22:36:18.593326+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:36:18.593326+0000","last_undegraded":"2024-09-15T22:36:18.593326+0000","last_fullsized":"2024-09-15T22:36:18.593326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591885+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:36:18.591885+0000","last_peered":"2024-09-15T22:36:18.591885+0000","last_clean":"2024-09-15T22:36:18.591885+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:36:18.591885+0000","last_undegraded":"2024-09-15T22:36:18.591885+0000","last_fullsized":"2024-09-15T22:36:18.591885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358212+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:36:18.358212+0000","last_peered":"2024-09-15T22:36:18.358212+0000","last_clean":"2024-09-15T22:36:18.358212+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:36:18.358212+0000","last_undegraded":"2024-09-15T22:36:18.358212+0000","last_fullsized":"2024-09-15T22:36:18.358212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591334+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:36:18.591334+0000","last_peered":"2024-09-15T22:36:18.591334+0000","last_clean":"2024-09-15T22:36:18.591334+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:36:18.591334+0000","last_undegraded":"2024-09-15T22:36:18.591334+0000","last_fullsized":"2024-09-15T22:36:18.591334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591525+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:36:18.591525+0000","last_peered":"2024-09-15T22:36:18.591525+0000","last_clean":"2024-09-15T22:36:18.591525+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:36:18.591525+0000","last_undegraded":"2024-09-15T22:36:18.591525+0000","last_fullsized":"2024-09-15T22:36:18.591525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292680+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:36:18.292680+0000","last_peered":"2024-09-15T22:36:18.292680+0000","last_clean":"2024-09-15T22:36:18.292680+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:36:18.292680+0000","last_undegraded":"2024-09-15T22:36:18.292680+0000","last_fullsized":"2024-09-15T22:36:18.292680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591615+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:36:18.591615+0000","last_peered":"2024-09-15T22:36:18.591615+0000","last_clean":"2024-09-15T22:36:18.591615+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:36:18.591615+0000","last_undegraded":"2024-09-15T22:36:18.591615+0000","last_fullsized":"2024-09-15T22:36:18.591615+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845130+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:36:19.845130+0000","last_peered":"2024-09-15T22:36:19.845130+0000","last_clean":"2024-09-15T22:36:19.845130+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:36:19.845130+0000","last_undegraded":"2024-09-15T22:36:19.845130+0000","last_fullsized":"2024-09-15T22:36:19.845130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293329+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:36:18.293329+0000","last_peered":"2024-09-15T22:36:18.293329+0000","last_clean":"2024-09-15T22:36:18.293329+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:36:18.293329+0000","last_undegraded":"2024-09-15T22:36:18.293329+0000","last_fullsized":"2024-09-15T22:36:18.293329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358968+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:36:18.358968+0000","last_peered":"2024-09-15T22:36:18.358968+0000","last_clean":"2024-09-15T22:36:18.358968+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:36:18.358968+0000","last_undegraded":"2024-09-15T22:36:18.358968+0000","last_fullsized":"2024-09-15T22:36:18.358968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592040+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:36:18.592040+0000","last_peered":"2024-09-15T22:36:18.592040+0000","last_clean":"2024-09-15T22:36:18.592040+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:36:18.592040+0000","last_undegraded":"2024-09-15T22:36:18.592040+0000","last_fullsized":"2024-09-15T22:36:18.592040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844646+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:36:19.844646+0000","last_peered":"2024-09-15T22:36:19.844646+0000","last_clean":"2024-09-15T22:36:19.844646+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:36:19.844646+0000","last_undegraded":"2024-09-15T22:36:19.844646+0000","last_fullsized":"2024-09-15T22:36:19.844646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292852+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:36:18.292852+0000","last_peered":"2024-09-15T22:36:18.292852+0000","last_clean":"2024-09-15T22:36:18.292852+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:36:18.292852+0000","last_undegraded":"2024-09-15T22:36:18.292852+0000","last_fullsized":"2024-09-15T22:36:18.292852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591023+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:36:18.591023+0000","last_peered":"2024-09-15T22:36:18.591023+0000","last_clean":"2024-09-15T22:36:18.591023+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:36:18.591023+0000","last_undegraded":"2024-09-15T22:36:18.591023+0000","last_fullsized":"2024-09-15T22:36:18.591023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591483+0000","last_change":"2024-09-15T22:36:08.252626+0000","last_active":"2024-09-15T22:36:18.591483+0000","last_peered":"2024-09-15T22:36:18.591483+0000","last_clean":"2024-09-15T22:36:18.591483+0000","last_became_active":"2024-09-15T22:36:08.252196+0000","last_became_peered":"2024-09-15T22:36:08.252196+0000","last_unstale":"2024-09-15T22:36:18.591483+0000","last_undegraded":"2024-09-15T22:36:18.591483+0000","last_fullsized":"2024-09-15T22:36:18.591483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:31.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592744+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:36:18.592744+0000","last_peered":"2024-09-15T22:36:18.592744+0000","last_clean":"2024-09-15T22:36:18.592744+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:36:18.592744+0000","last_undegraded":"2024-09-15T22:36:18.592744+0000","last_fullsized":"2024-09-15T22:36:18.592744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592414+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:36:18.592414+0000","last_peered":"2024-09-15T22:36:18.592414+0000","last_clean":"2024-09-15T22:36:18.592414+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:36:18.592414+0000","last_undegraded":"2024-09-15T22:36:18.592414+0000","last_fullsized":"2024-09-15T22:36:18.592414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591781+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:36:18.591781+0000","last_peered":"2024-09-15T22:36:18.591781+0000","last_clean":"2024-09-15T22:36:18.591781+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:36:18.591781+0000","last_undegraded":"2024-09-15T22:36:18.591781+0000","last_fullsized":"2024-09-15T22:36:18.591781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":100,"num_read_kb":101,"num_write":153,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":172676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612630,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":472536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841563,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1103149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005089,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":476868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":472692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462955,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1568768,"data_stored":1068603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626480,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":467948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855413,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1064191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986234,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":472598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":468142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":528262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":526519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":469336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:37:02.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:37:02.353 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:02 smithi059 ceph-mon[34872]: pgmap v310: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-15T22:37:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:02 smithi059 ceph-mon[34872]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:02.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:02 smithi016 ceph-mon[31492]: pgmap v310: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-15T22:37:02.735 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:02 smithi016 ceph-mon[31492]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:02.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:02 smithi130 ceph-mon[35742]: pgmap v310: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-15T22:37:02.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:02 smithi130 ceph-mon[35742]: from='client.14829 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 -- 172.21.15.16:0/2483461685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 msgr2=0x7efebc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 --2- 172.21.15.16:0/2483461685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1029b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efea8009900 tx=0x7efea802a480 comp rx=0 tx=0).stop 2024-09-15T22:37:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 -- 172.21.15.16:0/2483461685 shutdown_connections 2024-09-15T22:37:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 --2- 172.21.15.16:0/2483461685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efebc102ef0 0x7efebc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 --2- 172.21.15.16:0/2483461685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1029b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 --2- 172.21.15.16:0/2483461685 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efebc100e20 0x7efebc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 -- 172.21.15.16:0/2483461685 >> 172.21.15.16:0/2483461685 conn(0x7efebc0fc7d0 msgr2=0x7efebc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 -- 172.21.15.16:0/2483461685 shutdown_connections 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 -- 172.21.15.16:0/2483461685 wait complete. 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.817+0000 7efec2aed640 1 Processor -- start 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec2aed640 1 -- start start 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec2aed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efebc100e20 0x7efebc199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec2aed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec2aed640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efebc102ef0 0x7efebc1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec2aed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efebc110f80 con 0x7efebc102570 2024-09-15T22:37:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec2aed640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efebc110e00 con 0x7efebc102ef0 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec2aed640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efebc111100 con 0x7efebc100e20 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38806/0 (socket says 172.21.15.16:38806) 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 -- 172.21.15.16:0/2740148770 learned_addr learned my addr 172.21.15.16:0/2740148770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec0889640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efebc100e20 0x7efebc199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec108a640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efebc102ef0 0x7efebc1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 -- 172.21.15.16:0/2740148770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efebc100e20 msgr2=0x7efebc199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efebc100e20 0x7efebc199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 -- 172.21.15.16:0/2740148770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efebc102ef0 msgr2=0x7efebc1a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efebc102ef0 0x7efebc1a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efebbfff640 1 -- 172.21.15.16:0/2740148770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1a4950 con 0x7efebc102570 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.818+0000 7efec108a640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efebc102ef0 0x7efebc1a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.819+0000 7efebbfff640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1998d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efea8006010 tx=0x7efea8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.819+0000 7efeb9ffb640 1 -- 172.21.15.16:0/2740148770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea803f070 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.819+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc1a4b20 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.819+0000 7efeb9ffb640 1 -- 172.21.15.16:0/2740148770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efea8031c30 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.819+0000 7efeb9ffb640 1 -- 172.21.15.16:0/2740148770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8035030 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.819+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1a4f70 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.820+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc101200 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.821+0000 7efeb9ffb640 1 -- 172.21.15.16:0/2740148770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efea8042070 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.822+0000 7efeb9ffb640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efeac0a32d0 0x7efeac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.822+0000 7efeb9ffb640 1 -- 172.21.15.16:0/2740148770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7efea80f0500 con 0x7efebc102570 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.822+0000 7efec0889640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efeac0a32d0 0x7efeac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.822+0000 7efec0889640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efeac0a32d0 0x7efeac0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009b60 tx=0x7efeb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:02.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.826+0000 7efeb9ffb640 1 -- 172.21.15.16:0/2740148770 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efea8031de0 con 0x7efebc102570 2024-09-15T22:37:02.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:02.997+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efebc062630 con 0x7efeac0a32d0 2024-09-15T22:37:03.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.006+0000 7efeb9ffb640 1 -- 172.21.15.16:0/2740148770 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340335 (secure 0 0 0) 0x7efebc062630 con 0x7efeac0a32d0 2024-09-15T22:37:03.011 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:03.013 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efeac0a32d0 msgr2=0x7efeac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efeac0a32d0 0x7efeac0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009b60 tx=0x7efeb0005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 msgr2=0x7efebc1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1998d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efea8006010 tx=0x7efea8003c10 comp rx=0 tx=0).stop 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 shutdown_connections 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7efeac0a32d0 0x7efeac0a5790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efebc102ef0 0x7efebc1a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efebc102570 0x7efebc1998d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 --2- 172.21.15.16:0/2740148770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efebc100e20 0x7efebc199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 >> 172.21.15.16:0/2740148770 conn(0x7efebc0fc7d0 msgr2=0x7efebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.010+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 shutdown_connections 2024-09-15T22:37:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:03.011+0000 7efec2aed640 1 -- 172.21.15.16:0/2740148770 wait complete. 2024-09-15T22:37:03.826 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-15T22:37:02.588185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637950,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":246,"num_read_kb":225,"num_write":336,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412468,"kb_used_data":13300,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124529356,"statfs":{"total":1151940427776,"available":1151518060544,"internally_reserved":0,"allocated":13619200,"data_stored":7590844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39735,"num_objects":3,"num_object_clones":0,"num_object_copies":11,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":76,"num_write":26,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003253"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.386523+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:36:44.386523+0000","last_peered":"2024-09-15T22:36:44.386523+0000","last_clean":"2024-09-15T22:36:44.386523+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:36:44.386523+0000","last_undegraded":"2024-09-15T22:36:44.386523+0000","last_fullsized":"2024-09-15T22:36:44.386523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591308+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:36:18.591308+0000","last_peered":"2024-09-15T22:36:18.591308+0000","last_clean":"2024-09-15T22:36:18.591308+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:36:18.591308+0000","last_undegraded":"2024-09-15T22:36:18.591308+0000","last_fullsized":"2024-09-15T22:36:18.591308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295068+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:36:18.295068+0000","last_peered":"2024-09-15T22:36:18.295068+0000","last_clean":"2024-09-15T22:36:18.295068+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:36:18.295068+0000","last_undegraded":"2024-09-15T22:36:18.295068+0000","last_fullsized":"2024-09-15T22:36:18.295068+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591852+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:36:18.591852+0000","last_peered":"2024-09-15T22:36:18.591852+0000","last_clean":"2024-09-15T22:36:18.591852+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:36:18.591852+0000","last_undegraded":"2024-09-15T22:36:18.591852+0000","last_fullsized":"2024-09-15T22:36:18.591852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592321+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:36:18.592321+0000","last_peered":"2024-09-15T22:36:18.592321+0000","last_clean":"2024-09-15T22:36:18.592321+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:36:18.592321+0000","last_undegraded":"2024-09-15T22:36:18.592321+0000","last_fullsized":"2024-09-15T22:36:18.592321+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591003+0000","last_change":"2024-09-15T22:36:15.292609+0000","last_active":"2024-09-15T22:36:18.591003+0000","last_peered":"2024-09-15T22:36:18.591003+0000","last_clean":"2024-09-15T22:36:18.591003+0000","last_became_active":"2024-09-15T22:36:15.292169+0000","last_became_peered":"2024-09-15T22:36:15.292169+0000","last_unstale":"2024-09-15T22:36:18.591003+0000","last_undegraded":"2024-09-15T22:36:18.591003+0000","last_fullsized":"2024-09-15T22:36:18.591003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:58:40.449385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.384906+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:36:44.384906+0000","last_peered":"2024-09-15T22:36:44.384906+0000","last_clean":"2024-09-15T22:36:44.384906+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:36:44.384906+0000","last_undegraded":"2024-09-15T22:36:44.384906+0000","last_fullsized":"2024-09-15T22:36:44.384906+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358520+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:36:18.358520+0000","last_peered":"2024-09-15T22:36:18.358520+0000","last_clean":"2024-09-15T22:36:18.358520+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:36:18.358520+0000","last_undegraded":"2024-09-15T22:36:18.358520+0000","last_fullsized":"2024-09-15T22:36:18.358520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591207+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:36:18.591207+0000","last_peered":"2024-09-15T22:36:18.591207+0000","last_clean":"2024-09-15T22:36:18.591207+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:36:18.591207+0000","last_undegraded":"2024-09-15T22:36:18.591207+0000","last_fullsized":"2024-09-15T22:36:18.591207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592123+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:36:18.592123+0000","last_peered":"2024-09-15T22:36:18.592123+0000","last_clean":"2024-09-15T22:36:18.592123+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:36:18.592123+0000","last_undegraded":"2024-09-15T22:36:18.592123+0000","last_fullsized":"2024-09-15T22:36:18.592123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592812+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:36:18.592812+0000","last_peered":"2024-09-15T22:36:18.592812+0000","last_clean":"2024-09-15T22:36:18.592812+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:36:18.592812+0000","last_undegraded":"2024-09-15T22:36:18.592812+0000","last_fullsized":"2024-09-15T22:36:18.592812+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591115+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:36:18.591115+0000","last_peered":"2024-09-15T22:36:18.591115+0000","last_clean":"2024-09-15T22:36:18.591115+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:36:18.591115+0000","last_undegraded":"2024-09-15T22:36:18.591115+0000","last_fullsized":"2024-09-15T22:36:18.591115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844691+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:36:19.844691+0000","last_peered":"2024-09-15T22:36:19.844691+0000","last_clean":"2024-09-15T22:36:19.844691+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:36:19.844691+0000","last_undegraded":"2024-09-15T22:36:19.844691+0000","last_fullsized":"2024-09-15T22:36:19.844691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.380970+0000","last_change":"2024-09-15T22:36:06.225236+0000","last_active":"2024-09-15T22:36:43.380970+0000","last_peered":"2024-09-15T22:36:43.380970+0000","last_clean":"2024-09-15T22:36:43.380970+0000","last_became_active":"2024-09-15T22:36:06.224898+0000","last_became_peered":"2024-09-15T22:36:06.224898+0000","last_unstale":"2024-09-15T22:36:43.380970+0000","last_undegraded":"2024-09-15T22:36:43.380970+0000","last_fullsized":"2024-09-15T22:36:43.380970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:06.254724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591604+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:36:18.591604+0000","last_peered":"2024-09-15T22:36:18.591604+0000","last_clean":"2024-09-15T22:36:18.591604+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:36:18.591604+0000","last_undegraded":"2024-09-15T22:36:18.591604+0000","last_fullsized":"2024-09-15T22:36:18.591604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591879+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:36:18.591879+0000","last_peered":"2024-09-15T22:36:18.591879+0000","last_clean":"2024-09-15T22:36:18.591879+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:36:18.591879+0000","last_undegraded":"2024-09-15T22:36:18.591879+0000","last_fullsized":"2024-09-15T22:36:18.591879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294796+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:36:18.294796+0000","last_peered":"2024-09-15T22:36:18.294796+0000","last_clean":"2024-09-15T22:36:18.294796+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:36:18.294796+0000","last_undegraded":"2024-09-15T22:36:18.294796+0000","last_fullsized":"2024-09-15T22:36:18.294796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591548+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:36:18.591548+0000","last_peered":"2024-09-15T22:36:18.591548+0000","last_clean":"2024-09-15T22:36:18.591548+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:36:18.591548+0000","last_undegraded":"2024-09-15T22:36:18.591548+0000","last_fullsized":"2024-09-15T22:36:18.591548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591158+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:36:18.591158+0000","last_peered":"2024-09-15T22:36:18.591158+0000","last_clean":"2024-09-15T22:36:18.591158+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:36:18.591158+0000","last_undegraded":"2024-09-15T22:36:18.591158+0000","last_fullsized":"2024-09-15T22:36:18.591158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.383370+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:36:43.383370+0000","last_peered":"2024-09-15T22:36:43.383370+0000","last_clean":"2024-09-15T22:36:43.383370+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:36:43.383370+0000","last_undegraded":"2024-09-15T22:36:43.383370+0000","last_fullsized":"2024-09-15T22:36:43.383370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593415+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:36:18.593415+0000","last_peered":"2024-09-15T22:36:18.593415+0000","last_clean":"2024-09-15T22:36:18.593415+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:36:18.593415+0000","last_undegraded":"2024-09-15T22:36:18.593415+0000","last_fullsized":"2024-09-15T22:36:18.593415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.590940+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:36:18.590940+0000","last_peered":"2024-09-15T22:36:18.590940+0000","last_clean":"2024-09-15T22:36:18.590940+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:36:18.590940+0000","last_undegraded":"2024-09-15T22:36:18.590940+0000","last_fullsized":"2024-09-15T22:36:18.590940+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.379909+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:36:43.379909+0000","last_peered":"2024-09-15T22:36:43.379909+0000","last_clean":"2024-09-15T22:36:43.379909+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:36:43.379909+0000","last_undegraded":"2024-09-15T22:36:43.379909+0000","last_fullsized":"2024-09-15T22:36:43.379909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591655+0000","last_change":"2024-09-15T22:36:15.301702+0000","last_active":"2024-09-15T22:36:18.591655+0000","last_peered":"2024-09-15T22:36:18.591655+0000","last_clean":"2024-09-15T22:36:18.591655+0000","last_became_active":"2024-09-15T22:36:15.301097+0000","last_became_peered":"2024-09-15T22:36:15.301097+0000","last_unstale":"2024-09-15T22:36:18.591655+0000","last_undegraded":"2024-09-15T22:36:18.591655+0000","last_fullsized":"2024-09-15T22:36:18.591655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:46.779257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592525+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:36:18.592525+0000","last_peered":"2024-09-15T22:36:18.592525+0000","last_clean":"2024-09-15T22:36:18.592525+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:36:18.592525+0000","last_undegraded":"2024-09-15T22:36:18.592525+0000","last_fullsized":"2024-09-15T22:36:18.592525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.379604+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:36:43.379604+0000","last_peered":"2024-09-15T22:36:43.379604+0000","last_clean":"2024-09-15T22:36:43.379604+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:36:43.379604+0000","last_undegraded":"2024-09-15T22:36:43.379604+0000","last_fullsized":"2024-09-15T22:36:43.379604+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294716+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:36:18.294716+0000","last_peered":"2024-09-15T22:36:18.294716+0000","last_clean":"2024-09-15T22:36:18.294716+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:36:18.294716+0000","last_undegraded":"2024-09-15T22:36:18.294716+0000","last_fullsized":"2024-09-15T22:36:18.294716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591522+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:36:18.591522+0000","last_peered":"2024-09-15T22:36:18.591522+0000","last_clean":"2024-09-15T22:36:18.591522+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:36:18.591522+0000","last_undegraded":"2024-09-15T22:36:18.591522+0000","last_fullsized":"2024-09-15T22:36:18.591522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:55.264774+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:36:55.264774+0000","last_peered":"2024-09-15T22:36:55.264774+0000","last_clean":"2024-09-15T22:36:55.264774+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:36:55.264774+0000","last_undegraded":"2024-09-15T22:36:55.264774+0000","last_fullsized":"2024-09-15T22:36:55.264774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.590975+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:36:18.590975+0000","last_peered":"2024-09-15T22:36:18.590975+0000","last_clean":"2024-09-15T22:36:18.590975+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:36:18.590975+0000","last_undegraded":"2024-09-15T22:36:18.590975+0000","last_fullsized":"2024-09-15T22:36:18.590975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293096+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:36:18.293096+0000","last_peered":"2024-09-15T22:36:18.293096+0000","last_clean":"2024-09-15T22:36:18.293096+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:36:18.293096+0000","last_undegraded":"2024-09-15T22:36:18.293096+0000","last_fullsized":"2024-09-15T22:36:18.293096+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.378467+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:36:43.378467+0000","last_peered":"2024-09-15T22:36:43.378467+0000","last_clean":"2024-09-15T22:36:43.378467+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:36:43.378467+0000","last_undegraded":"2024-09-15T22:36:43.378467+0000","last_fullsized":"2024-09-15T22:36:43.378467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591612+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:36:18.591612+0000","last_peered":"2024-09-15T22:36:18.591612+0000","last_clean":"2024-09-15T22:36:18.591612+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:36:18.591612+0000","last_undegraded":"2024-09-15T22:36:18.591612+0000","last_fullsized":"2024-09-15T22:36:18.591612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293076+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:36:18.293076+0000","last_peered":"2024-09-15T22:36:18.293076+0000","last_clean":"2024-09-15T22:36:18.293076+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:36:18.293076+0000","last_undegraded":"2024-09-15T22:36:18.293076+0000","last_fullsized":"2024-09-15T22:36:18.293076+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593209+0000","last_change":"2024-09-15T22:36:06.335992+0000","last_active":"2024-09-15T22:36:18.593209+0000","last_peered":"2024-09-15T22:36:18.593209+0000","last_clean":"2024-09-15T22:36:18.593209+0000","last_became_active":"2024-09-15T22:36:06.335660+0000","last_became_peered":"2024-09-15T22:36:06.335660+0000","last_unstale":"2024-09-15T22:36:18.593209+0000","last_undegraded":"2024-09-15T22:36:18.593209+0000","last_fullsized":"2024-09-15T22:36:18.593209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:10:12.935878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591364+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:36:18.591364+0000","last_peered":"2024-09-15T22:36:18.591364+0000","last_clean":"2024-09-15T22:36:18.591364+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:36:18.591364+0000","last_undegraded":"2024-09-15T22:36:18.591364+0000","last_fullsized":"2024-09-15T22:36:18.591364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292881+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:36:18.292881+0000","last_peered":"2024-09-15T22:36:18.292881+0000","last_clean":"2024-09-15T22:36:18.292881+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:36:18.292881+0000","last_undegraded":"2024-09-15T22:36:18.292881+0000","last_fullsized":"2024-09-15T22:36:18.292881+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844941+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:36:19.844941+0000","last_peered":"2024-09-15T22:36:19.844941+0000","last_clean":"2024-09-15T22:36:19.844941+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:36:19.844941+0000","last_undegraded":"2024-09-15T22:36:19.844941+0000","last_fullsized":"2024-09-15T22:36:19.844941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292899+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:36:18.292899+0000","last_peered":"2024-09-15T22:36:18.292899+0000","last_clean":"2024-09-15T22:36:18.292899+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:36:18.292899+0000","last_undegraded":"2024-09-15T22:36:18.292899+0000","last_fullsized":"2024-09-15T22:36:18.292899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592422+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:36:18.592422+0000","last_peered":"2024-09-15T22:36:18.592422+0000","last_clean":"2024-09-15T22:36:18.592422+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:36:18.592422+0000","last_undegraded":"2024-09-15T22:36:18.592422+0000","last_fullsized":"2024-09-15T22:36:18.592422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:59.410382+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:36:59.410382+0000","last_peered":"2024-09-15T22:36:59.410382+0000","last_clean":"2024-09-15T22:36:59.410382+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:36:59.410382+0000","last_undegraded":"2024-09-15T22:36:59.410382+0000","last_fullsized":"2024-09-15T22:36:59.410382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292925+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:36:18.292925+0000","last_peered":"2024-09-15T22:36:18.292925+0000","last_clean":"2024-09-15T22:36:18.292925+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:36:18.292925+0000","last_undegraded":"2024-09-15T22:36:18.292925+0000","last_fullsized":"2024-09-15T22:36:18.292925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292237+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:36:18.292237+0000","last_peered":"2024-09-15T22:36:18.292237+0000","last_clean":"2024-09-15T22:36:18.292237+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:36:18.292237+0000","last_undegraded":"2024-09-15T22:36:18.292237+0000","last_fullsized":"2024-09-15T22:36:18.292237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.386813+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:36:44.386813+0000","last_peered":"2024-09-15T22:36:44.386813+0000","last_clean":"2024-09-15T22:36:44.386813+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:36:44.386813+0000","last_undegraded":"2024-09-15T22:36:44.386813+0000","last_fullsized":"2024-09-15T22:36:44.386813+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593023+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:36:18.593023+0000","last_peered":"2024-09-15T22:36:18.593023+0000","last_clean":"2024-09-15T22:36:18.593023+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:36:18.593023+0000","last_undegraded":"2024-09-15T22:36:18.593023+0000","last_fullsized":"2024-09-15T22:36:18.593023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591320+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:36:18.591320+0000","last_peered":"2024-09-15T22:36:18.591320+0000","last_clean":"2024-09-15T22:36:18.591320+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:36:18.591320+0000","last_undegraded":"2024-09-15T22:36:18.591320+0000","last_fullsized":"2024-09-15T22:36:18.591320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:45.387207+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:36:45.387207+0000","last_peered":"2024-09-15T22:36:45.387207+0000","last_clean":"2024-09-15T22:36:45.387207+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:36:45.387207+0000","last_undegraded":"2024-09-15T22:36:45.387207+0000","last_fullsized":"2024-09-15T22:36:45.387207+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844838+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:36:19.844838+0000","last_peered":"2024-09-15T22:36:19.844838+0000","last_clean":"2024-09-15T22:36:19.844838+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:36:19.844838+0000","last_undegraded":"2024-09-15T22:36:19.844838+0000","last_fullsized":"2024-09-15T22:36:19.844838+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845128+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:36:19.845128+0000","last_peered":"2024-09-15T22:36:19.845128+0000","last_clean":"2024-09-15T22:36:19.845128+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:36:19.845128+0000","last_undegraded":"2024-09-15T22:36:19.845128+0000","last_fullsized":"2024-09-15T22:36:19.845128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845209+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:36:19.845209+0000","last_peered":"2024-09-15T22:36:19.845209+0000","last_clean":"2024-09-15T22:36:19.845209+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:36:19.845209+0000","last_undegraded":"2024-09-15T22:36:19.845209+0000","last_fullsized":"2024-09-15T22:36:19.845209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.385825+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:36:44.385825+0000","last_peered":"2024-09-15T22:36:44.385825+0000","last_clean":"2024-09-15T22:36:44.385825+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:36:44.385825+0000","last_undegraded":"2024-09-15T22:36:44.385825+0000","last_fullsized":"2024-09-15T22:36:44.385825+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358561+0000","last_change":"2024-09-15T22:36:15.301170+0000","last_active":"2024-09-15T22:36:18.358561+0000","last_peered":"2024-09-15T22:36:18.358561+0000","last_clean":"2024-09-15T22:36:18.358561+0000","last_became_active":"2024-09-15T22:36:15.300721+0000","last_became_peered":"2024-09-15T22:36:15.300721+0000","last_unstale":"2024-09-15T22:36:18.358561+0000","last_undegraded":"2024-09-15T22:36:18.358561+0000","last_fullsized":"2024-09-15T22:36:18.358561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:25.955835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844985+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:36:19.844985+0000","last_peered":"2024-09-15T22:36:19.844985+0000","last_clean":"2024-09-15T22:36:19.844985+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:36:19.844985+0000","last_undegraded":"2024-09-15T22:36:19.844985+0000","last_fullsized":"2024-09-15T22:36:19.844985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593130+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:36:18.593130+0000","last_peered":"2024-09-15T22:36:18.593130+0000","last_clean":"2024-09-15T22:36:18.593130+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:36:18.593130+0000","last_undegraded":"2024-09-15T22:36:18.593130+0000","last_fullsized":"2024-09-15T22:36:18.593130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591670+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:36:18.591670+0000","last_peered":"2024-09-15T22:36:18.591670+0000","last_clean":"2024-09-15T22:36:18.591670+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:36:18.591670+0000","last_undegraded":"2024-09-15T22:36:18.591670+0000","last_fullsized":"2024-09-15T22:36:18.591670+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845197+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:36:19.845197+0000","last_peered":"2024-09-15T22:36:19.845197+0000","last_clean":"2024-09-15T22:36:19.845197+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:36:19.845197+0000","last_undegraded":"2024-09-15T22:36:19.845197+0000","last_fullsized":"2024-09-15T22:36:19.845197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293518+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:36:18.293518+0000","last_peered":"2024-09-15T22:36:18.293518+0000","last_clean":"2024-09-15T22:36:18.293518+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:36:18.293518+0000","last_undegraded":"2024-09-15T22:36:18.293518+0000","last_fullsized":"2024-09-15T22:36:18.293518+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592254+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:36:18.592254+0000","last_peered":"2024-09-15T22:36:18.592254+0000","last_clean":"2024-09-15T22:36:18.592254+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:36:18.592254+0000","last_undegraded":"2024-09-15T22:36:18.592254+0000","last_fullsized":"2024-09-15T22:36:18.592254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591981+0000","last_change":"2024-09-15T22:36:08.257268+0000","last_active":"2024-09-15T22:36:18.591981+0000","last_peered":"2024-09-15T22:36:18.591981+0000","last_clean":"2024-09-15T22:36:18.591981+0000","last_became_active":"2024-09-15T22:36:08.256970+0000","last_became_peered":"2024-09-15T22:36:08.256970+0000","last_unstale":"2024-09-15T22:36:18.591981+0000","last_undegraded":"2024-09-15T22:36:18.591981+0000","last_fullsized":"2024-09-15T22:36:18.591981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.481895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:45.388772+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:36:45.388772+0000","last_peered":"2024-09-15T22:36:45.388772+0000","last_clean":"2024-09-15T22:36:45.388772+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:36:45.388772+0000","last_undegraded":"2024-09-15T22:36:45.388772+0000","last_fullsized":"2024-09-15T22:36:45.388772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592457+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:36:18.592457+0000","last_peered":"2024-09-15T22:36:18.592457+0000","last_clean":"2024-09-15T22:36:18.592457+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:36:18.592457+0000","last_undegraded":"2024-09-15T22:36:18.592457+0000","last_fullsized":"2024-09-15T22:36:18.592457+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592502+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:36:18.592502+0000","last_peered":"2024-09-15T22:36:18.592502+0000","last_clean":"2024-09-15T22:36:18.592502+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:36:18.592502+0000","last_undegraded":"2024-09-15T22:36:18.592502+0000","last_fullsized":"2024-09-15T22:36:18.592502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.387272+0000","last_change":"2024-09-15T22:36:06.337927+0000","last_active":"2024-09-15T22:36:44.387272+0000","last_peered":"2024-09-15T22:36:44.387272+0000","last_clean":"2024-09-15T22:36:44.387272+0000","last_became_active":"2024-09-15T22:36:06.337478+0000","last_became_peered":"2024-09-15T22:36:06.337478+0000","last_unstale":"2024-09-15T22:36:44.387272+0000","last_undegraded":"2024-09-15T22:36:44.387272+0000","last_fullsized":"2024-09-15T22:36:44.387272+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:12.223022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293467+0000","last_change":"2024-09-15T22:36:15.297875+0000","last_active":"2024-09-15T22:36:18.293467+0000","last_peered":"2024-09-15T22:36:18.293467+0000","last_clean":"2024-09-15T22:36:18.293467+0000","last_became_active":"2024-09-15T22:36:15.297339+0000","last_became_peered":"2024-09-15T22:36:15.297339+0000","last_unstale":"2024-09-15T22:36:18.293467+0000","last_undegraded":"2024-09-15T22:36:18.293467+0000","last_fullsized":"2024-09-15T22:36:18.293467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:16.229871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844561+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:36:19.844561+0000","last_peered":"2024-09-15T22:36:19.844561+0000","last_clean":"2024-09-15T22:36:19.844561+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:36:19.844561+0000","last_undegraded":"2024-09-15T22:36:19.844561+0000","last_fullsized":"2024-09-15T22:36:19.844561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":51,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:59.410916+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:36:59.410916+0000","last_peered":"2024-09-15T22:36:59.410916+0000","last_clean":"2024-09-15T22:36:59.410916+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:36:59.410916+0000","last_undegraded":"2024-09-15T22:36:59.410916+0000","last_fullsized":"2024-09-15T22:36:59.410916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593462+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:36:18.593462+0000","last_peered":"2024-09-15T22:36:18.593462+0000","last_clean":"2024-09-15T22:36:18.593462+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:36:18.593462+0000","last_undegraded":"2024-09-15T22:36:18.593462+0000","last_fullsized":"2024-09-15T22:36:18.593462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844850+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:36:19.844850+0000","last_peered":"2024-09-15T22:36:19.844850+0000","last_clean":"2024-09-15T22:36:19.844850+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:36:19.844850+0000","last_undegraded":"2024-09-15T22:36:19.844850+0000","last_fullsized":"2024-09-15T22:36:19.844850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.383093+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:36:43.383093+0000","last_peered":"2024-09-15T22:36:43.383093+0000","last_clean":"2024-09-15T22:36:43.383093+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:36:43.383093+0000","last_undegraded":"2024-09-15T22:36:43.383093+0000","last_fullsized":"2024-09-15T22:36:43.383093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591369+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:36:18.591369+0000","last_peered":"2024-09-15T22:36:18.591369+0000","last_clean":"2024-09-15T22:36:18.591369+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:36:18.591369+0000","last_undegraded":"2024-09-15T22:36:18.591369+0000","last_fullsized":"2024-09-15T22:36:18.591369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292529+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:36:18.292529+0000","last_peered":"2024-09-15T22:36:18.292529+0000","last_clean":"2024-09-15T22:36:18.292529+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:36:18.292529+0000","last_undegraded":"2024-09-15T22:36:18.292529+0000","last_fullsized":"2024-09-15T22:36:18.292529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.384752+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:36:44.384752+0000","last_peered":"2024-09-15T22:36:44.384752+0000","last_clean":"2024-09-15T22:36:44.384752+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:36:44.384752+0000","last_undegraded":"2024-09-15T22:36:44.384752+0000","last_fullsized":"2024-09-15T22:36:44.384752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592960+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:36:18.592960+0000","last_peered":"2024-09-15T22:36:18.592960+0000","last_clean":"2024-09-15T22:36:18.592960+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:36:18.592960+0000","last_undegraded":"2024-09-15T22:36:18.592960+0000","last_fullsized":"2024-09-15T22:36:18.592960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591318+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:36:18.591318+0000","last_peered":"2024-09-15T22:36:18.591318+0000","last_clean":"2024-09-15T22:36:18.591318+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:36:18.591318+0000","last_undegraded":"2024-09-15T22:36:18.591318+0000","last_fullsized":"2024-09-15T22:36:18.591318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'25","reported_seq":87,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:57.412011+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:36:57.412011+0000","last_peered":"2024-09-15T22:36:57.412011+0000","last_clean":"2024-09-15T22:36:57.412011+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:36:57.412011+0000","last_undegraded":"2024-09-15T22:36:57.412011+0000","last_fullsized":"2024-09-15T22:36:57.412011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592950+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:36:18.592950+0000","last_peered":"2024-09-15T22:36:18.592950+0000","last_clean":"2024-09-15T22:36:18.592950+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:36:18.592950+0000","last_undegraded":"2024-09-15T22:36:18.592950+0000","last_fullsized":"2024-09-15T22:36:18.592950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358593+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:36:18.358593+0000","last_peered":"2024-09-15T22:36:18.358593+0000","last_clean":"2024-09-15T22:36:18.358593+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:36:18.358593+0000","last_undegraded":"2024-09-15T22:36:18.358593+0000","last_fullsized":"2024-09-15T22:36:18.358593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.381995+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:36:43.381995+0000","last_peered":"2024-09-15T22:36:43.381995+0000","last_clean":"2024-09-15T22:36:43.381995+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:36:43.381995+0000","last_undegraded":"2024-09-15T22:36:43.381995+0000","last_fullsized":"2024-09-15T22:36:43.381995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591582+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:36:18.591582+0000","last_peered":"2024-09-15T22:36:18.591582+0000","last_clean":"2024-09-15T22:36:18.591582+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:36:18.591582+0000","last_undegraded":"2024-09-15T22:36:18.591582+0000","last_fullsized":"2024-09-15T22:36:18.591582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592430+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:36:18.592430+0000","last_peered":"2024-09-15T22:36:18.592430+0000","last_clean":"2024-09-15T22:36:18.592430+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:36:18.592430+0000","last_undegraded":"2024-09-15T22:36:18.592430+0000","last_fullsized":"2024-09-15T22:36:18.592430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:43.383004+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:36:43.383004+0000","last_peered":"2024-09-15T22:36:43.383004+0000","last_clean":"2024-09-15T22:36:43.383004+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:36:43.383004+0000","last_undegraded":"2024-09-15T22:36:43.383004+0000","last_fullsized":"2024-09-15T22:36:43.383004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592339+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:36:18.592339+0000","last_peered":"2024-09-15T22:36:18.592339+0000","last_clean":"2024-09-15T22:36:18.592339+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:36:18.592339+0000","last_undegraded":"2024-09-15T22:36:18.592339+0000","last_fullsized":"2024-09-15T22:36:18.592339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358888+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:36:18.358888+0000","last_peered":"2024-09-15T22:36:18.358888+0000","last_clean":"2024-09-15T22:36:18.358888+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:36:18.358888+0000","last_undegraded":"2024-09-15T22:36:18.358888+0000","last_fullsized":"2024-09-15T22:36:18.358888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:56.405889+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:36:56.405889+0000","last_peered":"2024-09-15T22:36:56.405889+0000","last_clean":"2024-09-15T22:36:56.405889+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:36:56.405889+0000","last_undegraded":"2024-09-15T22:36:56.405889+0000","last_fullsized":"2024-09-15T22:36:56.405889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591308+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:36:18.591308+0000","last_peered":"2024-09-15T22:36:18.591308+0000","last_clean":"2024-09-15T22:36:18.591308+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:36:18.591308+0000","last_undegraded":"2024-09-15T22:36:18.591308+0000","last_fullsized":"2024-09-15T22:36:18.591308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592297+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:36:18.592297+0000","last_peered":"2024-09-15T22:36:18.592297+0000","last_clean":"2024-09-15T22:36:18.592297+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:36:18.592297+0000","last_undegraded":"2024-09-15T22:36:18.592297+0000","last_fullsized":"2024-09-15T22:36:18.592297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592495+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:36:18.592495+0000","last_peered":"2024-09-15T22:36:18.592495+0000","last_clean":"2024-09-15T22:36:18.592495+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:36:18.592495+0000","last_undegraded":"2024-09-15T22:36:18.592495+0000","last_fullsized":"2024-09-15T22:36:18.592495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592665+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:36:18.592665+0000","last_peered":"2024-09-15T22:36:18.592665+0000","last_clean":"2024-09-15T22:36:18.592665+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:36:18.592665+0000","last_undegraded":"2024-09-15T22:36:18.592665+0000","last_fullsized":"2024-09-15T22:36:18.592665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591525+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:36:18.591525+0000","last_peered":"2024-09-15T22:36:18.591525+0000","last_clean":"2024-09-15T22:36:18.591525+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:36:18.591525+0000","last_undegraded":"2024-09-15T22:36:18.591525+0000","last_fullsized":"2024-09-15T22:36:18.591525+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591108+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:36:18.591108+0000","last_peered":"2024-09-15T22:36:18.591108+0000","last_clean":"2024-09-15T22:36:18.591108+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:36:18.591108+0000","last_undegraded":"2024-09-15T22:36:18.591108+0000","last_fullsized":"2024-09-15T22:36:18.591108+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.385491+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:36:44.385491+0000","last_peered":"2024-09-15T22:36:44.385491+0000","last_clean":"2024-09-15T22:36:44.385491+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:36:44.385491+0000","last_undegraded":"2024-09-15T22:36:44.385491+0000","last_fullsized":"2024-09-15T22:36:44.385491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:44.384457+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:36:44.384457+0000","last_peered":"2024-09-15T22:36:44.384457+0000","last_clean":"2024-09-15T22:36:44.384457+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:36:44.384457+0000","last_undegraded":"2024-09-15T22:36:44.384457+0000","last_fullsized":"2024-09-15T22:36:44.384457+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592640+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:36:18.592640+0000","last_peered":"2024-09-15T22:36:18.592640+0000","last_clean":"2024-09-15T22:36:18.592640+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:36:18.592640+0000","last_undegraded":"2024-09-15T22:36:18.592640+0000","last_fullsized":"2024-09-15T22:36:18.592640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294883+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:36:18.294883+0000","last_peered":"2024-09-15T22:36:18.294883+0000","last_clean":"2024-09-15T22:36:18.294883+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:36:18.294883+0000","last_undegraded":"2024-09-15T22:36:18.294883+0000","last_fullsized":"2024-09-15T22:36:18.294883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:57.407594+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:36:57.407594+0000","last_peered":"2024-09-15T22:36:57.407594+0000","last_clean":"2024-09-15T22:36:57.407594+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:36:57.407594+0000","last_undegraded":"2024-09-15T22:36:57.407594+0000","last_fullsized":"2024-09-15T22:36:57.407594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591540+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:36:18.591540+0000","last_peered":"2024-09-15T22:36:18.591540+0000","last_clean":"2024-09-15T22:36:18.591540+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:36:18.591540+0000","last_undegraded":"2024-09-15T22:36:18.591540+0000","last_fullsized":"2024-09-15T22:36:18.591540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295048+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:36:18.295048+0000","last_peered":"2024-09-15T22:36:18.295048+0000","last_clean":"2024-09-15T22:36:18.295048+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:36:18.295048+0000","last_undegraded":"2024-09-15T22:36:18.295048+0000","last_fullsized":"2024-09-15T22:36:18.295048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591113+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:36:18.591113+0000","last_peered":"2024-09-15T22:36:18.591113+0000","last_clean":"2024-09-15T22:36:18.591113+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:36:18.591113+0000","last_undegraded":"2024-09-15T22:36:18.591113+0000","last_fullsized":"2024-09-15T22:36:18.591113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295384+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:36:18.295384+0000","last_peered":"2024-09-15T22:36:18.295384+0000","last_clean":"2024-09-15T22:36:18.295384+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:36:18.295384+0000","last_undegraded":"2024-09-15T22:36:18.295384+0000","last_fullsized":"2024-09-15T22:36:18.295384+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.294676+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:36:18.294676+0000","last_peered":"2024-09-15T22:36:18.294676+0000","last_clean":"2024-09-15T22:36:18.294676+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:36:18.294676+0000","last_undegraded":"2024-09-15T22:36:18.294676+0000","last_fullsized":"2024-09-15T22:36:18.294676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591741+0000","last_change":"2024-09-15T22:36:08.250464+0000","last_active":"2024-09-15T22:36:18.591741+0000","last_peered":"2024-09-15T22:36:18.591741+0000","last_clean":"2024-09-15T22:36:18.591741+0000","last_became_active":"2024-09-15T22:36:08.250134+0000","last_became_peered":"2024-09-15T22:36:08.250134+0000","last_unstale":"2024-09-15T22:36:18.591741+0000","last_undegraded":"2024-09-15T22:36:18.591741+0000","last_fullsized":"2024-09-15T22:36:18.591741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:20:41.969151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293025+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:36:18.293025+0000","last_peered":"2024-09-15T22:36:18.293025+0000","last_clean":"2024-09-15T22:36:18.293025+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:36:18.293025+0000","last_undegraded":"2024-09-15T22:36:18.293025+0000","last_fullsized":"2024-09-15T22:36:18.293025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591637+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:36:18.591637+0000","last_peered":"2024-09-15T22:36:18.591637+0000","last_clean":"2024-09-15T22:36:18.591637+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:36:18.591637+0000","last_undegraded":"2024-09-15T22:36:18.591637+0000","last_fullsized":"2024-09-15T22:36:18.591637+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295210+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:36:18.295210+0000","last_peered":"2024-09-15T22:36:18.295210+0000","last_clean":"2024-09-15T22:36:18.295210+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:36:18.295210+0000","last_undegraded":"2024-09-15T22:36:18.295210+0000","last_fullsized":"2024-09-15T22:36:18.295210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591394+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:36:18.591394+0000","last_peered":"2024-09-15T22:36:18.591394+0000","last_clean":"2024-09-15T22:36:18.591394+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:36:18.591394+0000","last_undegraded":"2024-09-15T22:36:18.591394+0000","last_fullsized":"2024-09-15T22:36:18.591394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592341+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:36:18.592341+0000","last_peered":"2024-09-15T22:36:18.592341+0000","last_clean":"2024-09-15T22:36:18.592341+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:36:18.592341+0000","last_undegraded":"2024-09-15T22:36:18.592341+0000","last_fullsized":"2024-09-15T22:36:18.592341+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591538+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:36:18.591538+0000","last_peered":"2024-09-15T22:36:18.591538+0000","last_clean":"2024-09-15T22:36:18.591538+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:36:18.591538+0000","last_undegraded":"2024-09-15T22:36:18.591538+0000","last_fullsized":"2024-09-15T22:36:18.591538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358589+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:36:18.358589+0000","last_peered":"2024-09-15T22:36:18.358589+0000","last_clean":"2024-09-15T22:36:18.358589+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:36:18.358589+0000","last_undegraded":"2024-09-15T22:36:18.358589+0000","last_fullsized":"2024-09-15T22:36:18.358589+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292689+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:36:18.292689+0000","last_peered":"2024-09-15T22:36:18.292689+0000","last_clean":"2024-09-15T22:36:18.292689+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:36:18.292689+0000","last_undegraded":"2024-09-15T22:36:18.292689+0000","last_fullsized":"2024-09-15T22:36:18.292689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591322+0000","last_change":"2024-09-15T22:36:08.262314+0000","last_active":"2024-09-15T22:36:18.591322+0000","last_peered":"2024-09-15T22:36:18.591322+0000","last_clean":"2024-09-15T22:36:18.591322+0000","last_became_active":"2024-09-15T22:36:08.261958+0000","last_became_peered":"2024-09-15T22:36:08.261958+0000","last_unstale":"2024-09-15T22:36:18.591322+0000","last_undegraded":"2024-09-15T22:36:18.591322+0000","last_fullsized":"2024-09-15T22:36:18.591322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:25.822697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.295177+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:36:18.295177+0000","last_peered":"2024-09-15T22:36:18.295177+0000","last_clean":"2024-09-15T22:36:18.295177+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:36:18.295177+0000","last_undegraded":"2024-09-15T22:36:18.295177+0000","last_fullsized":"2024-09-15T22:36:18.295177+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.593326+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:36:18.593326+0000","last_peered":"2024-09-15T22:36:18.593326+0000","last_clean":"2024-09-15T22:36:18.593326+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:36:18.593326+0000","last_undegraded":"2024-09-15T22:36:18.593326+0000","last_fullsized":"2024-09-15T22:36:18.593326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591885+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:36:18.591885+0000","last_peered":"2024-09-15T22:36:18.591885+0000","last_clean":"2024-09-15T22:36:18.591885+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:36:18.591885+0000","last_undegraded":"2024-09-15T22:36:18.591885+0000","last_fullsized":"2024-09-15T22:36:18.591885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358212+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:36:18.358212+0000","last_peered":"2024-09-15T22:36:18.358212+0000","last_clean":"2024-09-15T22:36:18.358212+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:36:18.358212+0000","last_undegraded":"2024-09-15T22:36:18.358212+0000","last_fullsized":"2024-09-15T22:36:18.358212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591334+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:36:18.591334+0000","last_peered":"2024-09-15T22:36:18.591334+0000","last_clean":"2024-09-15T22:36:18.591334+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:36:18.591334+0000","last_undegraded":"2024-09-15T22:36:18.591334+0000","last_fullsized":"2024-09-15T22:36:18.591334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591525+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:36:18.591525+0000","last_peered":"2024-09-15T22:36:18.591525+0000","last_clean":"2024-09-15T22:36:18.591525+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:36:18.591525+0000","last_undegraded":"2024-09-15T22:36:18.591525+0000","last_fullsized":"2024-09-15T22:36:18.591525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292680+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:36:18.292680+0000","last_peered":"2024-09-15T22:36:18.292680+0000","last_clean":"2024-09-15T22:36:18.292680+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:36:18.292680+0000","last_undegraded":"2024-09-15T22:36:18.292680+0000","last_fullsized":"2024-09-15T22:36:18.292680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591615+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:36:18.591615+0000","last_peered":"2024-09-15T22:36:18.591615+0000","last_clean":"2024-09-15T22:36:18.591615+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:36:18.591615+0000","last_undegraded":"2024-09-15T22:36:18.591615+0000","last_fullsized":"2024-09-15T22:36:18.591615+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.845130+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:36:19.845130+0000","last_peered":"2024-09-15T22:36:19.845130+0000","last_clean":"2024-09-15T22:36:19.845130+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:36:19.845130+0000","last_undegraded":"2024-09-15T22:36:19.845130+0000","last_fullsized":"2024-09-15T22:36:19.845130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.293329+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:36:18.293329+0000","last_peered":"2024-09-15T22:36:18.293329+0000","last_clean":"2024-09-15T22:36:18.293329+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:36:18.293329+0000","last_undegraded":"2024-09-15T22:36:18.293329+0000","last_fullsized":"2024-09-15T22:36:18.293329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.358968+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:36:18.358968+0000","last_peered":"2024-09-15T22:36:18.358968+0000","last_clean":"2024-09-15T22:36:18.358968+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:36:18.358968+0000","last_undegraded":"2024-09-15T22:36:18.358968+0000","last_fullsized":"2024-09-15T22:36:18.358968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592040+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:36:18.592040+0000","last_peered":"2024-09-15T22:36:18.592040+0000","last_clean":"2024-09-15T22:36:18.592040+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:36:18.592040+0000","last_undegraded":"2024-09-15T22:36:18.592040+0000","last_fullsized":"2024-09-15T22:36:18.592040+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:19.844646+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:36:19.844646+0000","last_peered":"2024-09-15T22:36:19.844646+0000","last_clean":"2024-09-15T22:36:19.844646+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:36:19.844646+0000","last_undegraded":"2024-09-15T22:36:19.844646+0000","last_fullsized":"2024-09-15T22:36:19.844646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.292852+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:36:18.292852+0000","last_peered":"2024-09-15T22:36:18.292852+0000","last_clean":"2024-09-15T22:36:18.292852+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:36:18.292852+0000","last_undegraded":"2024-09-15T22:36:18.292852+0000","last_fullsized":"2024-09-15T22:36:18.292852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591023+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:36:18.591023+0000","last_peered":"2024-09-15T22:36:18.591023+0000","last_clean":"2024-09-15T22:36:18.591023+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:36:18.591023+0000","last_undegraded":"2024-09-15T22:36:18.591023+0000","last_fullsized":"2024-09-15T22:36:18.591023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591483+0000","last_change":"2024-09-15T22:36:08.252626+0000","last_active":"2024-09-15T22:36:18.591483+0000","last_peered":"2024-09-15T22:36:18.591483+0000","last_clean":"2024-09-15T22:36:18.591483+0000","last_became_active":"2024-09-15T22:36:08.252196+0000","last_became_peered":"2024-09-15T22:36:08.252196+0000","last_unstale":"2024-09-15T22:36:18.591483+0000","last_undegraded":"2024-09-15T22:36:18.591483+0000","last_fullsized":"2024-09-15T22:36:18.591483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:47:31.776267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592744+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:36:18.592744+0000","last_peered":"2024-09-15T22:36:18.592744+0000","last_clean":"2024-09-15T22:36:18.592744+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:36:18.592744+0000","last_undegraded":"2024-09-15T22:36:18.592744+0000","last_fullsized":"2024-09-15T22:36:18.592744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.592414+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:36:18.592414+0000","last_peered":"2024-09-15T22:36:18.592414+0000","last_clean":"2024-09-15T22:36:18.592414+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:36:18.592414+0000","last_undegraded":"2024-09-15T22:36:18.592414+0000","last_fullsized":"2024-09-15T22:36:18.592414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-15T22:36:18.591781+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:36:18.591781+0000","last_peered":"2024-09-15T22:36:18.591781+0000","last_clean":"2024-09-15T22:36:18.591781+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:36:18.591781+0000","last_undegraded":"2024-09-15T22:36:18.591781+0000","last_fullsized":"2024-09-15T22:36:18.591781+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":110,"num_read_kb":106,"num_write":153,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":172676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612630,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":472536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841563,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1103149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005089,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":476868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234022,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":472692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462956,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36876,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708276,"statfs":{"total":95995035648,"available":95957274624,"internally_reserved":0,"allocated":1568768,"data_stored":1068603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626481,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":467948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855413,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1576960,"data_stored":1064191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":472598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":468142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":528262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1019904,"data_stored":526519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":469336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:37:03.831 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:37:03.831 INFO:tasks.cephadm:Setting up client nodes... 2024-09-15T22:37:03.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-15T22:37:04.101 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:04.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:04 smithi016 ceph-mon[31492]: pgmap v311: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-15T22:37:04.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:04 smithi016 ceph-mon[31492]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:04.488 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:04 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.578+0000 7f0cec2cd640 1 -- 172.21.15.16:0/3212995810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4101d10 msgr2=0x7f0ce4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.578+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/3212995810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4101d10 0x7f0ce4102190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009900 tx=0x7f0cd402a410 comp rx=0 tx=0).stop 2024-09-15T22:37:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 -- 172.21.15.16:0/3212995810 shutdown_connections 2024-09-15T22:37:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/3212995810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ce41026d0 0x7f0ce4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/3212995810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce4101d10 0x7f0ce4102190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/3212995810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4100b10 0x7f0ce4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 -- 172.21.15.16:0/3212995810 >> 172.21.15.16:0/3212995810 conn(0x7f0ce40fc2a0 msgr2=0x7f0ce40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 -- 172.21.15.16:0/3212995810 shutdown_connections 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 -- 172.21.15.16:0/3212995810 wait complete. 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 Processor -- start 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.579+0000 7f0cec2cd640 1 -- start start 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cec2cd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ce4100b10 0x7f0ce406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cec2cd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4101d10 0x7f0ce419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cec2cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce41026d0 0x7f0ce41a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cec2cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce410c790 con 0x7f0ce41026d0 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cec2cd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce410c610 con 0x7f0ce4101d10 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cec2cd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce410c910 con 0x7f0ce4100b10 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4101d10 0x7f0ce419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cea069640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ce4100b10 0x7f0ce406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0cea86a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce41026d0 0x7f0ce41a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4101d10 0x7f0ce419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:32828/0 (socket says 172.21.15.16:32828) 2024-09-15T22:37:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 -- 172.21.15.16:0/1916117908 learned_addr learned my addr 172.21.15.16:0/1916117908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 -- 172.21.15.16:0/1916117908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ce4100b10 msgr2=0x7f0ce406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ce4100b10 0x7f0ce406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 -- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce41026d0 msgr2=0x7f0ce41a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce41026d0 0x7f0ce41a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 -- 172.21.15.16:0/1916117908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce41a1db0 con 0x7f0ce4101d10 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.580+0000 7f0ce9868640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4101d10 0x7f0ce419f070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009230 tx=0x7f0cd40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.581+0000 7f0cdb7fe640 1 -- 172.21.15.16:0/1916117908 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd403f030 con 0x7f0ce4101d10 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.581+0000 7f0cdb7fe640 1 -- 172.21.15.16:0/1916117908 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4003ce0 con 0x7f0ce4101d10 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.581+0000 7f0cdb7fe640 1 -- 172.21.15.16:0/1916117908 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4035070 con 0x7f0ce4101d10 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.581+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce41a1fe0 con 0x7f0ce4101d10 2024-09-15T22:37:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.581+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41a2430 con 0x7f0ce4101d10 2024-09-15T22:37:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.582+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce4107f10 con 0x7f0ce4101d10 2024-09-15T22:37:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.583+0000 7f0cdb7fe640 1 -- 172.21.15.16:0/1916117908 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0cd4042030 con 0x7f0ce4101d10 2024-09-15T22:37:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.583+0000 7f0cdb7fe640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cd00a3300 0x7f0cd00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.584+0000 7f0cdb7fe640 1 -- 172.21.15.16:0/1916117908 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f0cd40f0120 con 0x7f0ce4101d10 2024-09-15T22:37:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.584+0000 7f0cea069640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cd00a3300 0x7f0cd00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.584+0000 7f0cea069640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cd00a3300 0x7f0cd00a57c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4101b70 tx=0x7f0ce0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.588+0000 7f0cdb7fe640 1 -- 172.21.15.16:0/1916117908 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cd40bc430 con 0x7f0ce4101d10 2024-09-15T22:37:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:04 smithi059 ceph-mon[34872]: pgmap v311: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-15T22:37:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:04 smithi059 ceph-mon[34872]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:04.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:04 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:04.715 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:04 smithi130 ceph-mon[35742]: pgmap v311: 129 pgs: 129 active+clean; 623 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-15T22:37:04.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:04 smithi130 ceph-mon[35742]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:04.716 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:04 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.840+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0ce4062e20 con 0x7f0ce4101d10 2024-09-15T22:37:04.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.847+0000 7f0cdb7fe640 1 -- 172.21.15.16:0/1916117908 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f0cd40c0730 con 0x7f0ce4101d10 2024-09-15T22:37:04.848 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-15T22:37:04.849 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCQYedm1UcyMhAAumKd187oS6BEhnrcvZfhrw== 2024-09-15T22:37:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cd00a3300 msgr2=0x7f0cd00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cd00a3300 0x7f0cd00a57c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4101b70 tx=0x7f0ce0005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4101d10 msgr2=0x7f0ce419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4101d10 0x7f0ce419f070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009230 tx=0x7f0cd40042f0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 shutdown_connections 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cd00a3300 0x7f0cd00a57c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ce41026d0 0x7f0ce41a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ce4101d10 0x7f0ce419f070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 --2- 172.21.15.16:0/1916117908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0ce4100b10 0x7f0ce406b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 >> 172.21.15.16:0/1916117908 conn(0x7f0ce40fc2a0 msgr2=0x7f0ce41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 shutdown_connections 2024-09-15T22:37:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:04.849+0000 7f0cec2cd640 1 -- 172.21.15.16:0/1916117908 wait complete. 2024-09-15T22:37:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:05 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1916117908' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:37:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:05 smithi059 ceph-mon[34872]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:37:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:05 smithi059 ceph-mon[34872]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:37:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:05 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:05.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:05 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1916117908' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:37:05.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:05 smithi016 ceph-mon[31492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:37:05.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:05 smithi016 ceph-mon[31492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:37:05.722 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:05 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:05 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1916117908' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:37:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:05 smithi130 ceph-mon[35742]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:37:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:05 smithi130 ceph-mon[35742]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:37:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:05 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.067 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:37:06.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-15T22:37:06.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T22:37:06.108 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T22:37:06.109 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-15T22:37:06.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-15T22:37:06.432 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:06.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:06 smithi016 ceph-mon[31492]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-15T22:37:06.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.583 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:06 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:06 smithi059 ceph-mon[34872]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-15T22:37:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:06 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:06 smithi130 ceph-mon[35742]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-15T22:37:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:06 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 -- 172.21.15.16:0/2140253525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 msgr2=0x7f90d0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 --2- 172.21.15.16:0/2140253525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0107b90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009890 tx=0x7f90c002a380 comp rx=0 tx=0).stop 2024-09-15T22:37:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 -- 172.21.15.16:0/2140253525 shutdown_connections 2024-09-15T22:37:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 --2- 172.21.15.16:0/2140253525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90d01082a0 0x7f90d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 --2- 172.21.15.16:0/2140253525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0107b90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 --2- 172.21.15.16:0/2140253525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90d0102d30 0x7f90d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 -- 172.21.15.16:0/2140253525 >> 172.21.15.16:0/2140253525 conn(0x7f90d00fc7d0 msgr2=0x7f90d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 -- 172.21.15.16:0/2140253525 shutdown_connections 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.941+0000 7f90d8011640 1 -- 172.21.15.16:0/2140253525 wait complete. 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.942+0000 7f90d8011640 1 Processor -- start 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.942+0000 7f90d8011640 1 -- start start 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d8011640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90d0102d30 0x7f90d01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d8011640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d8011640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90d01082a0 0x7f90d019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d8011640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90d0112060 con 0x7f90d01036e0 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d8011640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90d0111ee0 con 0x7f90d01082a0 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d8011640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90d01121e0 con 0x7f90d0102d30 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38840/0 (socket says 172.21.15.16:38840) 2024-09-15T22:37:06.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 -- 172.21.15.16:0/2249367460 learned_addr learned my addr 172.21.15.16:0/2249367460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d5dad640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90d0102d30 0x7f90d01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d65ae640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90d01082a0 0x7f90d019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 -- 172.21.15.16:0/2249367460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90d0102d30 msgr2=0x7f90d01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90d0102d30 0x7f90d01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 -- 172.21.15.16:0/2249367460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90d01082a0 msgr2=0x7f90d019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90d01082a0 0x7f90d019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 -- 172.21.15.16:0/2249367460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d019d570 con 0x7f90d01036e0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d65ae640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90d01082a0 0x7f90d019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.943+0000 7f90d55ac640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0198b00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f90c002a970 tx=0x7f90c0033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.944+0000 7f90c6ffd640 1 -- 172.21.15.16:0/2249367460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0004240 con 0x7f90d01036e0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.944+0000 7f90c6ffd640 1 -- 172.21.15.16:0/2249367460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c00043e0 con 0x7f90d01036e0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.944+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d019d800 con 0x7f90d01036e0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.944+0000 7f90c6ffd640 1 -- 172.21.15.16:0/2249367460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c003a580 con 0x7f90d01036e0 2024-09-15T22:37:06.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.944+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d01008c0 con 0x7f90d01036e0 2024-09-15T22:37:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.945+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d006a850 con 0x7f90d01036e0 2024-09-15T22:37:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.946+0000 7f90c6ffd640 1 -- 172.21.15.16:0/2249367460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f90c00035f0 con 0x7f90d01036e0 2024-09-15T22:37:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.947+0000 7f90c6ffd640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90bc0a32d0 0x7f90bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.947+0000 7f90c6ffd640 1 -- 172.21.15.16:0/2249367460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f90c00f41e0 con 0x7f90d01036e0 2024-09-15T22:37:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.948+0000 7f90d5dad640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90bc0a32d0 0x7f90bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.948+0000 7f90d5dad640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90bc0a32d0 0x7f90bc0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005fd0 tx=0x7f90cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:06.956+0000 7f90c6ffd640 1 -- 172.21.15.16:0/2249367460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90c00c0d10 con 0x7f90d01036e0 2024-09-15T22:37:07.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.179+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f90d0199920 con 0x7f90d01036e0 2024-09-15T22:37:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.186+0000 7f90c6ffd640 1 -- 172.21.15.16:0/2249367460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264044 (secure 0 0 0) 0x7f90c00c0610 con 0x7f90d01036e0 2024-09-15T22:37:07.191 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90bc0a32d0 msgr2=0x7f90bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90bc0a32d0 0x7f90bc0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005fd0 tx=0x7f90cc009260 comp rx=0 tx=0).stop 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 msgr2=0x7f90d0198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0198b00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f90c002a970 tx=0x7f90c0033b60 comp rx=0 tx=0).stop 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 shutdown_connections 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90bc0a32d0 0x7f90bc0a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90d01082a0 0x7f90d019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d01036e0 0x7f90d0198b00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 --2- 172.21.15.16:0/2249367460 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90d0102d30 0x7f90d01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 >> 172.21.15.16:0/2249367460 conn(0x7f90d00fc7d0 msgr2=0x7f90d0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 shutdown_connections 2024-09-15T22:37:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:07.190+0000 7f90d8011640 1 -- 172.21.15.16:0/2249367460 wait complete. 2024-09-15T22:37:07.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:07 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:07 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:07 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:07.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:07 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:07.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:07 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:07 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2249367460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:37:07.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:07 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:07 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:07 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:07.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:07 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:07.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:07 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.804 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:07 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2249367460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:37:07.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":81475521},{"type":"v1","addr":"172.21.15.16:6801","nonce":81475521}]},"active_addr":"172.21.15.16:6801/81475521","active_change":"2024-09-15T22:29:10.483720+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3377264779}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1746300554}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3984482177}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3275020767}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3446468097}]}]} 2024-09-15T22:37:07.874 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-15T22:37:07.874 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-15T22:37:07.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:37:07.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:07 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:07 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:07 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:07.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:07 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:07.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:07 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:37:07.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:07 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2249367460' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:37:08.133 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:08.553 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:08 smithi016 ceph-mon[31492]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-15T22:37:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 -- 172.21.15.16:0/1965813771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 msgr2=0x7fc8501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 --2- 172.21.15.16:0/1965813771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501021c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc840009900 tx=0x7fc84002a410 comp rx=0 tx=0).stop 2024-09-15T22:37:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 -- 172.21.15.16:0/1965813771 shutdown_connections 2024-09-15T22:37:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 --2- 172.21.15.16:0/1965813771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc850106f10 0x7fc85010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 --2- 172.21.15.16:0/1965813771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc850102790 0x7fc8501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 --2- 172.21.15.16:0/1965813771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501021c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 -- 172.21.15.16:0/1965813771 >> 172.21.15.16:0/1965813771 conn(0x7fc8500fd8c0 msgr2=0x7fc8500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 -- 172.21.15.16:0/1965813771 shutdown_connections 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.635+0000 7fc854a48640 1 -- 172.21.15.16:0/1965813771 wait complete. 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 Processor -- start 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 -- start start 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc850102790 0x7fc850198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc850106f10 0x7fc85019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc850111f70 con 0x7fc850101de0 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc850111df0 con 0x7fc850106f10 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc854a48640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501120f0 con 0x7fc850102790 2024-09-15T22:37:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc84e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc84e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38858/0 (socket says 172.21.15.16:38858) 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.636+0000 7fc84e59c640 1 -- 172.21.15.16:0/549401574 learned_addr learned my addr 172.21.15.16:0/549401574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84dd9b640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc850102790 0x7fc850198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84ed9d640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc850106f10 0x7fc85019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84e59c640 1 -- 172.21.15.16:0/549401574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc850102790 msgr2=0x7fc850198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84e59c640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc850102790 0x7fc850198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84e59c640 1 -- 172.21.15.16:0/549401574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc850106f10 msgr2=0x7fc85019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84e59c640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc850106f10 0x7fc85019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84e59c640 1 -- 172.21.15.16:0/549401574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85019d380 con 0x7fc850101de0 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84dd9b640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc850102790 0x7fc850198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc84e59c640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501983d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc840005ab0 tx=0x7fc8400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc83f7fe640 1 -- 172.21.15.16:0/549401574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84003f030 con 0x7fc850101de0 2024-09-15T22:37:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc83f7fe640 1 -- 172.21.15.16:0/549401574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc840003ce0 con 0x7fc850101de0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85019d610 con 0x7fc850101de0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc83f7fe640 1 -- 172.21.15.16:0/549401574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840035070 con 0x7fc850101de0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.637+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8501a4f50 con 0x7fc850101de0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.638+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8501032c0 con 0x7fc850101de0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.639+0000 7fc83f7fe640 1 -- 172.21.15.16:0/549401574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc840042030 con 0x7fc850101de0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.640+0000 7fc83f7fe640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8380a32d0 0x7fc8380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.640+0000 7fc83f7fe640 1 -- 172.21.15.16:0/549401574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fc8400f06e0 con 0x7fc850101de0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.640+0000 7fc84dd9b640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8380a32d0 0x7fc8380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.640+0000 7fc84dd9b640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8380a32d0 0x7fc8380a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8500fdce0 tx=0x7fc844005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.644+0000 7fc83f7fe640 1 -- 172.21.15.16:0/549401574 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8400bd000 con 0x7fc850101de0 2024-09-15T22:37:08.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:08 smithi130 ceph-mon[35742]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-15T22:37:08.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.824+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc85010a0f0 con 0x7fc850101de0 2024-09-15T22:37:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.826+0000 7fc83f7fe640 1 -- 172.21.15.16:0/549401574 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7fc8400bc900 con 0x7fc850101de0 2024-09-15T22:37:08.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:08.830 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8380a32d0 msgr2=0x7fc8380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8380a32d0 0x7fc8380a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8500fdce0 tx=0x7fc844005ec0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 msgr2=0x7fc8501983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501983d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc840005ab0 tx=0x7fc8400042f0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 shutdown_connections 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fc8380a32d0 0x7fc8380a5790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc850106f10 0x7fc85019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc850102790 0x7fc850198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 --2- 172.21.15.16:0/549401574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc850101de0 0x7fc8501983d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 >> 172.21.15.16:0/549401574 conn(0x7fc8500fd8c0 msgr2=0x7fc8500febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 shutdown_connections 2024-09-15T22:37:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:08.828+0000 7fc854a48640 1 -- 172.21.15.16:0/549401574 wait complete. 2024-09-15T22:37:08.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:08 smithi059 ceph-mon[34872]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-15T22:37:09.647 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-15T22:37:09.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:37:09.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:09 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/549401574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:09.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:09 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/549401574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:09.906 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:09.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:09 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/549401574' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.368+0000 7f2f09640640 1 -- 172.21.15.16:0/3061190210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 msgr2=0x7f2f04100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.368+0000 7f2f09640640 1 --2- 172.21.15.16:0/3061190210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f04100ee0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009930 tx=0x7f2ef802a420 comp rx=0 tx=0).stop 2024-09-15T22:37:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 -- 172.21.15.16:0/3061190210 shutdown_connections 2024-09-15T22:37:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 --2- 172.21.15.16:0/3061190210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f04101420 0x7f2f0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 --2- 172.21.15.16:0/3061190210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f04100ee0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 --2- 172.21.15.16:0/3061190210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f04106a80 0x7f2f04106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 -- 172.21.15.16:0/3061190210 >> 172.21.15.16:0/3061190210 conn(0x7f2f040fc7a0 msgr2=0x7f2f040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 -- 172.21.15.16:0/3061190210 shutdown_connections 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 -- 172.21.15.16:0/3061190210 wait complete. 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 Processor -- start 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.369+0000 7f2f09640640 1 -- start start 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f09640640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f041100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f09640640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f04101420 0x7f2f041105e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f09640640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f04106a80 0x7f2f04110b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f09640640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0406b690 con 0x7f2f04100a80 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f09640640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0406b510 con 0x7f2f04101420 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f09640640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2f041121e0 con 0x7f2f04106a80 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f041100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f041100a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38878/0 (socket says 172.21.15.16:38878) 2024-09-15T22:37:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 -- 172.21.15.16:0/1370296806 learned_addr learned my addr 172.21.15.16:0/1370296806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f027fc640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f04101420 0x7f2f041105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f037fe640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f04106a80 0x7f2f04110b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 -- 172.21.15.16:0/1370296806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f04106a80 msgr2=0x7f2f04110b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f04106a80 0x7f2f04110b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 -- 172.21.15.16:0/1370296806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f04101420 msgr2=0x7f2f041105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f04101420 0x7f2f041105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 -- 172.21.15.16:0/1370296806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f041a6910 con 0x7f2f04100a80 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f027fc640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f04101420 0x7f2f041105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f02ffd640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f041100a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400e9f0 tx=0x7f2ef400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2eebfff640 1 -- 172.21.15.16:0/1370296806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef400cd80 con 0x7f2f04100a80 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f041a6c00 con 0x7f2f04100a80 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2eebfff640 1 -- 172.21.15.16:0/1370296806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ef4010910 con 0x7f2f04100a80 2024-09-15T22:37:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.370+0000 7f2eebfff640 1 -- 172.21.15.16:0/1370296806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40188f0 con 0x7f2f04100a80 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.371+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f041a7160 con 0x7f2f04100a80 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.371+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f04100ee0 con 0x7f2f04100a80 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.373+0000 7f2eebfff640 1 -- 172.21.15.16:0/1370296806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2ef4010ac0 con 0x7f2f04100a80 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.373+0000 7f2eebfff640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2eec0a32d0 0x7f2eec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.373+0000 7f2f027fc640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2eec0a32d0 0x7f2eec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.373+0000 7f2eebfff640 1 -- 172.21.15.16:0/1370296806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f2ef40d0550 con 0x7f2f04100a80 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.374+0000 7f2f027fc640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2eec0a32d0 0x7f2eec0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006010 tx=0x7f2ef8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:10.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.377+0000 7f2eebfff640 1 -- 172.21.15.16:0/1370296806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ef409ced0 con 0x7f2f04100a80 2024-09-15T22:37:10.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:10 smithi016 ceph-mon[31492]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-15T22:37:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.558+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f0402d7c0 con 0x7f2f04100a80 2024-09-15T22:37:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.560+0000 7f2eebfff640 1 -- 172.21.15.16:0/1370296806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24190 (secure 0 0 0) 0x7f2ef409c7d0 con 0x7f2f04100a80 2024-09-15T22:37:10.561 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:10.561 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":85,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:36:17.885768+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:37:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2eec0a32d0 msgr2=0x7f2eec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2eec0a32d0 0x7f2eec0a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006010 tx=0x7f2ef8005a20 comp rx=0 tx=0).stop 2024-09-15T22:37:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 msgr2=0x7f2f041100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f041100a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400e9f0 tx=0x7f2ef400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 shutdown_connections 2024-09-15T22:37:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2eec0a32d0 0x7f2eec0a5790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f04106a80 0x7f2f04110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f04101420 0x7f2f041105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 --2- 172.21.15.16:0/1370296806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f04100a80 0x7f2f041100a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 >> 172.21.15.16:0/1370296806 conn(0x7f2f040fc7a0 msgr2=0x7f2f040fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 shutdown_connections 2024-09-15T22:37:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:10.562+0000 7f2f09640640 1 -- 172.21.15.16:0/1370296806 wait complete. 2024-09-15T22:37:10.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:10 smithi130 ceph-mon[35742]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-15T22:37:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:10 smithi059 ceph-mon[34872]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-15T22:37:11.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-15T22:37:11.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-15T22:37:11.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-15T22:37:11.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-15T22:37:11.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-15T22:37:11.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-15T22:37:11.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-15T22:37:11.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-15T22:37:11.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-15T22:37:11.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-15T22:37:11.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-15T22:37:11.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-15T22:37:11.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-15T22:37:11.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-15T22:37:11.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-15T22:37:11.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-15T22:37:11.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-15T22:37:11.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-15T22:37:11.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2024-09-15T22:37:11.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-09-15T22:37:11.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-15T22:37:11.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-15T22:37:11.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:11 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1370296806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2024-09-15T22:37:11.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-09-15T22:37:11.811 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-15T22:37:11.811 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-15T22:37:11.811 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:11 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1370296806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-15T22:37:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-09-15T22:37:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-09-15T22:37:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-15T22:37:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-15T22:37:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-15T22:37:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2024-09-15T22:37:11.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-09-15T22:37:11.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-09-15T22:37:11.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-15T22:37:11.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:11 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1370296806' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:12.086 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.089 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.090 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.117 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2024-09-15T22:37:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-15T22:37:12.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-15T22:37:12.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-15T22:37:12.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:12 smithi130 ceph-mon[35742]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-15T22:37:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2024-09-15T22:37:12.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-15T22:37:12.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-15T22:37:12.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-15T22:37:12.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:12 smithi016 ceph-mon[31492]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:37:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-15T22:37:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-15T22:37:12.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-09-15T22:37:12.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-09-15T22:37:12.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-15T22:37:12.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-15T22:37:12.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-15T22:37:12.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2024-09-15T22:37:12.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-15T22:37:12.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-09-15T22:37:12.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-15T22:37:12.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:12 smithi059 ceph-mon[34872]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:37:13.682 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:13 smithi016 ceph-mon[31492]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:37:13.682 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:13 smithi016 ceph-mon[31492]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-15T22:37:13.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:13 smithi130 ceph-mon[35742]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:37:13.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:13 smithi130 ceph-mon[35742]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-15T22:37:13.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:13 smithi059 ceph-mon[34872]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:37:13.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:13 smithi059 ceph-mon[34872]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-15T22:37:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 -- 172.21.15.16:0/3878447887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698106f40 msgr2=0x7fe69810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 --2- 172.21.15.16:0/3878447887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698106f40 0x7fe69810eac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe690009900 tx=0x7fe69002a410 comp rx=0 tx=0).stop 2024-09-15T22:37:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 -- 172.21.15.16:0/3878447887 shutdown_connections 2024-09-15T22:37:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 --2- 172.21.15.16:0/3878447887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698106f40 0x7fe69810eac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 --2- 172.21.15.16:0/3878447887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981027c0 0x7fe698106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 --2- 172.21.15.16:0/3878447887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe698101e10 0x7fe6981021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 -- 172.21.15.16:0/3878447887 >> 172.21.15.16:0/3878447887 conn(0x7fe6980fd8d0 msgr2=0x7fe6980ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 -- 172.21.15.16:0/3878447887 shutdown_connections 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.053+0000 7fe6a063e640 1 -- 172.21.15.16:0/3878447887 wait complete. 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 Processor -- start 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 -- start start 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698101e10 0x7fe6980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6981027c0 0x7fe698074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe698106f40 0x7fe698074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111fc0 con 0x7fe698101e10 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111e40 con 0x7fe698106f40 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe6a063e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe698112140 con 0x7fe6981027c0 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe69e3da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698101e10 0x7fe6980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe69e3da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698101e10 0x7fe6980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48752/0 (socket says 172.21.15.16:48752) 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe69e3da640 1 -- 172.21.15.16:0/3748032718 learned_addr learned my addr 172.21.15.16:0/3748032718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe69ebdb640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe698106f40 0x7fe698074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.054+0000 7fe69dbd9640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6981027c0 0x7fe698074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe69e3da640 1 -- 172.21.15.16:0/3748032718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6981027c0 msgr2=0x7fe698074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe69e3da640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6981027c0 0x7fe698074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe69e3da640 1 -- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe698106f40 msgr2=0x7fe698074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe69e3da640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe698106f40 0x7fe698074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe69e3da640 1 -- 172.21.15.16:0/3748032718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6981a6900 con 0x7fe698101e10 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe69e3da640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698101e10 0x7fe6980744f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe68800ba50 tx=0x7fe68800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688019070 con 0x7fe698101e10 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68800cbe0 con 0x7fe698101e10 2024-09-15T22:37:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6880076e0 con 0x7fe698101e10 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6981a6bf0 con 0x7fe698101e10 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.055+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6981a7130 con 0x7fe698101e10 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.056+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6980628c0 con 0x7fe698101e10 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.057+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe68801c070 con 0x7fe698101e10 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.058+0000 7fe6977fe640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6800a3350 0x7fe6800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.058+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe6880cfa50 con 0x7fe698101e10 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.058+0000 7fe69dbd9640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6800a3350 0x7fe6800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.058+0000 7fe6977fe640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] conn(0x7fe6800ae520 0x7fe6800b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.058+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 --> [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6800b1090 con 0x7fe6800ae520 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.058+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe6880cfea0 con 0x7fe698101e10 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.059+0000 7fe69dbd9640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6800a3350 0x7fe6800a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005fd0 tx=0x7fe68c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:14.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.059+0000 7fe69ebdb640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] conn(0x7fe6800ae520 0x7fe6800b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.059+0000 7fe69ebdb640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] conn(0x7fe6800ae520 0x7fe6800b09c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.060+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== osd.1 v2:172.21.15.59:6800/1149325299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6800b1090 con 0x7fe6800ae520 2024-09-15T22:37:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.074+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 --> [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe698062cc0 con 0x7fe6800ae520 2024-09-15T22:37:14.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.074+0000 7fe6977fe640 1 -- 172.21.15.16:0/3748032718 <== osd.1 v2:172.21.15.59:6800/1149325299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe698062cc0 con 0x7fe6800ae520 2024-09-15T22:37:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] conn(0x7fe6800ae520 msgr2=0x7fe6800b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] conn(0x7fe6800ae520 0x7fe6800b09c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6800a3350 msgr2=0x7fe6800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6800a3350 0x7fe6800a5810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005fd0 tx=0x7fe68c005850 comp rx=0 tx=0).stop 2024-09-15T22:37:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698101e10 msgr2=0x7fe6980744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 --2- 172.21.15.16:0/3748032718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe698101e10 0x7fe6980744f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe68800ba50 tx=0x7fe68800bf10 comp rx=0 tx=0).stop 2024-09-15T22:37:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe69ebdb640 1 -- 172.21.15.16:0/3748032718 reap_dead start 2024-09-15T22:37:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 shutdown_connections 2024-09-15T22:37:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 >> 172.21.15.16:0/3748032718 conn(0x7fe6980fd8d0 msgr2=0x7fe6980fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 shutdown_connections 2024-09-15T22:37:14.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.075+0000 7fe6a063e640 1 -- 172.21.15.16:0/3748032718 wait complete. 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.919+0000 7f236bcfd640 1 -- 172.21.15.16:0/1487109618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 msgr2=0x7f2364101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.919+0000 7f236bcfd640 1 --2- 172.21.15.16:0/1487109618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 0x7f2364101200 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2360009900 tx=0x7f236002a480 comp rx=0 tx=0).stop 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 -- 172.21.15.16:0/1487109618 shutdown_connections 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 --2- 172.21.15.16:0/1487109618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2364102ef0 0x7f236410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 --2- 172.21.15.16:0/1487109618 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2364102570 0x7f23641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 --2- 172.21.15.16:0/1487109618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 0x7f2364101200 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 -- 172.21.15.16:0/1487109618 >> 172.21.15.16:0/1487109618 conn(0x7f23640fc7d0 msgr2=0x7f23640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 -- 172.21.15.16:0/1487109618 shutdown_connections 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 -- 172.21.15.16:0/1487109618 wait complete. 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 Processor -- start 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.920+0000 7f236bcfd640 1 -- start start 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236bcfd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 0x7f2364103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236bcfd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2364102570 0x7f2364103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236bcfd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2364102ef0 0x7f23641041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236bcfd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2364110f80 con 0x7f2364100e20 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236bcfd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2364110e00 con 0x7f2364102ef0 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236bcfd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2364111100 con 0x7f2364102570 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 0x7f2364103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 0x7f2364103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48788/0 (socket says 172.21.15.16:48788) 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 -- 172.21.15.16:0/3729243313 learned_addr learned my addr 172.21.15.16:0/3729243313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236a29a640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2364102ef0 0x7f23641041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369298640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2364102570 0x7f2364103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 -- 172.21.15.16:0/3729243313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2364102570 msgr2=0x7f2364103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2364102570 0x7f2364103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 -- 172.21.15.16:0/3729243313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2364102ef0 msgr2=0x7f23641041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2364102ef0 0x7f23641041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 -- 172.21.15.16:0/3729243313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23641a6940 con 0x7f2364100e20 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236a29a640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2364102ef0 0x7f23641041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f2369a99640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 0x7f2364103750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2360006010 tx=0x7f2360003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236003f070 con 0x7f2364100e20 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2360031c30 con 0x7f2364100e20 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360035030 con 0x7f2364100e20 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.921+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23641a6bd0 con 0x7f2364100e20 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.922+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23641a7100 con 0x7f2364100e20 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.922+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2364106bb0 con 0x7f2364100e20 2024-09-15T22:37:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.924+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2360042070 con 0x7f2364100e20 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.924+0000 7f235affd640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f23500a3350 0x7f23500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.924+0000 7f2369298640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f23500a3350 0x7f23500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.924+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f23600f0f10 con 0x7f2364100e20 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.925+0000 7f235affd640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] conn(0x7f23500ae510 0x7f23500b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.925+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 --> [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2360042e50 con 0x7f23500ae510 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.925+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f23600f1360 con 0x7f2364100e20 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.925+0000 7f236a29a640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] conn(0x7f23500ae510 0x7f23500b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.925+0000 7f2369298640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f23500a3350 0x7f23500a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2354005fd0 tx=0x7f2354005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.925+0000 7f236a29a640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] conn(0x7f23500ae510 0x7f23500b09b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.926+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== osd.6 v2:172.21.15.16:6818/2712238672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2360042e50 con 0x7f23500ae510 2024-09-15T22:37:14.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.940+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 --> [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23641021b0 con 0x7f23500ae510 2024-09-15T22:37:14.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.940+0000 7f235affd640 1 -- 172.21.15.16:0/3729243313 <== osd.6 v2:172.21.15.16:6818/2712238672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f23641021b0 con 0x7f23500ae510 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] conn(0x7f23500ae510 msgr2=0x7f23500b09b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] conn(0x7f23500ae510 0x7f23500b09b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f23500a3350 msgr2=0x7f23500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f23500a3350 0x7f23500a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2354005fd0 tx=0x7f2354005850 comp rx=0 tx=0).stop 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 msgr2=0x7f2364103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 --2- 172.21.15.16:0/3729243313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2364100e20 0x7f2364103750 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2360006010 tx=0x7f2360003c10 comp rx=0 tx=0).stop 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236a29a640 1 -- 172.21.15.16:0/3729243313 reap_dead start 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 shutdown_connections 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 >> 172.21.15.16:0/3729243313 conn(0x7f23640fc7d0 msgr2=0x7f236406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:14.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 shutdown_connections 2024-09-15T22:37:14.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.941+0000 7f236bcfd640 1 -- 172.21.15.16:0/3729243313 wait complete. 2024-09-15T22:37:15.612 INFO:teuthology.orchestra.run.smithi016.stdout:180388626484 2024-09-15T22:37:15.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-15T22:37:15.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:15 smithi016 ceph-mon[31492]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1022 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-15T22:37:15.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:15 smithi059 ceph-mon[34872]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1022 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-15T22:37:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 -- 172.21.15.16:0/2629991509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 msgr2=0x7f0d04106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 --2- 172.21.15.16:0/2629991509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 0x7f0d04106e60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009930 tx=0x7f0cf402a420 comp rx=0 tx=0).stop 2024-09-15T22:37:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 -- 172.21.15.16:0/2629991509 shutdown_connections 2024-09-15T22:37:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 --2- 172.21.15.16:0/2629991509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d04101420 0x7f0d0410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 --2- 172.21.15.16:0/2629991509 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d04100a80 0x7f0d04100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 --2- 172.21.15.16:0/2629991509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 0x7f0d04106e60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 -- 172.21.15.16:0/2629991509 >> 172.21.15.16:0/2629991509 conn(0x7f0d040fc7a0 msgr2=0x7f0d040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 -- 172.21.15.16:0/2629991509 shutdown_connections 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.977+0000 7f0d08e38640 1 -- 172.21.15.16:0/2629991509 wait complete. 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 Processor -- start 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 -- start start 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d04100a80 0x7f0d04199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d04101420 0x7f0d04199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 0x7f0d041a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04110ea0 con 0x7f0d04106a80 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04110d20 con 0x7f0d04101420 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.978+0000 7f0d08e38640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0d04111020 con 0x7f0d04100a80 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 0x7f0d041a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 0x7f0d041a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48802/0 (socket says 172.21.15.16:48802) 2024-09-15T22:37:16.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 -- 172.21.15.16:0/989738211 learned_addr learned my addr 172.21.15.16:0/989738211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0cfbfff640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d04101420 0x7f0d04199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d0259c640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d04100a80 0x7f0d04199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 -- 172.21.15.16:0/989738211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d04100a80 msgr2=0x7f0d04199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0d04100a80 0x7f0d04199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 -- 172.21.15.16:0/989738211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d04101420 msgr2=0x7f0d04199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d04101420 0x7f0d04199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 -- 172.21.15.16:0/989738211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d041a48d0 con 0x7f0d04106a80 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d02d9d640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 0x7f0d041a4390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00cce0 tx=0x7f0cfc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00f450 con 0x7f0d04106a80 2024-09-15T22:37:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cfc00fad0 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d041a4aa0 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc018b60 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.979+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d041a4f60 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.980+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d04062900 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.982+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0cfc00fc80 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.982+0000 7f0cfb7fe640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cf00a32d0 0x7f0cf00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.982+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f0cfc0d07d0 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.982+0000 7f0d0259c640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cf00a32d0 0x7f0cf00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.983+0000 7f0cfb7fe640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] conn(0x7f0cf00ae4a0 0x7f0cf00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.983+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 --> [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cfc012ec0 con 0x7f0cf00ae4a0 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.983+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc0d0c20 con 0x7f0d04106a80 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.983+0000 7f0cfbfff640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] conn(0x7f0cf00ae4a0 0x7f0cf00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.983+0000 7f0d0259c640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cf00a32d0 0x7f0cf00a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009900 tx=0x7f0cf40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.983+0000 7f0cfbfff640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] conn(0x7f0cf00ae4a0 0x7f0cf00b0940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.984+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== osd.9 v2:172.21.15.16:6826/2698575932 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cfc012ec0 con 0x7f0cf00ae4a0 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.997+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 --> [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d04062d80 con 0x7f0cf00ae4a0 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0cfb7fe640 1 -- 172.21.15.16:0/989738211 <== osd.9 v2:172.21.15.16:6826/2698575932 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0d04062d80 con 0x7f0cf00ae4a0 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] conn(0x7f0cf00ae4a0 msgr2=0x7f0cf00b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] conn(0x7f0cf00ae4a0 0x7f0cf00b0940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cf00a32d0 msgr2=0x7f0cf00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f0cf00a32d0 0x7f0cf00a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009900 tx=0x7f0cf40058d0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 msgr2=0x7f0d041a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 --2- 172.21.15.16:0/989738211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d04106a80 0x7f0d041a4390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00cce0 tx=0x7f0cfc00a620 comp rx=0 tx=0).stop 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d02d9d640 1 -- 172.21.15.16:0/989738211 reap_dead start 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 shutdown_connections 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 >> 172.21.15.16:0/989738211 conn(0x7f0d040fc7a0 msgr2=0x7f0d040fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 shutdown_connections 2024-09-15T22:37:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:15.998+0000 7f0d08e38640 1 -- 172.21.15.16:0/989738211 wait complete. 2024-09-15T22:37:16.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:15 smithi130 ceph-mon[35742]: pgmap v319: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1022 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-15T22:37:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 -- 172.21.15.16:0/508620055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844101de0 msgr2=0x7fa8441021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 --2- 172.21.15.16:0/508620055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844101de0 0x7fa8441021c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa834009900 tx=0x7fa83402a410 comp rx=0 tx=0).stop 2024-09-15T22:37:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 -- 172.21.15.16:0/508620055 shutdown_connections 2024-09-15T22:37:16.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 --2- 172.21.15.16:0/508620055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa844106f10 0x7fa84410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 --2- 172.21.15.16:0/508620055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa844102790 0x7fa8441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 --2- 172.21.15.16:0/508620055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844101de0 0x7fa8441021c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 -- 172.21.15.16:0/508620055 >> 172.21.15.16:0/508620055 conn(0x7fa8440fd8c0 msgr2=0x7fa8440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 -- 172.21.15.16:0/508620055 shutdown_connections 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.388+0000 7fa848963640 1 -- 172.21.15.16:0/508620055 wait complete. 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 Processor -- start 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 -- start start 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa844101de0 0x7fa844198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844102790 0x7fa844198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa844106f10 0x7fa84419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa844111f70 con 0x7fa844102790 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa844111df0 con 0x7fa844106f10 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.389+0000 7fa848963640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441120f0 con 0x7fa844101de0 2024-09-15T22:37:16.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844102790 0x7fa844198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844102790 0x7fa844198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48826/0 (socket says 172.21.15.16:48826) 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 -- 172.21.15.16:0/2194556229 learned_addr learned my addr 172.21.15.16:0/2194556229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa842d9d640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa844106f10 0x7fa84419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 -- 172.21.15.16:0/2194556229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa844101de0 msgr2=0x7fa844198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa844101de0 0x7fa844198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 -- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa844106f10 msgr2=0x7fa84419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa844106f10 0x7fa84419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 -- 172.21.15.16:0/2194556229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419d3e0 con 0x7fa844102790 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa842d9d640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa844106f10 0x7fa84419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa841d9b640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844102790 0x7fa844198970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa83800d950 tx=0x7fa83800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838017030 con 0x7fa844102790 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa838004020 con 0x7fa844102790 2024-09-15T22:37:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84419d6d0 con 0x7fa844102790 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8380047a0 con 0x7fa844102790 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.390+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a4fb0 con 0x7fa844102790 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.391+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa844062940 con 0x7fa844102790 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.393+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa8380041d0 con 0x7fa844102790 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.393+0000 7fa8337fe640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa82c0a32d0 0x7fa82c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.394+0000 7fa84259c640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa82c0a32d0 0x7fa82c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.394+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa8380cff60 con 0x7fa844102790 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.394+0000 7fa8337fe640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] conn(0x7fa82c0ae4a0 0x7fa82c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.394+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 --> [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa82c0b1010 con 0x7fa82c0ae4a0 2024-09-15T22:37:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.394+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fa8380d03b0 con 0x7fa844102790 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.394+0000 7fa84259c640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa82c0a32d0 0x7fa82c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa83402a930 tx=0x7fa834000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.394+0000 7fa842d9d640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] conn(0x7fa82c0ae4a0 0x7fa82c0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.395+0000 7fa842d9d640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] conn(0x7fa82c0ae4a0 0x7fa82c0b0940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.395+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== osd.7 v2:172.21.15.59:6816/1929682102 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa82c0b1010 con 0x7fa82c0ae4a0 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.409+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 --> [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa844062dc0 con 0x7fa82c0ae4a0 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa8337fe640 1 -- 172.21.15.16:0/2194556229 <== osd.7 v2:172.21.15.59:6816/1929682102 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa844062dc0 con 0x7fa82c0ae4a0 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] conn(0x7fa82c0ae4a0 msgr2=0x7fa82c0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] conn(0x7fa82c0ae4a0 0x7fa82c0b0940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa82c0a32d0 msgr2=0x7fa82c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa82c0a32d0 0x7fa82c0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa83402a930 tx=0x7fa834000bc0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844102790 msgr2=0x7fa844198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 --2- 172.21.15.16:0/2194556229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa844102790 0x7fa844198970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa83800d950 tx=0x7fa83800de10 comp rx=0 tx=0).stop 2024-09-15T22:37:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa842d9d640 1 -- 172.21.15.16:0/2194556229 reap_dead start 2024-09-15T22:37:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 shutdown_connections 2024-09-15T22:37:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 >> 172.21.15.16:0/2194556229 conn(0x7fa8440fd8c0 msgr2=0x7fa8440febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 shutdown_connections 2024-09-15T22:37:16.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:16.410+0000 7fa848963640 1 -- 172.21.15.16:0/2194556229 wait complete. 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 -- 172.21.15.16:0/3025271542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe26010c190 msgr2=0x7fe26010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 --2- 172.21.15.16:0/3025271542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe26010c190 0x7fe26010e600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe258009ca0 tx=0x7fe25802a950 comp rx=0 tx=0).stop 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 -- 172.21.15.16:0/3025271542 shutdown_connections 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 --2- 172.21.15.16:0/3025271542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe26010c190 0x7fe26010e600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 --2- 172.21.15.16:0/3025271542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2601036d0 0x7fe26010bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 --2- 172.21.15.16:0/3025271542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe260102db0 0x7fe260103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 -- 172.21.15.16:0/3025271542 >> 172.21.15.16:0/3025271542 conn(0x7fe2600fc790 msgr2=0x7fe2600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 -- 172.21.15.16:0/3025271542 shutdown_connections 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.826+0000 7fe265dbf640 1 -- 172.21.15.16:0/3025271542 wait complete. 2024-09-15T22:37:16.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 Processor -- start 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 -- start start 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe260102db0 0x7fe260198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2601036d0 0x7fe2601989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe26010c190 0x7fe26019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe260111fa0 con 0x7fe2601036d0 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe260111e20 con 0x7fe26010c190 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe265dbf640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe260112120 con 0x7fe260102db0 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.827+0000 7fe25effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2601036d0 0x7fe2601989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2601036d0 0x7fe2601989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48770/0 (socket says 172.21.15.16:48770) 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 -- 172.21.15.16:0/1426620149 learned_addr learned my addr 172.21.15.16:0/1426620149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25f7fe640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe260102db0 0x7fe260198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25ffff640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe26010c190 0x7fe26019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 -- 172.21.15.16:0/1426620149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe260102db0 msgr2=0x7fe260198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe260102db0 0x7fe260198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 -- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe26010c190 msgr2=0x7fe26019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe26010c190 0x7fe26019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 -- 172.21.15.16:0/1426620149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26019d410 con 0x7fe2601036d0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25effd640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2601036d0 0x7fe2601989a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe25400b4a0 tx=0x7fe25400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254014070 con 0x7fe2601036d0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe254010af0 con 0x7fe2601036d0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26019d700 con 0x7fe2601036d0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254017d10 con 0x7fe2601036d0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.828+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601a4fe0 con 0x7fe2601036d0 2024-09-15T22:37:16.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.829+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe260062940 con 0x7fe2601036d0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.830+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe254017070 con 0x7fe2601036d0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.831+0000 7fe25cff9640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe2480a32d0 0x7fe2480a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.831+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe2540cf450 con 0x7fe2601036d0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.831+0000 7fe25f7fe640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe2480a32d0 0x7fe2480a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.831+0000 7fe25cff9640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] conn(0x7fe2480ae4a0 0x7fe2480b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.831+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 --> [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe25400dec0 con 0x7fe2480ae4a0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.831+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe2540cf8a0 con 0x7fe2601036d0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.832+0000 7fe25f7fe640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe2480a32d0 0x7fe2480a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe25000bbe0 tx=0x7fe25000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.832+0000 7fe25ffff640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] conn(0x7fe2480ae4a0 0x7fe2480b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:16.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.832+0000 7fe25ffff640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] conn(0x7fe2480ae4a0 0x7fe2480b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.833+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== osd.10 v2:172.21.15.59:6824/2888838982 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe25400dec0 con 0x7fe2480ae4a0 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.847+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 --> [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe260062dc0 con 0x7fe2480ae4a0 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.847+0000 7fe25cff9640 1 -- 172.21.15.16:0/1426620149 <== osd.10 v2:172.21.15.59:6824/2888838982 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe260062dc0 con 0x7fe2480ae4a0 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] conn(0x7fe2480ae4a0 msgr2=0x7fe2480b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] conn(0x7fe2480ae4a0 0x7fe2480b0940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe2480a32d0 msgr2=0x7fe2480a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe2480a32d0 0x7fe2480a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe25000bbe0 tx=0x7fe25000b390 comp rx=0 tx=0).stop 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2601036d0 msgr2=0x7fe2601989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 --2- 172.21.15.16:0/1426620149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2601036d0 0x7fe2601989a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe25400b4a0 tx=0x7fe25400b960 comp rx=0 tx=0).stop 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe25ffff640 1 -- 172.21.15.16:0/1426620149 reap_dead start 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 shutdown_connections 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 >> 172.21.15.16:0/1426620149 conn(0x7fe2600fc790 msgr2=0x7fe2600fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 shutdown_connections 2024-09-15T22:37:16.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:14.848+0000 7fe265dbf640 1 -- 172.21.15.16:0/1426620149 wait complete. 2024-09-15T22:37:17.326 INFO:teuthology.orchestra.run.smithi016.stdout:283467841567 2024-09-15T22:37:17.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-15T22:37:17.912 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:17 smithi016 ceph-mon[31492]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:37:17.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:17 smithi059 ceph-mon[34872]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:37:18.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:17 smithi130 ceph-mon[35742]: pgmap v320: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.733+0000 7f799888c640 1 -- 172.21.15.16:0/3326330669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994103660 msgr2=0x7f799410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.733+0000 7f799888c640 1 --2- 172.21.15.16:0/3326330669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994103660 0x7f799410bb50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7988009900 tx=0x7f798802a480 comp rx=0 tx=0).stop 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 -- 172.21.15.16:0/3326330669 shutdown_connections 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 --2- 172.21.15.16:0/3326330669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f799410c170 0x7f799410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 --2- 172.21.15.16:0/3326330669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994103660 0x7f799410bb50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 --2- 172.21.15.16:0/3326330669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7994102d40 0x7f7994103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 -- 172.21.15.16:0/3326330669 >> 172.21.15.16:0/3326330669 conn(0x7f79940fc7a0 msgr2=0x7f79940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 -- 172.21.15.16:0/3326330669 shutdown_connections 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 -- 172.21.15.16:0/3326330669 wait complete. 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 Processor -- start 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.734+0000 7f799888c640 1 -- start start 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799888c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994102d40 0x7f7994198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799888c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7994103660 0x7f7994198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799888c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f799410c170 0x7f799419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799888c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111f70 con 0x7f7994102d40 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799888c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111df0 con 0x7f799410c170 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799888c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f79941120f0 con 0x7f7994103660 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994102d40 0x7f7994198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994102d40 0x7f7994198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48842/0 (socket says 172.21.15.16:48842) 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 -- 172.21.15.16:0/1262136635 learned_addr learned my addr 172.21.15.16:0/1262136635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f7991d9b640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7994103660 0x7f7994198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f7992d9d640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f799410c170 0x7f799419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 -- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7994103660 msgr2=0x7f7994198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7994103660 0x7f7994198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 -- 172.21.15.16:0/1262136635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f799410c170 msgr2=0x7f799419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f799410c170 0x7f799419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 -- 172.21.15.16:0/1262136635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799419d4e0 con 0x7f7994102d40 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799259c640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994102d40 0x7f7994198530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f798400c970 tx=0x7f798400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984007bb0 con 0x7f7994102d40 2024-09-15T22:37:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7984007d50 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799419d7d0 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.735+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798400f450 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.736+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941a50b0 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.736+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7994062900 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.738+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7984004fa0 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.738+0000 7f79837fe640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f797c0a3350 0x7f797c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.739+0000 7f7991d9b640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f797c0a3350 0x7f797c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.739+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f79840d0070 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.739+0000 7f79837fe640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] conn(0x7f797c0ae520 0x7f797c0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.739+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 --> [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f797c0b1090 con 0x7f797c0ae520 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.739+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f79840d0420 con 0x7f7994102d40 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.739+0000 7f7991d9b640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f797c0a3350 0x7f797c0a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f79940fee70 tx=0x7f7988031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.739+0000 7f7992d9d640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] conn(0x7f797c0ae520 0x7f797c0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.740+0000 7f7992d9d640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] conn(0x7f797c0ae520 0x7f797c0b09c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.740+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== osd.8 v2:172.21.15.130:6816/3134922308 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f797c0b1090 con 0x7f797c0ae520 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.754+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 --> [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7994062d80 con 0x7f797c0ae520 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f79837fe640 1 -- 172.21.15.16:0/1262136635 <== osd.8 v2:172.21.15.130:6816/3134922308 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7994062d80 con 0x7f797c0ae520 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] conn(0x7f797c0ae520 msgr2=0x7f797c0b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] conn(0x7f797c0ae520 0x7f797c0b09c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f797c0a3350 msgr2=0x7f797c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f797c0a3350 0x7f797c0a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f79940fee70 tx=0x7f7988031490 comp rx=0 tx=0).stop 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994102d40 msgr2=0x7f7994198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 --2- 172.21.15.16:0/1262136635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7994102d40 0x7f7994198530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f798400c970 tx=0x7f798400ce30 comp rx=0 tx=0).stop 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f7992d9d640 1 -- 172.21.15.16:0/1262136635 reap_dead start 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 shutdown_connections 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 >> 172.21.15.16:0/1262136635 conn(0x7f79940fc7a0 msgr2=0x7f79940fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 shutdown_connections 2024-09-15T22:37:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:17.755+0000 7f799888c640 1 -- 172.21.15.16:0/1262136635 wait complete. 2024-09-15T22:37:19.042 INFO:teuthology.orchestra.run.smithi016.stdout:231928234026 2024-09-15T22:37:19.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 -- 172.21.15.16:0/3502092151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc1037a0 msgr2=0x7f9ffc10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 --2- 172.21.15.16:0/3502092151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc10bc00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009900 tx=0x7f9ff002a410 comp rx=0 tx=0).stop 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 -- 172.21.15.16:0/3502092151 shutdown_connections 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 --2- 172.21.15.16:0/3502092151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ffc10c2f0 0x7f9ffc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 --2- 172.21.15.16:0/3502092151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc10bc00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 --2- 172.21.15.16:0/3502092151 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ffc102df0 0x7f9ffc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 -- 172.21.15.16:0/3502092151 >> 172.21.15.16:0/3502092151 conn(0x7f9ffc0fc7d0 msgr2=0x7f9ffc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 -- 172.21.15.16:0/3502092151 shutdown_connections 2024-09-15T22:37:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.212+0000 7fa000a66640 1 -- 172.21.15.16:0/3502092151 wait complete. 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.213+0000 7fa000a66640 1 Processor -- start 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.213+0000 7fa000a66640 1 -- start start 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7fa000a66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc102df0 0x7f9ffc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7fa000a66640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7fa000a66640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ffc10c2f0 0x7f9ffc19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7fa000a66640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc112150 con 0x7f9ffc102df0 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7fa000a66640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc111fd0 con 0x7f9ffc1037a0 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7fa000a66640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1122d0 con 0x7f9ffc10c2f0 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc102df0 0x7f9ffc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc102df0 0x7f9ffc198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48870/0 (socket says 172.21.15.16:48870) 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 -- 172.21.15.16:0/3964761599 learned_addr learned my addr 172.21.15.16:0/3964761599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 -- 172.21.15.16:0/3964761599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ffc10c2f0 msgr2=0x7f9ffc19cf60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ffc10c2f0 0x7f9ffc19cf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 -- 172.21.15.16:0/3964761599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ffc1037a0 msgr2=0x7f9ffc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ffc1037a0 0x7f9ffc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 -- 172.21.15.16:0/3964761599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc19d640 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.214+0000 7f9ffa59c640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc102df0 0x7f9ffc198690 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b450 tx=0x7f9fec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.215+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec015020 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.215+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc19d930 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.215+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc100980 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.215+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fec011d70 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.215+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00f340 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.216+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9fdc001030 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.218+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9fec018020 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.218+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9fe40a3350 0x7f9fe40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.219+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9fec0cfe30 con 0x7f9ffc102df0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.219+0000 7f9ff9d9b640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9fe40a3350 0x7f9fe40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.219+0000 7f9feb7fe640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] conn(0x7f9fe40ae520 0x7f9fe40b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.219+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 --> [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fe40b1090 con 0x7f9fe40ae520 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.219+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9fec0d0280 con 0x7f9ffc102df0 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.219+0000 7f9ffad9d640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] conn(0x7f9fe40ae520 0x7f9fe40b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.219+0000 7f9ff9d9b640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9fe40a3350 0x7f9fe40a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9ff002a930 tx=0x7f9ff0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.220+0000 7f9ffad9d640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] conn(0x7f9fe40ae520 0x7f9fe40b09c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.220+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== osd.5 v2:172.21.15.130:6808/4126131610 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fe40b1090 con 0x7f9fe40ae520 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.234+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 --> [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9fdc002dd0 con 0x7f9fe40ae520 2024-09-15T22:37:19.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7f9feb7fe640 1 -- 172.21.15.16:0/3964761599 <== osd.5 v2:172.21.15.130:6808/4126131610 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9fdc002dd0 con 0x7f9fe40ae520 2024-09-15T22:37:19.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 >> [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] conn(0x7f9fe40ae520 msgr2=0x7f9fe40b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7fa000a66640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] conn(0x7f9fe40ae520 0x7f9fe40b09c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9fe40a3350 msgr2=0x7f9fe40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7fa000a66640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9fe40a3350 0x7f9fe40a5810 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9ff002a930 tx=0x7f9ff0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc102df0 msgr2=0x7f9ffc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7fa000a66640 1 --2- 172.21.15.16:0/3964761599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ffc102df0 0x7f9ffc198690 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b450 tx=0x7f9fec00b910 comp rx=0 tx=0).stop 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.235+0000 7f9ffad9d640 1 -- 172.21.15.16:0/3964761599 reap_dead start 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.236+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 shutdown_connections 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.236+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 >> 172.21.15.16:0/3964761599 conn(0x7f9ffc0fc7d0 msgr2=0x7f9ffc0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.236+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 shutdown_connections 2024-09-15T22:37:19.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.236+0000 7fa000a66640 1 -- 172.21.15.16:0/3964761599 wait complete. 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 -- 172.21.15.16:0/377660882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 msgr2=0x7f0938106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 --2- 172.21.15.16:0/377660882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 0x7f0938106e60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09280099b0 tx=0x7f092802c650 comp rx=0 tx=0).stop 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 -- 172.21.15.16:0/377660882 shutdown_connections 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 --2- 172.21.15.16:0/377660882 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0938101420 0x7f093810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 --2- 172.21.15.16:0/377660882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0938100a80 0x7f0938100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 --2- 172.21.15.16:0/377660882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 0x7f0938106e60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 -- 172.21.15.16:0/377660882 >> 172.21.15.16:0/377660882 conn(0x7f09380fc7a0 msgr2=0x7f09380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 -- 172.21.15.16:0/377660882 shutdown_connections 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.805+0000 7f093d6cd640 1 -- 172.21.15.16:0/377660882 wait complete. 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 Processor -- start 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 -- start start 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0938100a80 0x7f0938196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0938101420 0x7f0938196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 0x7f093819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f093810fd70 con 0x7f0938106a80 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f093810fbf0 con 0x7f0938101420 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f093d6cd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f093810fef0 con 0x7f0938100a80 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.806+0000 7f09377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 0x7f093819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 0x7f093819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48894/0 (socket says 172.21.15.16:48894) 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 -- 172.21.15.16:0/2700536336 learned_addr learned my addr 172.21.15.16:0/2700536336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f092e7fc640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0938101420 0x7f0938196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f0936ffd640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0938100a80 0x7f0938196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 -- 172.21.15.16:0/2700536336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0938100a80 msgr2=0x7f0938196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0938100a80 0x7f0938196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 -- 172.21.15.16:0/2700536336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0938101420 msgr2=0x7f0938196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0938101420 0x7f0938196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 -- 172.21.15.16:0/2700536336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f093819b1c0 con 0x7f0938106a80 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f092e7fc640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0938101420 0x7f0938196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f09377fe640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 0x7f093819aae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f093000cce0 tx=0x7f093000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093000f450 con 0x7f0938106a80 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f093819b4b0 con 0x7f0938106a80 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f093000fad0 con 0x7f0938106a80 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0930018c10 con 0x7f0938106a80 2024-09-15T22:37:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.807+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09381a2d90 con 0x7f0938106a80 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.808+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0938062980 con 0x7f0938106a80 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.809+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f093000f5f0 con 0x7f0938106a80 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.810+0000 7f0934ff9640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f09240a32d0 0x7f09240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.810+0000 7f0936ffd640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f09240a32d0 0x7f09240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.810+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f09300d07d0 con 0x7f0938106a80 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.810+0000 7f0934ff9640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] conn(0x7f09240ae4a0 0x7f09240b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.810+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 --> [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0930015e70 con 0x7f09240ae4a0 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.810+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f09300d0b80 con 0x7f0938106a80 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.811+0000 7f092e7fc640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] conn(0x7f09240ae4a0 0x7f09240b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.811+0000 7f0936ffd640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f09240a32d0 0x7f09240a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0928000c00 tx=0x7f09280334d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.811+0000 7f092e7fc640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] conn(0x7f09240ae4a0 0x7f09240b0940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.812+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== osd.0 v2:172.21.15.16:6802/211991038 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0930015e70 con 0x7f09240ae4a0 2024-09-15T22:37:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.825+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 --> [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0938062e00 con 0x7f09240ae4a0 2024-09-15T22:37:19.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f0934ff9640 1 -- 172.21.15.16:0/2700536336 <== osd.0 v2:172.21.15.16:6802/211991038 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0938062e00 con 0x7f09240ae4a0 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] conn(0x7f09240ae4a0 msgr2=0x7f09240b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] conn(0x7f09240ae4a0 0x7f09240b0940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f09240a32d0 msgr2=0x7f09240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f09240a32d0 0x7f09240a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0928000c00 tx=0x7f09280334d0 comp rx=0 tx=0).stop 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 msgr2=0x7f093819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 --2- 172.21.15.16:0/2700536336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0938106a80 0x7f093819aae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f093000cce0 tx=0x7f093000a620 comp rx=0 tx=0).stop 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f09377fe640 1 -- 172.21.15.16:0/2700536336 reap_dead start 2024-09-15T22:37:19.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 shutdown_connections 2024-09-15T22:37:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.826+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 >> 172.21.15.16:0/2700536336 conn(0x7f09380fc7a0 msgr2=0x7f09380fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.827+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 shutdown_connections 2024-09-15T22:37:19.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:19.827+0000 7f093d6cd640 1 -- 172.21.15.16:0/2700536336 wait complete. 2024-09-15T22:37:19.931 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:19 smithi016 ceph-mon[31492]: pgmap v321: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:37:19.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:19 smithi059 ceph-mon[34872]: pgmap v321: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:37:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:19 smithi130 ceph-mon[35742]: pgmap v321: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 383 B/s wr, 6 op/s; 13 B/s, 1 objects/s recovering 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 -- 172.21.15.16:0/1355938209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 msgr2=0x7fdc9c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 --2- 172.21.15.16:0/1355938209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 0x7fdc9c1067e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009900 tx=0x7fdc8802a410 comp rx=0 tx=0).stop 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 -- 172.21.15.16:0/1355938209 shutdown_connections 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 --2- 172.21.15.16:0/1355938209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc9c106f10 0x7fdc9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 --2- 172.21.15.16:0/1355938209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 0x7fdc9c1067e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 --2- 172.21.15.16:0/1355938209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc9c101de0 0x7fdc9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 -- 172.21.15.16:0/1355938209 >> 172.21.15.16:0/1355938209 conn(0x7fdc9c0fd8c0 msgr2=0x7fdc9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 -- 172.21.15.16:0/1355938209 shutdown_connections 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.567+0000 7fdca2d3d640 1 -- 172.21.15.16:0/1355938209 wait complete. 2024-09-15T22:37:20.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 Processor -- start 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 -- start start 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc9c101de0 0x7fdc9c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 0x7fdc9c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc9c106f10 0x7fdc9c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c111f70 con 0x7fdc9c102790 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c111df0 con 0x7fdc9c106f10 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca2d3d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c1120f0 con 0x7fdc9c101de0 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 0x7fdc9c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 0x7fdc9c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48932/0 (socket says 172.21.15.16:48932) 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 -- 172.21.15.16:0/2144829461 learned_addr learned my addr 172.21.15.16:0/2144829461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca0ad9640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc9c101de0 0x7fdc9c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:20.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca12da640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc9c106f10 0x7fdc9c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 -- 172.21.15.16:0/2144829461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc9c101de0 msgr2=0x7fdc9c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc9c101de0 0x7fdc9c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 -- 172.21.15.16:0/2144829461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc9c106f10 msgr2=0x7fdc9c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc9c106f10 0x7fdc9c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdc9bfff640 1 -- 172.21.15.16:0/2144829461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c19d360 con 0x7fdc9c102790 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca12da640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc9c106f10 0x7fdc9c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.568+0000 7fdca0ad9640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc9c101de0 0x7fdc9c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.569+0000 7fdc9bfff640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 0x7fdc9c1988f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005ab0 tx=0x7fdc880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.569+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8803f030 con 0x7fdc9c102790 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.569+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9c19d5f0 con 0x7fdc9c102790 2024-09-15T22:37:20.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.569+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc88003ce0 con 0x7fdc9c102790 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.569+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88035070 con 0x7fdc9c102790 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.569+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9c1a4f30 con 0x7fdc9c102790 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.570+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc9c062980 con 0x7fdc9c102790 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.571+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdc88042030 con 0x7fdc9c102790 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdc99ffb640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdc8c0a3350 0x7fdc8c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fdc880f09c0 con 0x7fdc9c102790 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdca0ad9640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdc8c0a3350 0x7fdc8c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdc99ffb640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] conn(0x7fdc8c0ae520 0x7fdc8c0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 --> [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc88001d30 con 0x7fdc8c0ae520 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fdc880f0e10 con 0x7fdc9c102790 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdca12da640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] conn(0x7fdc8c0ae520 0x7fdc8c0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.572+0000 7fdca0ad9640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdc8c0a3350 0x7fdc8c0a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc900058c0 tx=0x7fdc90005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.573+0000 7fdca12da640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] conn(0x7fdc8c0ae520 0x7fdc8c0b09c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:20.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.573+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== osd.3 v2:172.21.15.16:6810/1167232164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc88001d30 con 0x7fdc8c0ae520 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.587+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 --> [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc9c062d80 con 0x7fdc8c0ae520 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.587+0000 7fdc99ffb640 1 -- 172.21.15.16:0/2144829461 <== osd.3 v2:172.21.15.16:6810/1167232164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdc9c062d80 con 0x7fdc8c0ae520 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.587+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] conn(0x7fdc8c0ae520 msgr2=0x7fdc8c0b09c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.587+0000 7fdca2d3d640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] conn(0x7fdc8c0ae520 0x7fdc8c0b09c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdc8c0a3350 msgr2=0x7fdc8c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdc8c0a3350 0x7fdc8c0a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc900058c0 tx=0x7fdc90005ec0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 msgr2=0x7fdc9c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 --2- 172.21.15.16:0/2144829461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9c102790 0x7fdc9c1988f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005ab0 tx=0x7fdc880042f0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca12da640 1 -- 172.21.15.16:0/2144829461 reap_dead start 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 shutdown_connections 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 >> 172.21.15.16:0/2144829461 conn(0x7fdc9c0fd8c0 msgr2=0x7fdc9c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 shutdown_connections 2024-09-15T22:37:20.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.588+0000 7fdca2d3d640 1 -- 172.21.15.16:0/2144829461 wait complete. 2024-09-15T22:37:21.337 INFO:teuthology.orchestra.run.smithi016.stdout:55834574925 2024-09-15T22:37:21.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-15T22:37:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.014+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3557734104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 msgr2=0x7fe8181021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.014+0000 7fe81c8fc640 1 --2- 172.21.15.16:0/3557734104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 0x7fe8181021d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe808009900 tx=0x7fe80802a410 comp rx=0 tx=0).stop 2024-09-15T22:37:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3557734104 shutdown_connections 2024-09-15T22:37:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 --2- 172.21.15.16:0/3557734104 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe818106f20 0x7fe81810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 --2- 172.21.15.16:0/3557734104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8181027a0 0x7fe8181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 --2- 172.21.15.16:0/3557734104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 0x7fe8181021d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3557734104 >> 172.21.15.16:0/3557734104 conn(0x7fe8180fd8b0 msgr2=0x7fe8180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3557734104 shutdown_connections 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3557734104 wait complete. 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 Processor -- start 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.015+0000 7fe81c8fc640 1 -- start start 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81c8fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 0x7fe818198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81c8fc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8181027a0 0x7fe818198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81c8fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe818106f20 0x7fe81819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81c8fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe818111f80 con 0x7fe818101df0 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81c8fc640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe818111e00 con 0x7fe818106f20 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81c8fc640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe818112100 con 0x7fe8181027a0 2024-09-15T22:37:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 0x7fe818198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 0x7fe818198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48908/0 (socket says 172.21.15.16:48908) 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 -- 172.21.15.16:0/3442626792 learned_addr learned my addr 172.21.15.16:0/3442626792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe815d9b640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8181027a0 0x7fe818198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe816d9d640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe818106f20 0x7fe81819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 -- 172.21.15.16:0/3442626792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8181027a0 msgr2=0x7fe818198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe8181027a0 0x7fe818198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 -- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe818106f20 msgr2=0x7fe81819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe818106f20 0x7fe81819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 -- 172.21.15.16:0/3442626792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81819d4d0 con 0x7fe818101df0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.016+0000 7fe81659c640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 0x7fe818198520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe808006010 tx=0x7fe8080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.017+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80803f030 con 0x7fe818101df0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.017+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe808003ce0 con 0x7fe818101df0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.017+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81819d760 con 0x7fe818101df0 2024-09-15T22:37:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.017+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808035070 con 0x7fe818101df0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.017+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181a50a0 con 0x7fe818101df0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.018+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe818062900 con 0x7fe818101df0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.019+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe808042030 con 0x7fe818101df0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe8077fe640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe8000a32d0 0x7fe8000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe815d9b640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe8000a32d0 0x7fe8000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe8080f0a10 con 0x7fe818101df0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe8077fe640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] conn(0x7fe8000ae4a0 0x7fe8000b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 --> [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe808040e70 con 0x7fe8000ae4a0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe8080f0dc0 con 0x7fe818101df0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe815d9b640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe8000a32d0 0x7fe8000a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe8180fde10 tx=0x7fe80c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.020+0000 7fe816d9d640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] conn(0x7fe8000ae4a0 0x7fe8000b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.021+0000 7fe816d9d640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] conn(0x7fe8000ae4a0 0x7fe8000b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.022+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== osd.4 v2:172.21.15.59:6808/2983797571 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe808040e70 con 0x7fe8000ae4a0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.035+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 --> [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe818062d00 con 0x7fe8000ae4a0 2024-09-15T22:37:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe8077fe640 1 -- 172.21.15.16:0/3442626792 <== osd.4 v2:172.21.15.59:6808/2983797571 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe818062d00 con 0x7fe8000ae4a0 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] conn(0x7fe8000ae4a0 msgr2=0x7fe8000b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] conn(0x7fe8000ae4a0 0x7fe8000b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe8000a32d0 msgr2=0x7fe8000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe8000a32d0 0x7fe8000a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe8180fde10 tx=0x7fe80c00b390 comp rx=0 tx=0).stop 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 msgr2=0x7fe818198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 --2- 172.21.15.16:0/3442626792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe818101df0 0x7fe818198520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe808006010 tx=0x7fe8080042f0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe816d9d640 1 -- 172.21.15.16:0/3442626792 reap_dead start 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 shutdown_connections 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.036+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 >> 172.21.15.16:0/3442626792 conn(0x7fe8180fd8b0 msgr2=0x7fe8180febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.037+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 shutdown_connections 2024-09-15T22:37:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:20.037+0000 7fe81c8fc640 1 -- 172.21.15.16:0/3442626792 wait complete. 2024-09-15T22:37:21.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:21 smithi059 ceph-mon[34872]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 338 B/s wr, 6 op/s; 12 B/s, 1 objects/s recovering 2024-09-15T22:37:22.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:21 smithi130 ceph-mon[35742]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 338 B/s wr, 6 op/s; 12 B/s, 1 objects/s recovering 2024-09-15T22:37:22.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:21 smithi016 ceph-mon[31492]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 338 B/s wr, 6 op/s; 12 B/s, 1 objects/s recovering 2024-09-15T22:37:22.097 INFO:teuthology.orchestra.run.smithi016.stdout:261993005093 2024-09-15T22:37:22.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-15T22:37:22.374 INFO:teuthology.orchestra.run.smithi016.stdout:201863462959 2024-09-15T22:37:22.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 -- 172.21.15.16:0/3924850593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc01027a0 msgr2=0x7f6bc01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 --2- 172.21.15.16:0/3924850593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc01027a0 0x7f6bc01067f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009900 tx=0x7f6bb002a410 comp rx=0 tx=0).stop 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 -- 172.21.15.16:0/3924850593 shutdown_connections 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 --2- 172.21.15.16:0/3924850593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc0106f20 0x7f6bc010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 --2- 172.21.15.16:0/3924850593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc01027a0 0x7f6bc01067f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 --2- 172.21.15.16:0/3924850593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6bc0101df0 0x7f6bc01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 -- 172.21.15.16:0/3924850593 >> 172.21.15.16:0/3924850593 conn(0x7f6bc00fd8f0 msgr2=0x7f6bc00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 -- 172.21.15.16:0/3924850593 shutdown_connections 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 -- 172.21.15.16:0/3924850593 wait complete. 2024-09-15T22:37:22.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.573+0000 7f6bc790c640 1 Processor -- start 2024-09-15T22:37:22.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc790c640 1 -- start start 2024-09-15T22:37:22.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc790c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc0101df0 0x7f6bc019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:22.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc790c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6bc01027a0 0x7f6bc019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc790c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc0106f20 0x7f6bc019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc790c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0111f80 con 0x7f6bc0106f20 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc790c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0111e00 con 0x7f6bc0101df0 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc790c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc0112100 con 0x7f6bc01027a0 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc0106f20 0x7f6bc019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc0106f20 0x7f6bc019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32778/0 (socket says 172.21.15.16:32778) 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 -- 172.21.15.16:0/1777439023 learned_addr learned my addr 172.21.15.16:0/1777439023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:22.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc56a8640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc0101df0 0x7f6bc019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc4ea7640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6bc01027a0 0x7f6bc019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 -- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6bc01027a0 msgr2=0x7f6bc019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6bc01027a0 0x7f6bc019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 -- 172.21.15.16:0/1777439023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc0101df0 msgr2=0x7f6bc019a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bc0101df0 0x7f6bc019a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc5ea9640 1 -- 172.21.15.16:0/1777439023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc019e700 con 0x7f6bc0106f20 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.574+0000 7f6bc4ea7640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6bc01027a0 0x7f6bc019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.575+0000 7f6bc5ea9640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc0106f20 0x7f6bc019dfd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009fd0 tx=0x7f6bb800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.575+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8017070 con 0x7f6bc0106f20 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.575+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc019e9f0 con 0x7f6bc0106f20 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.575+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb8003e50 con 0x7f6bc0106f20 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.575+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80106c0 con 0x7f6bc0106f20 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.575+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc01a7360 con 0x7f6bc0106f20 2024-09-15T22:37:22.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.576+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bc010e100 con 0x7f6bc0106f20 2024-09-15T22:37:22.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.577+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6bb8003970 con 0x7f6bc0106f20 2024-09-15T22:37:22.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.578+0000 7f6bb67fc640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6bac0a32d0 0x7f6bac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:22.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.578+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6bb80cff80 con 0x7f6bc0106f20 2024-09-15T22:37:22.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.578+0000 7f6bc56a8640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6bac0a32d0 0x7f6bac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:22.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.578+0000 7f6bb67fc640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] conn(0x7f6bac0ae4a0 0x7f6bac0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:22.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.578+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 --> [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bb801de50 con 0x7f6bac0ae4a0 2024-09-15T22:37:22.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.578+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6bb80d0330 con 0x7f6bc0106f20 2024-09-15T22:37:22.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.579+0000 7f6bc56a8640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6bac0a32d0 0x7f6bac0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc005fd0 tx=0x7f6bbc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:22.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.579+0000 7f6bc4ea7640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] conn(0x7f6bac0ae4a0 0x7f6bac0b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.579+0000 7f6bc4ea7640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] conn(0x7f6bac0ae4a0 0x7f6bac0b0940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:22.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.580+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== osd.11 v2:172.21.15.130:6824/790310360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bb801de50 con 0x7f6bac0ae4a0 2024-09-15T22:37:22.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.595+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 --> [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bc01032d0 con 0x7f6bac0ae4a0 2024-09-15T22:37:22.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.595+0000 7f6bb67fc640 1 -- 172.21.15.16:0/1777439023 <== osd.11 v2:172.21.15.130:6824/790310360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bc01032d0 con 0x7f6bac0ae4a0 2024-09-15T22:37:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] conn(0x7f6bac0ae4a0 msgr2=0x7f6bac0b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] conn(0x7f6bac0ae4a0 0x7f6bac0b0940 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6bac0a32d0 msgr2=0x7f6bac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6bac0a32d0 0x7f6bac0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc005fd0 tx=0x7f6bbc00b390 comp rx=0 tx=0).stop 2024-09-15T22:37:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc0106f20 msgr2=0x7f6bc019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 --2- 172.21.15.16:0/1777439023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bc0106f20 0x7f6bc019dfd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8009fd0 tx=0x7f6bb800b260 comp rx=0 tx=0).stop 2024-09-15T22:37:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc5ea9640 1 -- 172.21.15.16:0/1777439023 reap_dead start 2024-09-15T22:37:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 shutdown_connections 2024-09-15T22:37:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 >> 172.21.15.16:0/1777439023 conn(0x7f6bc00fd8f0 msgr2=0x7f6bc00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.596+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 shutdown_connections 2024-09-15T22:37:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:22.597+0000 7f6bc790c640 1 -- 172.21.15.16:0/1777439023 wait complete. 2024-09-15T22:37:23.043 INFO:teuthology.orchestra.run.smithi016.stdout:103079215173 2024-09-15T22:37:23.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-15T22:37:23.498 INFO:teuthology.orchestra.run.smithi016.stdout:34359738452 2024-09-15T22:37:23.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-15T22:37:23.780 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:23 smithi016 ceph-mon[31492]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 304 B/s wr, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 -- 172.21.15.16:0/1909666129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074100e20 msgr2=0x7f2074101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 --2- 172.21.15.16:0/1909666129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074100e20 0x7f2074101200 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2068009900 tx=0x7f206802a480 comp rx=0 tx=0).stop 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 -- 172.21.15.16:0/1909666129 shutdown_connections 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 --2- 172.21.15.16:0/1909666129 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2074102ef0 0x7f207410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 --2- 172.21.15.16:0/1909666129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2074102570 0x7f20741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 --2- 172.21.15.16:0/1909666129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074100e20 0x7f2074101200 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 -- 172.21.15.16:0/1909666129 >> 172.21.15.16:0/1909666129 conn(0x7f20740fc7d0 msgr2=0x7f20740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 -- 172.21.15.16:0/1909666129 shutdown_connections 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.779+0000 7f207b25a640 1 -- 172.21.15.16:0/1909666129 wait complete. 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 Processor -- start 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 -- start start 2024-09-15T22:37:23.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2074100e20 0x7f2074198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2074102570 0x7f20741989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074102ef0 0x7f207419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2074110f80 con 0x7f2074102ef0 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2074110e00 con 0x7f2074100e20 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f207b25a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2074111100 con 0x7f2074102570 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074102ef0 0x7f207419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074102ef0 0x7f207419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32804/0 (socket says 172.21.15.16:32804) 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 -- 172.21.15.16:0/808158967 learned_addr learned my addr 172.21.15.16:0/808158967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f2073fff640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2074102570 0x7f20741989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f2078ff6640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2074100e20 0x7f2074198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 -- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2074102570 msgr2=0x7f20741989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2074102570 0x7f20741989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 -- 172.21.15.16:0/808158967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2074100e20 msgr2=0x7f2074198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2074100e20 0x7f2074198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.780+0000 7f20797f7640 1 -- 172.21.15.16:0/808158967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207419d440 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.781+0000 7f20797f7640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074102ef0 0x7f207419cd60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f206c00cce0 tx=0x7f206c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.781+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c00f450 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.781+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207419d730 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.781+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c00fad0 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.781+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c018c10 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.781+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20741a5010 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.782+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2074106bf0 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.783+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f206c00f5f0 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.784+0000 7f2071ffb640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f20640a32d0 0x7f20640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.784+0000 7f2078ff6640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f20640a32d0 0x7f20640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.784+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f206c0d0700 con 0x7f2074102ef0 2024-09-15T22:37:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.784+0000 7f2071ffb640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] conn(0x7f20640ae490 0x7f20640b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.784+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 --> [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20640b1000 con 0x7f20640ae490 2024-09-15T22:37:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.784+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f206c0d0ab0 con 0x7f2074102ef0 2024-09-15T22:37:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.784+0000 7f2078ff6640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f20640a32d0 0x7f20640a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2068000c00 tx=0x7f20680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.785+0000 7f2073fff640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] conn(0x7f20640ae490 0x7f20640b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.785+0000 7f2073fff640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] conn(0x7f20640ae490 0x7f20640b0930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.786+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== osd.2 v2:172.21.15.130:6800/4062265693 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20640b1000 con 0x7f20640ae490 2024-09-15T22:37:23.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.800+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 --> [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20741021b0 con 0x7f20640ae490 2024-09-15T22:37:23.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.800+0000 7f2071ffb640 1 -- 172.21.15.16:0/808158967 <== osd.2 v2:172.21.15.130:6800/4062265693 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f20741021b0 con 0x7f20640ae490 2024-09-15T22:37:23.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] conn(0x7f20640ae490 msgr2=0x7f20640b0930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] conn(0x7f20640ae490 0x7f20640b0930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f20640a32d0 msgr2=0x7f20640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f20640a32d0 0x7f20640a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2068000c00 tx=0x7f20680058d0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074102ef0 msgr2=0x7f207419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 --2- 172.21.15.16:0/808158967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2074102ef0 0x7f207419cd60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f206c00cce0 tx=0x7f206c00a620 comp rx=0 tx=0).stop 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f20797f7640 1 -- 172.21.15.16:0/808158967 reap_dead start 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 shutdown_connections 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 >> 172.21.15.16:0/808158967 conn(0x7f20740fc7d0 msgr2=0x7f207406a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 shutdown_connections 2024-09-15T22:37:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:23.801+0000 7f207b25a640 1 -- 172.21.15.16:0/808158967 wait complete. 2024-09-15T22:37:23.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:23 smithi059 ceph-mon[34872]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 304 B/s wr, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-15T22:37:24.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:23 smithi130 ceph-mon[35742]: pgmap v323: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 304 B/s wr, 6 op/s; 11 B/s, 0 objects/s recovering 2024-09-15T22:37:24.877 INFO:teuthology.orchestra.run.smithi016.stdout:150323855418 2024-09-15T22:37:24.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-15T22:37:25.290 INFO:teuthology.orchestra.run.smithi016.stdout:133143986239 2024-09-15T22:37:25.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-15T22:37:25.772 INFO:teuthology.orchestra.run.smithi016.stdout:313532612635 2024-09-15T22:37:25.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-15T22:37:25.795 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:25 smithi016 ceph-mon[31492]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:37:25.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:25 smithi059 ceph-mon[34872]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:37:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:25 smithi130 ceph-mon[35742]: pgmap v324: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:37:26.258 INFO:teuthology.orchestra.run.smithi016.stdout:77309411402 2024-09-15T22:37:26.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-15T22:37:26.291 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:26.344 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:26.355 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:27.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 -- 172.21.15.16:0/3966706345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330103740 msgr2=0x7f7330107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 --2- 172.21.15.16:0/3966706345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330103740 0x7f7330107b80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7318009930 tx=0x7f731802a420 comp rx=0 tx=0).stop 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 -- 172.21.15.16:0/3966706345 shutdown_connections 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 --2- 172.21.15.16:0/3966706345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330108290 0x7f733010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 --2- 172.21.15.16:0/3966706345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330103740 0x7f7330107b80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 --2- 172.21.15.16:0/3966706345 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7330102d90 0x7f7330103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 -- 172.21.15.16:0/3966706345 >> 172.21.15.16:0/3966706345 conn(0x7f73300fc7d0 msgr2=0x7f73300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 -- 172.21.15.16:0/3966706345 shutdown_connections 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 -- 172.21.15.16:0/3966706345 wait complete. 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.073+0000 7f73362f5640 1 Processor -- start 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f73362f5640 1 -- start start 2024-09-15T22:37:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f73362f5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330102d90 0x7f73301985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f73362f5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7330103740 0x7f7330198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f73362f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330108290 0x7f733019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f73362f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7330112090 con 0x7f7330108290 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f73362f5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7330111f10 con 0x7f7330102d90 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f73362f5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7330112210 con 0x7f7330103740 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f7334892640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330108290 0x7f733019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330102d90 0x7f73301985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330102d90 0x7f73301985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:39394/0 (socket says 172.21.15.16:39394) 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f7334892640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330108290 0x7f733019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32820/0 (socket says 172.21.15.16:32820) 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 -- 172.21.15.16:0/3387806593 learned_addr learned my addr 172.21.15.16:0/3387806593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f7327fff640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7330103740 0x7f7330198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 -- 172.21.15.16:0/3387806593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7330103740 msgr2=0x7f7330198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7330103740 0x7f7330198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 -- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330108290 msgr2=0x7f733019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330108290 0x7f733019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 -- 172.21.15.16:0/3387806593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733019d580 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f7334892640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330108290 0x7f733019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.074+0000 7f732ffff640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330102d90 0x7f73301985d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f732000e9f0 tx=0x7f732000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.075+0000 7f732dffb640 1 -- 172.21.15.16:0/3387806593 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732000cd80 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.075+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733019d870 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.075+0000 7f732dffb640 1 -- 172.21.15.16:0/3387806593 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7320005500 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.075+0000 7f732dffb640 1 -- 172.21.15.16:0/3387806593 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320010690 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.075+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73301a5150 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.076+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7330103170 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.077+0000 7f732dffb640 1 -- 172.21.15.16:0/3387806593 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f73200046c0 con 0x7f7330102d90 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.077+0000 7f732dffb640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f731c0a32d0 0x7f731c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.078+0000 7f732dffb640 1 -- 172.21.15.16:0/3387806593 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f73200d04c0 con 0x7f7330102d90 2024-09-15T22:37:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.078+0000 7f7327fff640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f731c0a32d0 0x7f731c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.078+0000 7f7327fff640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f731c0a32d0 0x7f731c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7318004770 tx=0x7f73180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.081+0000 7f732dffb640 1 -- 172.21.15.16:0/3387806593 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f732009cd00 con 0x7f7330102d90 2024-09-15T22:37:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.258+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f733010a2a0 con 0x7f7330102d90 2024-09-15T22:37:27.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.258+0000 7f732dffb640 1 -- 172.21.15.16:0/3387806593 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f732009c600 con 0x7f7330102d90 2024-09-15T22:37:27.260 INFO:teuthology.orchestra.run.smithi016.stdout:261993005095 2024-09-15T22:37:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.260+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f731c0a32d0 msgr2=0x7f731c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.260+0000 7f73362f5640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f731c0a32d0 0x7f731c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7318004770 tx=0x7f73180058d0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.260+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330102d90 msgr2=0x7f73301985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.260+0000 7f73362f5640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330102d90 0x7f73301985d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f732000e9f0 tx=0x7f732000edf0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 shutdown_connections 2024-09-15T22:37:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f731c0a32d0 0x7f731c0a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7330108290 0x7f733019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7330103740 0x7f7330198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 --2- 172.21.15.16:0/3387806593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7330102d90 0x7f73301985d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 >> 172.21.15.16:0/3387806593 conn(0x7f73300fc7d0 msgr2=0x7f7330105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 shutdown_connections 2024-09-15T22:37:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.261+0000 7f73362f5640 1 -- 172.21.15.16:0/3387806593 wait complete. 2024-09-15T22:37:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 -- 172.21.15.16:0/1611024453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 msgr2=0x7ff14c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 --2- 172.21.15.16:0/1611024453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c10d950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff144009cd0 tx=0x7ff14402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 -- 172.21.15.16:0/1611024453 shutdown_connections 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 --2- 172.21.15.16:0/1611024453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c10d950 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 --2- 172.21.15.16:0/1611024453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14c100a80 0x7ff14c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 --2- 172.21.15.16:0/1611024453 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff14c106a80 0x7ff14c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 -- 172.21.15.16:0/1611024453 >> 172.21.15.16:0/1611024453 conn(0x7ff14c0fc7a0 msgr2=0x7ff14c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 -- 172.21.15.16:0/1611024453 shutdown_connections 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.280+0000 7ff151aa6640 1 -- 172.21.15.16:0/1611024453 wait complete. 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 Processor -- start 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 -- start start 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14c100a80 0x7ff14c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff14c106a80 0x7ff14c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c110e80 con 0x7ff14c101420 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c110d00 con 0x7ff14c100a80 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff151aa6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c111000 con 0x7ff14c106a80 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff14affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff14affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32834/0 (socket says 172.21.15.16:32834) 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff14affd640 1 -- 172.21.15.16:0/2496936325 learned_addr learned my addr 172.21.15.16:0/2496936325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.281+0000 7ff14b7fe640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14c100a80 0x7ff14c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14bfff640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff14c106a80 0x7ff14c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14b7fe640 1 -- 172.21.15.16:0/2496936325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff14c106a80 msgr2=0x7ff14c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14b7fe640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff14c106a80 0x7ff14c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14b7fe640 1 -- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 msgr2=0x7ff14c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14b7fe640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14b7fe640 1 -- 172.21.15.16:0/2496936325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c19d440 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14affd640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff14b7fe640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14c100a80 0x7ff14c198490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e9f0 tx=0x7ff13c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff148ff9640 1 -- 172.21.15.16:0/2496936325 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c00ccc0 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14c19d730 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff148ff9640 1 -- 172.21.15.16:0/2496936325 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff13c005020 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff148ff9640 1 -- 172.21.15.16:0/2496936325 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c0049e0 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.282+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14c1a5010 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.283+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14c100ee0 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.284+0000 7ff148ff9640 1 -- 172.21.15.16:0/2496936325 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff13c004c00 con 0x7ff14c100a80 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.285+0000 7ff148ff9640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff1340a32d0 0x7ff1340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.285+0000 7ff14affd640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff1340a32d0 0x7ff1340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.285+0000 7ff148ff9640 1 -- 172.21.15.16:0/2496936325 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff13c0cfd90 con 0x7ff14c100a80 2024-09-15T22:37:27.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.286+0000 7ff14affd640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff1340a32d0 0x7ff1340a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff14c105200 tx=0x7ff140005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:27.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.289+0000 7ff148ff9640 1 -- 172.21.15.16:0/2496936325 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff13c09c5f0 con 0x7ff14c100a80 2024-09-15T22:37:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.480+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff14c10a000 con 0x7ff14c100a80 2024-09-15T22:37:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.480+0000 7ff148ff9640 1 -- 172.21.15.16:0/2496936325 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff13c0051d0 con 0x7ff14c100a80 2024-09-15T22:37:27.482 INFO:teuthology.orchestra.run.smithi016.stdout:103079215174 2024-09-15T22:37:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff1340a32d0 msgr2=0x7ff1340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff1340a32d0 0x7ff1340a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff14c105200 tx=0x7ff140005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14c100a80 msgr2=0x7ff14c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14c100a80 0x7ff14c198490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00e9f0 tx=0x7ff13c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 shutdown_connections 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff1340a32d0 0x7ff1340a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff14c106a80 0x7ff14c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14c101420 0x7ff14c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 --2- 172.21.15.16:0/2496936325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14c100a80 0x7ff14c198490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 >> 172.21.15.16:0/2496936325 conn(0x7ff14c0fc7a0 msgr2=0x7ff14c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 shutdown_connections 2024-09-15T22:37:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:27.482+0000 7ff151aa6640 1 -- 172.21.15.16:0/2496936325 wait complete. 2024-09-15T22:37:27.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:27 smithi016 ceph-mon[31492]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:37:27.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:27 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3387806593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:37:27.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:27 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2496936325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:37:28.027 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:27 smithi059 ceph-mon[34872]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:37:28.027 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:27 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3387806593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:37:28.028 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:27 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2496936325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:37:28.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:27 smithi130 ceph-mon[35742]: pgmap v325: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:37:28.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:27 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3387806593' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:37:28.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:27 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2496936325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:37:28.227 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215173 got 103079215174 for osd.3 2024-09-15T22:37:28.228 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.579+0000 7f682e2b3640 1 -- 172.21.15.16:0/3314649720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 msgr2=0x7f682810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.579+0000 7f682e2b3640 1 --2- 172.21.15.16:0/3314649720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f682810d970 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6820009ca0 tx=0x7f682002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.579+0000 7f682e2b3640 1 -- 172.21.15.16:0/3314649720 shutdown_connections 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.579+0000 7f682e2b3640 1 --2- 172.21.15.16:0/3314649720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f682810d970 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.579+0000 7f682e2b3640 1 --2- 172.21.15.16:0/3314649720 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f68281011c0 0x7f6828101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.579+0000 7f682e2b3640 1 --2- 172.21.15.16:0/3314649720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6828105ad0 0x7f6828105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.580+0000 7f682e2b3640 1 -- 172.21.15.16:0/3314649720 >> 172.21.15.16:0/3314649720 conn(0x7f6828077210 msgr2=0x7f68280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.580+0000 7f682e2b3640 1 -- 172.21.15.16:0/3314649720 shutdown_connections 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.580+0000 7f682e2b3640 1 -- 172.21.15.16:0/3314649720 wait complete. 2024-09-15T22:37:28.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.580+0000 7f682e2b3640 1 Processor -- start 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.580+0000 7f682e2b3640 1 -- start start 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682e2b3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68281011c0 0x7f68281983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682e2b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f68281988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682e2b3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6828105ad0 0x7f682819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682e2b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6828111fe0 con 0x7f6828101b60 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682e2b3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6828111e60 con 0x7f68281011c0 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682e2b3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112160 con 0x7f6828105ad0 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f68281988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f68281988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32860/0 (socket says 172.21.15.16:32860) 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 -- 172.21.15.16:0/714291999 learned_addr learned my addr 172.21.15.16:0/714291999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f6827fff640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68281011c0 0x7f68281983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:28.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682c850640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6828105ad0 0x7f682819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 -- 172.21.15.16:0/714291999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6828105ad0 msgr2=0x7f682819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6828105ad0 0x7f682819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 -- 172.21.15.16:0/714291999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68281011c0 msgr2=0x7f68281983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68281011c0 0x7f68281983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 -- 172.21.15.16:0/714291999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682819d350 con 0x7f6828101b60 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f682c850640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6828105ad0 0x7f682819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68277fe640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f68281988e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e9f0 tx=0x7f681c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.581+0000 7f68257fa640 1 -- 172.21.15.16:0/714291999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c00cd80 con 0x7f6828101b60 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.582+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682819d640 con 0x7f6828101b60 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.582+0000 7f68257fa640 1 -- 172.21.15.16:0/714291999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f681c005500 con 0x7f6828101b60 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.582+0000 7f68257fa640 1 -- 172.21.15.16:0/714291999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c010690 con 0x7f6828101b60 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.582+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281a4f20 con 0x7f6828101b60 2024-09-15T22:37:28.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.583+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6828101620 con 0x7f6828101b60 2024-09-15T22:37:28.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.585+0000 7f68257fa640 1 -- 172.21.15.16:0/714291999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f681c0046c0 con 0x7f6828101b60 2024-09-15T22:37:28.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.585+0000 7f68257fa640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f68100a32d0 0x7f68100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:28.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.586+0000 7f6827fff640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f68100a32d0 0x7f68100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:28.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.586+0000 7f68257fa640 1 -- 172.21.15.16:0/714291999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f681c0d04c0 con 0x7f6828101b60 2024-09-15T22:37:28.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.586+0000 7f6827fff640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f68100a32d0 0x7f68100a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6818009bd0 tx=0x7f6818005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:28.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.589+0000 7f68257fa640 1 -- 172.21.15.16:0/714291999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f681c0541b0 con 0x7f6828101b60 2024-09-15T22:37:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.769+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f68281a51b0 con 0x7f6828101b60 2024-09-15T22:37:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.770+0000 7f68257fa640 1 -- 172.21.15.16:0/714291999 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f681c09cd20 con 0x7f6828101b60 2024-09-15T22:37:28.771 INFO:teuthology.orchestra.run.smithi016.stdout:55834574928 2024-09-15T22:37:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f68100a32d0 msgr2=0x7f68100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f68100a32d0 0x7f68100a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6818009bd0 tx=0x7f6818005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 msgr2=0x7f68281988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f68281988e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e9f0 tx=0x7f681c00edf0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 shutdown_connections 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f68100a32d0 0x7f68100a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6828105ad0 0x7f682819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6828101b60 0x7f68281988e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 --2- 172.21.15.16:0/714291999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68281011c0 0x7f68281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 >> 172.21.15.16:0/714291999 conn(0x7f6828077210 msgr2=0x7f6828103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 shutdown_connections 2024-09-15T22:37:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:28.772+0000 7f682e2b3640 1 -- 172.21.15.16:0/714291999 wait complete. 2024-09-15T22:37:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:29.093 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005093 got 261993005095 for osd.9 2024-09-15T22:37:29.093 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:29.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:29 smithi016 ceph-mon[31492]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-15T22:37:29.872 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:29 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/714291999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:37:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.707+0000 7f5ee127f640 1 -- 172.21.15.16:0/1532923555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc10c170 msgr2=0x7f5edc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.707+0000 7f5ee127f640 1 --2- 172.21.15.16:0/1532923555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc10c170 0x7f5edc10e610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009ca0 tx=0x7f5ed402a950 comp rx=0 tx=0).stop 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.707+0000 7f5ee127f640 1 -- 172.21.15.16:0/1532923555 shutdown_connections 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.707+0000 7f5ee127f640 1 --2- 172.21.15.16:0/1532923555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc10c170 0x7f5edc10e610 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.707+0000 7f5ee127f640 1 --2- 172.21.15.16:0/1532923555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5edc103660 0x7f5edc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.707+0000 7f5ee127f640 1 --2- 172.21.15.16:0/1532923555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5edc102d40 0x7f5edc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.707+0000 7f5ee127f640 1 -- 172.21.15.16:0/1532923555 >> 172.21.15.16:0/1532923555 conn(0x7f5edc0fc7a0 msgr2=0x7f5edc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 -- 172.21.15.16:0/1532923555 shutdown_connections 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 -- 172.21.15.16:0/1532923555 wait complete. 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 Processor -- start 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 -- start start 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5edc102d40 0x7f5edc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc103660 0x7f5edc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5edc10c170 0x7f5edc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc111f70 con 0x7f5edc103660 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc111df0 con 0x7f5edc102d40 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ee127f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc1120f0 con 0x7f5edc10c170 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.708+0000 7f5ed3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc103660 0x7f5edc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc103660 0x7f5edc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32884/0 (socket says 172.21.15.16:32884) 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 -- 172.21.15.16:0/354163209 learned_addr learned my addr 172.21.15.16:0/354163209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5edb7fe640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5edc10c170 0x7f5edc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5edaffd640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5edc102d40 0x7f5edc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 -- 172.21.15.16:0/354163209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5edc10c170 msgr2=0x7f5edc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5edc10c170 0x7f5edc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 -- 172.21.15.16:0/354163209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5edc102d40 msgr2=0x7f5edc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5edc102d40 0x7f5edc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 -- 172.21.15.16:0/354163209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc1a6910 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed3fff640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc103660 0x7f5edc100ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400d950 tx=0x7f5ec400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed8ff9640 1 -- 172.21.15.16:0/354163209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4017030 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed8ff9640 1 -- 172.21.15.16:0/354163209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec4004020 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc1a6c00 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ed8ff9640 1 -- 172.21.15.16:0/354163209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40047a0 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.709+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc1a7190 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.710+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc103120 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.712+0000 7f5ed8ff9640 1 -- 172.21.15.16:0/354163209 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5ec40041d0 con 0x7f5edc103660 2024-09-15T22:37:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.712+0000 7f5ed8ff9640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ec80a3350 0x7f5ec80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.712+0000 7f5ed8ff9640 1 -- 172.21.15.16:0/354163209 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f5ec40cfc40 con 0x7f5edc103660 2024-09-15T22:37:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.712+0000 7f5edaffd640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ec80a3350 0x7f5ec80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:29.982 INFO:teuthology.orchestra.run.smithi016.stdout:34359738453 2024-09-15T22:37:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.713+0000 7f5edaffd640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ec80a3350 0x7f5ec80a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009bd0 tx=0x7f5ecc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.716+0000 7f5ed8ff9640 1 -- 172.21.15.16:0/354163209 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ec409c520 con 0x7f5edc103660 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.911+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5edc101ed0 con 0x7f5edc103660 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.911+0000 7f5ed8ff9640 1 -- 172.21.15.16:0/354163209 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5edc101ed0 con 0x7f5edc103660 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ec80a3350 msgr2=0x7f5ec80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ec80a3350 0x7f5ec80a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009bd0 tx=0x7f5ecc005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc103660 msgr2=0x7f5edc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc103660 0x7f5edc100ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400d950 tx=0x7f5ec400de10 comp rx=0 tx=0).stop 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 shutdown_connections 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f5ec80a3350 0x7f5ec80a5810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5edc10c170 0x7f5edc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5edc103660 0x7f5edc100ee0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 --2- 172.21.15.16:0/354163209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5edc102d40 0x7f5edc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.913+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 >> 172.21.15.16:0/354163209 conn(0x7f5edc0fc7a0 msgr2=0x7f5edc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.914+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 shutdown_connections 2024-09-15T22:37:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:29.914+0000 7f5ee127f640 1 -- 172.21.15.16:0/354163209 wait complete. 2024-09-15T22:37:30.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:29 smithi130 ceph-mon[35742]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-15T22:37:30.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:29 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/714291999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:37:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:29 smithi059 ceph-mon[34872]: pgmap v326: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-15T22:37:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:29 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/714291999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:37:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 -- 172.21.15.16:0/926545620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 msgr2=0x7f01201021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 --2- 172.21.15.16:0/926545620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201021c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0110009900 tx=0x7f011002a410 comp rx=0 tx=0).stop 2024-09-15T22:37:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 -- 172.21.15.16:0/926545620 shutdown_connections 2024-09-15T22:37:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 --2- 172.21.15.16:0/926545620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0120106f10 0x7f012010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 --2- 172.21.15.16:0/926545620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0120102790 0x7f01201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 --2- 172.21.15.16:0/926545620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201021c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 -- 172.21.15.16:0/926545620 >> 172.21.15.16:0/926545620 conn(0x7f01200fd8c0 msgr2=0x7f01200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 -- 172.21.15.16:0/926545620 shutdown_connections 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.572+0000 7f01253e8640 1 -- 172.21.15.16:0/926545620 wait complete. 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 Processor -- start 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 -- start start 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0120102790 0x7f0120198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0120106f10 0x7f012019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0120111f70 con 0x7f0120101de0 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0120111df0 con 0x7f0120106f10 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f01253e8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f01201120f0 con 0x7f0120102790 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f011effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f011effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32904/0 (socket says 172.21.15.16:32904) 2024-09-15T22:37:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.573+0000 7f011effd640 1 -- 172.21.15.16:0/370319818 learned_addr learned my addr 172.21.15.16:0/370319818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f0117fff640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0120102790 0x7f0120198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011f7fe640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0120106f10 0x7f012019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011effd640 1 -- 172.21.15.16:0/370319818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0120102790 msgr2=0x7f0120198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011effd640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0120102790 0x7f0120198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011effd640 1 -- 172.21.15.16:0/370319818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0120106f10 msgr2=0x7f012019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011effd640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0120106f10 0x7f012019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011effd640 1 -- 172.21.15.16:0/370319818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012019d380 con 0x7f0120101de0 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011f7fe640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0120106f10 0x7f012019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011effd640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201983d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0110006010 tx=0x7f01100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011cff9640 1 -- 172.21.15.16:0/370319818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011003f030 con 0x7f0120101de0 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012019d610 con 0x7f0120101de0 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011cff9640 1 -- 172.21.15.16:0/370319818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0110003ce0 con 0x7f0120101de0 2024-09-15T22:37:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f011cff9640 1 -- 172.21.15.16:0/370319818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110035070 con 0x7f0120101de0 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.574+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01201a4f50 con 0x7f0120101de0 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.575+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01201032c0 con 0x7f0120101de0 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.576+0000 7f011cff9640 1 -- 172.21.15.16:0/370319818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0110042030 con 0x7f0120101de0 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.577+0000 7f011cff9640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f010c0a32d0 0x7f010c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.577+0000 7f0117fff640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f010c0a32d0 0x7f010c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.577+0000 7f011cff9640 1 -- 172.21.15.16:0/370319818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f01100f0320 con 0x7f0120101de0 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.578+0000 7f0117fff640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f010c0a32d0 0x7f010c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f0108005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.582+0000 7f011cff9640 1 -- 172.21.15.16:0/370319818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01100bcce0 con 0x7f0120101de0 2024-09-15T22:37:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.765+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f012010a0f0 con 0x7f0120101de0 2024-09-15T22:37:30.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.765+0000 7f011cff9640 1 -- 172.21.15.16:0/370319818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f01100bc5e0 con 0x7f0120101de0 2024-09-15T22:37:30.768 INFO:teuthology.orchestra.run.smithi016.stdout:180388626487 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.767+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f010c0a32d0 msgr2=0x7f010c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.767+0000 7f01253e8640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f010c0a32d0 0x7f010c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f0108005850 comp rx=0 tx=0).stop 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.767+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 msgr2=0x7f01201983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.767+0000 7f01253e8640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201983d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0110006010 tx=0x7f01100042f0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 shutdown_connections 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f010c0a32d0 0x7f010c0a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0120106f10 0x7f012019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0120102790 0x7f0120198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 --2- 172.21.15.16:0/370319818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0120101de0 0x7f01201983d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 >> 172.21.15.16:0/370319818 conn(0x7f01200fd8c0 msgr2=0x7f01200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 shutdown_connections 2024-09-15T22:37:30.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:30.768+0000 7f01253e8640 1 -- 172.21.15.16:0/370319818 wait complete. 2024-09-15T22:37:31.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:30 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/354163209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:37:31.048 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:30 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/354163209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:37:31.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:30 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/354163209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:37:31.413 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574928 for osd.1 2024-09-15T22:37:31.413 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2083944229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4102790 msgr2=0x7f19e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2083944229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4102790 0x7f19e41067e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009900 tx=0x7f19d402a410 comp rx=0 tx=0).stop 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2083944229 shutdown_connections 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2083944229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19e4106f10 0x7f19e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2083944229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4102790 0x7f19e41067e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2083944229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19e4101de0 0x7f19e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2083944229 >> 172.21.15.16:0/2083944229 conn(0x7f19e40fd8c0 msgr2=0x7f19e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2083944229 shutdown_connections 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.576+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2083944229 wait complete. 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 Processor -- start 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 -- start start 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19e4101de0 0x7f19e41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19e4102790 0x7f19e4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:31.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4106f10 0x7f19e419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4112010 con 0x7f19e4106f10 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4111e90 con 0x7f19e4102790 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19eb4fd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f19e4112190 con 0x7f19e4101de0 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4106f10 0x7f19e419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4106f10 0x7f19e419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39454/0 (socket says 172.21.15.16:39454) 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 -- 172.21.15.16:0/2172288368 learned_addr learned my addr 172.21.15.16:0/2172288368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e8a98640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19e4102790 0x7f19e4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9299640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19e4101de0 0x7f19e41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 -- 172.21.15.16:0/2172288368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19e4101de0 msgr2=0x7f19e41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19e4101de0 0x7f19e41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 -- 172.21.15.16:0/2172288368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19e4102790 msgr2=0x7f19e4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19e4102790 0x7f19e4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.577+0000 7f19e9a9a640 1 -- 172.21.15.16:0/2172288368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e419d560 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.578+0000 7f19e9a9a640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4106f10 0x7f19e419ce80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b970 tx=0x7f19dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.578+0000 7f19da7fc640 1 -- 172.21.15.16:0/2172288368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc019070 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.578+0000 7f19da7fc640 1 -- 172.21.15.16:0/2172288368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc00cb40 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.578+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e419d850 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.578+0000 7f19da7fc640 1 -- 172.21.15.16:0/2172288368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc007650 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.578+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41a5130 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.579+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e4102240 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.580+0000 7f19da7fc640 1 -- 172.21.15.16:0/2172288368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f19dc01c070 con 0x7f19e4106f10 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.581+0000 7f19da7fc640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f19d00a32d0 0x7f19d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.581+0000 7f19e9299640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f19d00a32d0 0x7f19d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:31.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.581+0000 7f19da7fc640 1 -- 172.21.15.16:0/2172288368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f19dc0cf390 con 0x7f19e4106f10 2024-09-15T22:37:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.581+0000 7f19e9299640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f19d00a32d0 0x7f19d00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19e000a810 tx=0x7f19e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:31.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.585+0000 7f19da7fc640 1 -- 172.21.15.16:0/2172288368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19dc09bd00 con 0x7f19e4106f10 2024-09-15T22:37:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.764+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f19e4109b60 con 0x7f19e4106f10 2024-09-15T22:37:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.764+0000 7f19da7fc640 1 -- 172.21.15.16:0/2172288368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f19dc09b600 con 0x7f19e4106f10 2024-09-15T22:37:31.766 INFO:teuthology.orchestra.run.smithi016.stdout:231928234028 2024-09-15T22:37:31.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f19d00a32d0 msgr2=0x7f19d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:31.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f19d00a32d0 0x7f19d00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f19e000a810 tx=0x7f19e0008040 comp rx=0 tx=0).stop 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4106f10 msgr2=0x7f19e419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4106f10 0x7f19e419ce80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b970 tx=0x7f19dc00be30 comp rx=0 tx=0).stop 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 shutdown_connections 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f19d00a32d0 0x7f19d00a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4106f10 0x7f19e419ce80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19e4102790 0x7f19e4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 --2- 172.21.15.16:0/2172288368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19e4101de0 0x7f19e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 >> 172.21.15.16:0/2172288368 conn(0x7f19e40fd8c0 msgr2=0x7f19e40fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 shutdown_connections 2024-09-15T22:37:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:31.766+0000 7f19eb4fd640 1 -- 172.21.15.16:0/2172288368 wait complete. 2024-09-15T22:37:32.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:31 smithi130 ceph-mon[35742]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:32.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:31 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/370319818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:37:32.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:31 smithi016 ceph-mon[31492]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:31 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/370319818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:37:32.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:31 smithi059 ceph-mon[34872]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:32.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:31 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/370319818' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:37:32.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 -- 172.21.15.16:0/3798961624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 msgr2=0x7f16b4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 --2- 172.21.15.16:0/3798961624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b4100f00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009ca0 tx=0x7f16b002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 -- 172.21.15.16:0/3798961624 shutdown_connections 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 --2- 172.21.15.16:0/3798961624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16b410e500 0x7f16b4110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 --2- 172.21.15.16:0/3798961624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16b4101440 0x7f16b410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 --2- 172.21.15.16:0/3798961624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b4100f00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 -- 172.21.15.16:0/3798961624 >> 172.21.15.16:0/3798961624 conn(0x7f16b40fc7d0 msgr2=0x7f16b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 -- 172.21.15.16:0/3798961624 shutdown_connections 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 -- 172.21.15.16:0/3798961624 wait complete. 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 Processor -- start 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.308+0000 7f16bc122640 1 -- start start 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16bc122640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16bc122640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16b4101440 0x7f16b419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:32.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16bc122640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16b410e500 0x7f16b419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16bc122640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b41142d0 con 0x7f16b4100b20 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16bc122640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4114150 con 0x7f16b410e500 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16bc122640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4114450 con 0x7f16b4101440 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39480/0 (socket says 172.21.15.16:39480) 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 -- 172.21.15.16:0/4283342155 learned_addr learned my addr 172.21.15.16:0/4283342155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16ba6bf640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16b410e500 0x7f16b419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b96bd640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16b4101440 0x7f16b419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 -- 172.21.15.16:0/4283342155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16b4101440 msgr2=0x7f16b419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16b4101440 0x7f16b419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 -- 172.21.15.16:0/4283342155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16b410e500 msgr2=0x7f16b419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16b410e500 0x7f16b419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 -- 172.21.15.16:0/4283342155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b419f700 con 0x7f16b4100b20 2024-09-15T22:37:32.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16ba6bf640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16b410e500 0x7f16b419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16b9ebe640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b419a750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16b0005e30 tx=0x7f16b0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.309+0000 7f16a6ffd640 1 -- 172.21.15.16:0/4283342155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b00316d0 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.310+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b419f990 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.310+0000 7f16a6ffd640 1 -- 172.21.15.16:0/4283342155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16b0031870 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.310+0000 7f16a6ffd640 1 -- 172.21.15.16:0/4283342155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b002fb90 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.310+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41a72d0 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.311+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b406a850 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.312+0000 7f16a6ffd640 1 -- 172.21.15.16:0/4283342155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f16b002fd50 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.312+0000 7f16a6ffd640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f16a00a32d0 0x7f16a00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.312+0000 7f16a6ffd640 1 -- 172.21.15.16:0/4283342155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f16b00f1300 con 0x7f16b4100b20 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.312+0000 7f16b96bd640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f16a00a32d0 0x7f16a00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.313+0000 7f16b96bd640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f16a00a32d0 0x7f16a00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005fd0 tx=0x7f16a8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:32.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.317+0000 7f16a6ffd640 1 -- 172.21.15.16:0/4283342155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16b00bdd10 con 0x7f16b4100b20 2024-09-15T22:37:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.497+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f16b4109600 con 0x7f16b4100b20 2024-09-15T22:37:32.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.497+0000 7f16a6ffd640 1 -- 172.21.15.16:0/4283342155 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f16b00bd610 con 0x7f16b4100b20 2024-09-15T22:37:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:201863462962 2024-09-15T22:37:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.499+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f16a00a32d0 msgr2=0x7f16a00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.499+0000 7f16bc122640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f16a00a32d0 0x7f16a00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f16a8005fd0 tx=0x7f16a8005850 comp rx=0 tx=0).stop 2024-09-15T22:37:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 msgr2=0x7f16b419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b419a750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16b0005e30 tx=0x7f16b0003f90 comp rx=0 tx=0).stop 2024-09-15T22:37:32.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 shutdown_connections 2024-09-15T22:37:32.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f16a00a32d0 0x7f16a00a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16b410e500 0x7f16b419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f16b4101440 0x7f16b419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 --2- 172.21.15.16:0/4283342155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4100b20 0x7f16b419a750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:32.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 >> 172.21.15.16:0/4283342155 conn(0x7f16b40fc7d0 msgr2=0x7f16b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:32.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 shutdown_connections 2024-09-15T22:37:32.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:32.500+0000 7f16bc122640 1 -- 172.21.15.16:0/4283342155 wait complete. 2024-09-15T22:37:32.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:32 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2172288368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:37:32.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:32 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4283342155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:37:33.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:32 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2172288368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:37:33.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:32 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4283342155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:37:33.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:32 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2172288368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:37:33.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:32 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4283342155' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:37:33.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738453 for osd.0 2024-09-15T22:37:33.303 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:33.411 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462959 got 201863462962 for osd.7 2024-09-15T22:37:33.411 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:33.858 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:33 smithi016 ceph-mon[31492]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 -- 172.21.15.16:0/564964234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c101de0 msgr2=0x7f517c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 --2- 172.21.15.16:0/564964234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c101de0 0x7f517c1021c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f516c009900 tx=0x7f516c02a410 comp rx=0 tx=0).stop 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 -- 172.21.15.16:0/564964234 shutdown_connections 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 --2- 172.21.15.16:0/564964234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f517c106f10 0x7f517c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 --2- 172.21.15.16:0/564964234 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f517c102790 0x7f517c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 --2- 172.21.15.16:0/564964234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c101de0 0x7f517c1021c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 -- 172.21.15.16:0/564964234 >> 172.21.15.16:0/564964234 conn(0x7f517c0fd8c0 msgr2=0x7f517c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 -- 172.21.15.16:0/564964234 shutdown_connections 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 -- 172.21.15.16:0/564964234 wait complete. 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.008+0000 7f5181ab2640 1 Processor -- start 2024-09-15T22:37:34.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f5181ab2640 1 -- start start 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f5181ab2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f517c101de0 0x7f517c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f5181ab2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c102790 0x7f517c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f5181ab2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f517c106f10 0x7f517c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f5181ab2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f517c111f70 con 0x7f517c102790 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f5181ab2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f517c111df0 con 0x7f517c101de0 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f5181ab2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1120f0 con 0x7f517c106f10 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c102790 0x7f517c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c102790 0x7f517c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39502/0 (socket says 172.21.15.16:39502) 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 -- 172.21.15.16:0/3278366179 learned_addr learned my addr 172.21.15.16:0/3278366179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517b7fe640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f517c101de0 0x7f517c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517bfff640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f517c106f10 0x7f517c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 -- 172.21.15.16:0/3278366179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f517c106f10 msgr2=0x7f517c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f517c106f10 0x7f517c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 -- 172.21.15.16:0/3278366179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f517c101de0 msgr2=0x7f517c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f517c101de0 0x7f517c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.009+0000 7f517affd640 1 -- 172.21.15.16:0/3278366179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c19d3d0 con 0x7f517c102790 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.010+0000 7f517affd640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c102790 0x7f517c198960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f517000d950 tx=0x7f517000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.010+0000 7f5178ff9640 1 -- 172.21.15.16:0/3278366179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5170017070 con 0x7f517c102790 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.010+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c19d6c0 con 0x7f517c102790 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.010+0000 7f5178ff9640 1 -- 172.21.15.16:0/3278366179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5170009c30 con 0x7f517c102790 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.010+0000 7f5178ff9640 1 -- 172.21.15.16:0/3278366179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517000f6b0 con 0x7f517c102790 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.010+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c1a4fa0 con 0x7f517c102790 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.011+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f517c1021c0 con 0x7f517c102790 2024-09-15T22:37:34.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.012+0000 7f5178ff9640 1 -- 172.21.15.16:0/3278366179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f51700109d0 con 0x7f517c102790 2024-09-15T22:37:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.013+0000 7f5178ff9640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f51640a32d0 0x7f51640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.013+0000 7f517b7fe640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f51640a32d0 0x7f51640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.013+0000 7f5178ff9640 1 -- 172.21.15.16:0/3278366179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f51700cfa30 con 0x7f517c102790 2024-09-15T22:37:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.013+0000 7f517b7fe640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f51640a32d0 0x7f51640a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f516c009230 tx=0x7f516c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:34.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.016+0000 7f5178ff9640 1 -- 172.21.15.16:0/3278366179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5170009de0 con 0x7f517c102790 2024-09-15T22:37:34.024 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:34.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:33 smithi130 ceph-mon[35742]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:34.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:33 smithi059 ceph-mon[34872]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.199+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f517c109b60 con 0x7f517c102790 2024-09-15T22:37:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.199+0000 7f5178ff9640 1 -- 172.21.15.16:0/3278366179 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f517009c290 con 0x7f517c102790 2024-09-15T22:37:34.201 INFO:teuthology.orchestra.run.smithi016.stdout:133143986242 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f51640a32d0 msgr2=0x7f51640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f51640a32d0 0x7f51640a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f516c009230 tx=0x7f516c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c102790 msgr2=0x7f517c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c102790 0x7f517c198960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f517000d950 tx=0x7f517000de10 comp rx=0 tx=0).stop 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 shutdown_connections 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f51640a32d0 0x7f51640a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f517c106f10 0x7f517c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f517c102790 0x7f517c198960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 --2- 172.21.15.16:0/3278366179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f517c101de0 0x7f517c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.201+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 >> 172.21.15.16:0/3278366179 conn(0x7f517c0fd8c0 msgr2=0x7f517c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.202+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 shutdown_connections 2024-09-15T22:37:34.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.202+0000 7f5181ab2640 1 -- 172.21.15.16:0/3278366179 wait complete. 2024-09-15T22:37:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 -- 172.21.15.16:0/3764151478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 msgr2=0x7fa0bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 --2- 172.21.15.16:0/3764151478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1067e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009900 tx=0x7fa0a802a410 comp rx=0 tx=0).stop 2024-09-15T22:37:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 -- 172.21.15.16:0/3764151478 shutdown_connections 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 --2- 172.21.15.16:0/3764151478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc106f10 0x7fa0bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 --2- 172.21.15.16:0/3764151478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1067e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 --2- 172.21.15.16:0/3764151478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0bc101de0 0x7fa0bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 -- 172.21.15.16:0/3764151478 >> 172.21.15.16:0/3764151478 conn(0x7fa0bc0fd8c0 msgr2=0x7fa0bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 -- 172.21.15.16:0/3764151478 shutdown_connections 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 -- 172.21.15.16:0/3764151478 wait complete. 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.678+0000 7fa0c3107640 1 Processor -- start 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c3107640 1 -- start start 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c3107640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0bc101de0 0x7fa0bc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c3107640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c3107640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc106f10 0x7fa0bc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c3107640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc111f80 con 0x7fa0bc102790 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c3107640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc111e00 con 0x7fa0bc106f10 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c3107640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc112100 con 0x7fa0bc101de0 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39518/0 (socket says 172.21.15.16:39518) 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 -- 172.21.15.16:0/125165145 learned_addr learned my addr 172.21.15.16:0/125165145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c16a4640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc106f10 0x7fa0bc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0c0ea3640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0bc101de0 0x7fa0bc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 -- 172.21.15.16:0/125165145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0bc101de0 msgr2=0x7fa0bc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0bc101de0 0x7fa0bc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 -- 172.21.15.16:0/125165145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc106f10 msgr2=0x7fa0bc19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc106f10 0x7fa0bc19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 -- 172.21.15.16:0/125165145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc19d320 con 0x7fa0bc102790 2024-09-15T22:37:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.679+0000 7fa0bbfff640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1988b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8004770 tx=0x7fa0a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.680+0000 7fa0b9ffb640 1 -- 172.21.15.16:0/125165145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a803f070 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.680+0000 7fa0b9ffb640 1 -- 172.21.15.16:0/125165145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a8031ca0 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.680+0000 7fa0b9ffb640 1 -- 172.21.15.16:0/125165145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8035030 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.680+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0bc19d5b0 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.680+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0bc1a4ef0 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.681+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0bc1032c0 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.682+0000 7fa0b9ffb640 1 -- 172.21.15.16:0/125165145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa0a8042070 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.683+0000 7fa0b9ffb640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa0ac0a32d0 0x7fa0ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.683+0000 7fa0c0ea3640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa0ac0a32d0 0x7fa0ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.683+0000 7fa0b9ffb640 1 -- 172.21.15.16:0/125165145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa0a80f0a10 con 0x7fa0bc102790 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.684+0000 7fa0c0ea3640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa0ac0a32d0 0x7fa0ac0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0005e10 tx=0x7fa0b0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:34.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.686+0000 7fa0b9ffb640 1 -- 172.21.15.16:0/125165145 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0a80bd320 con 0x7fa0bc102790 2024-09-15T22:37:34.866 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:34 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3278366179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:37:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.866+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa0bc10a0f0 con 0x7fa0bc102790 2024-09-15T22:37:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.867+0000 7fa0b9ffb640 1 -- 172.21.15.16:0/125165145 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa0a80bcc20 con 0x7fa0bc102790 2024-09-15T22:37:34.868 INFO:teuthology.orchestra.run.smithi016.stdout:283467841571 2024-09-15T22:37:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa0ac0a32d0 msgr2=0x7fa0ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa0ac0a32d0 0x7fa0ac0a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0005e10 tx=0x7fa0b0005da0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 msgr2=0x7fa0bc1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1988b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8004770 tx=0x7fa0a8003c10 comp rx=0 tx=0).stop 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 shutdown_connections 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fa0ac0a32d0 0x7fa0ac0a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0bc106f10 0x7fa0bc19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0bc102790 0x7fa0bc1988b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 --2- 172.21.15.16:0/125165145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa0bc101de0 0x7fa0bc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 >> 172.21.15.16:0/125165145 conn(0x7fa0bc0fd8c0 msgr2=0x7fa0bc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 shutdown_connections 2024-09-15T22:37:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:34.869+0000 7fa0c3107640 1 -- 172.21.15.16:0/125165145 wait complete. 2024-09-15T22:37:35.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:34 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3278366179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:37:35.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986239 got 133143986242 for osd.4 2024-09-15T22:37:35.151 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:35.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:34 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3278366179' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:37:35.543 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626484 got 180388626487 for osd.6 2024-09-15T22:37:35.543 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:35.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:35 smithi016 ceph-mon[31492]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:35.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:35 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/125165145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:37:36.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:35 smithi130 ceph-mon[35742]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:35 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/125165145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:37:36.032 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234026 got 231928234028 for osd.8 2024-09-15T22:37:36.032 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:35 smithi059 ceph-mon[34872]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:35 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/125165145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:37:36.646 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841567 got 283467841571 for osd.10 2024-09-15T22:37:36.647 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/1193883620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a81036e0 msgr2=0x7fe0a8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/1193883620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a81036e0 0x7fe0a8107b90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009930 tx=0x7fe09c02a420 comp rx=0 tx=0).stop 2024-09-15T22:37:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/1193883620 shutdown_connections 2024-09-15T22:37:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/1193883620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0a81082a0 0x7fe0a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/1193883620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a81036e0 0x7fe0a8107b90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/1193883620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0a8102d30 0x7fe0a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/1193883620 >> 172.21.15.16:0/1193883620 conn(0x7fe0a80fc7d0 msgr2=0x7fe0a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/1193883620 shutdown_connections 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.291+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/1193883620 wait complete. 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 Processor -- start 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 -- start start 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a8102d30 0x7fe0a8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0a81036e0 0x7fe0a8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0a81082a0 0x7fe0a8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a81120a0 con 0x7fe0a8102d30 2024-09-15T22:37:37.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8111f20 con 0x7fe0a81082a0 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0ae0e5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a8112220 con 0x7fe0a81036e0 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a8102d30 0x7fe0a8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a8102d30 0x7fe0a8100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39536/0 (socket says 172.21.15.16:39536) 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0a77fe640 1 -- 172.21.15.16:0/2333577393 learned_addr learned my addr 172.21.15.16:0/2333577393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.292+0000 7fe0a7fff640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0a81082a0 0x7fe0a8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a6ffd640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0a81036e0 0x7fe0a8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a77fe640 1 -- 172.21.15.16:0/2333577393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0a81036e0 msgr2=0x7fe0a8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a77fe640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0a81036e0 0x7fe0a8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a77fe640 1 -- 172.21.15.16:0/2333577393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0a81082a0 msgr2=0x7fe0a8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a77fe640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0a81082a0 0x7fe0a8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a77fe640 1 -- 172.21.15.16:0/2333577393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a81a6b20 con 0x7fe0a8102d30 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a77fe640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a8102d30 0x7fe0a8100990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe09800d950 tx=0x7fe09800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a4ff9640 1 -- 172.21.15.16:0/2333577393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe098017070 con 0x7fe0a8102d30 2024-09-15T22:37:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a4ff9640 1 -- 172.21.15.16:0/2333577393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe098009c30 con 0x7fe0a8102d30 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a81a6e10 con 0x7fe0a8102d30 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0a4ff9640 1 -- 172.21.15.16:0/2333577393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09800f6b0 con 0x7fe0a8102d30 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.293+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a81a7270 con 0x7fe0a8102d30 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.294+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0a8103110 con 0x7fe0a8102d30 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.296+0000 7fe0a4ff9640 1 -- 172.21.15.16:0/2333577393 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe0980109d0 con 0x7fe0a8102d30 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.296+0000 7fe0a4ff9640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe0900a32d0 0x7fe0900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.296+0000 7fe0a6ffd640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe0900a32d0 0x7fe0900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.296+0000 7fe0a4ff9640 1 -- 172.21.15.16:0/2333577393 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe0980cf9d0 con 0x7fe0a8102d30 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.297+0000 7fe0a6ffd640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe0900a32d0 0x7fe0900a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009900 tx=0x7fe09c032d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.300+0000 7fe0a4ff9640 1 -- 172.21.15.16:0/2333577393 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe098009de0 con 0x7fe0a8102d30 2024-09-15T22:37:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.480+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe0a810a2b0 con 0x7fe0a8102d30 2024-09-15T22:37:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.480+0000 7fe0a4ff9640 1 -- 172.21.15.16:0/2333577393 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe09809c230 con 0x7fe0a8102d30 2024-09-15T22:37:37.482 INFO:teuthology.orchestra.run.smithi016.stdout:313532612638 2024-09-15T22:37:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.482+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe0900a32d0 msgr2=0x7fe0900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.482+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe0900a32d0 0x7fe0900a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe09c009900 tx=0x7fe09c032d10 comp rx=0 tx=0).stop 2024-09-15T22:37:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.482+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a8102d30 msgr2=0x7fe0a8100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.482+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a8102d30 0x7fe0a8100990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe09800d950 tx=0x7fe09800de10 comp rx=0 tx=0).stop 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 shutdown_connections 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe0900a32d0 0x7fe0900a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0a81082a0 0x7fe0a8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0a81036e0 0x7fe0a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 --2- 172.21.15.16:0/2333577393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0a8102d30 0x7fe0a8100990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 >> 172.21.15.16:0/2333577393 conn(0x7fe0a80fc7d0 msgr2=0x7fe0a8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 shutdown_connections 2024-09-15T22:37:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.483+0000 7fe0ae0e5640 1 -- 172.21.15.16:0/2333577393 wait complete. 2024-09-15T22:37:37.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 -- 172.21.15.16:0/3477921071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608106a90 msgr2=0x7ff608106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 --2- 172.21.15.16:0/3477921071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608106a90 0x7ff608106e70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009930 tx=0x7ff5f802a420 comp rx=0 tx=0).stop 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 -- 172.21.15.16:0/3477921071 shutdown_connections 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 --2- 172.21.15.16:0/3477921071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff608101430 0x7ff60810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 --2- 172.21.15.16:0/3477921071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff608100a90 0x7ff608100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 --2- 172.21.15.16:0/3477921071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608106a90 0x7ff608106e70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 -- 172.21.15.16:0/3477921071 >> 172.21.15.16:0/3477921071 conn(0x7ff6080fc7d0 msgr2=0x7ff6080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 -- 172.21.15.16:0/3477921071 shutdown_connections 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.500+0000 7ff60db87640 1 -- 172.21.15.16:0/3477921071 wait complete. 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.501+0000 7ff60db87640 1 Processor -- start 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff60db87640 1 -- start start 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff60db87640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff608100a90 0x7ff608198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff60db87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608101430 0x7ff6081989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff60db87640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff608106a90 0x7ff60819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff60db87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff608110e90 con 0x7ff608101430 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff60db87640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff608110d10 con 0x7ff608100a90 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff60db87640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff608111010 con 0x7ff608106a90 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608101430 0x7ff6081989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608101430 0x7ff6081989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39564/0 (socket says 172.21.15.16:39564) 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 -- 172.21.15.16:0/955488839 learned_addr learned my addr 172.21.15.16:0/955488839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff6077fe640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff608100a90 0x7ff608198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff607fff640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff608106a90 0x7ff60819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 -- 172.21.15.16:0/955488839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff608106a90 msgr2=0x7ff60819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff608106a90 0x7ff60819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 -- 172.21.15.16:0/955488839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff608100a90 msgr2=0x7ff608198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff608100a90 0x7ff608198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.502+0000 7ff606ffd640 1 -- 172.21.15.16:0/955488839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60819d420 con 0x7ff608101430 2024-09-15T22:37:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.503+0000 7ff606ffd640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608101430 0x7ff6081989b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d950 tx=0x7ff5fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.503+0000 7ff604ff9640 1 -- 172.21.15.16:0/955488839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc017070 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.503+0000 7ff604ff9640 1 -- 172.21.15.16:0/955488839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc009c30 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.503+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60819d710 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.503+0000 7ff604ff9640 1 -- 172.21.15.16:0/955488839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc00f6b0 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.503+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081a4ff0 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.505+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff608100ef0 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.506+0000 7ff604ff9640 1 -- 172.21.15.16:0/955488839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff5fc0109d0 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.507+0000 7ff604ff9640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff5f00a32d0 0x7ff5f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.507+0000 7ff604ff9640 1 -- 172.21.15.16:0/955488839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff5fc0cfd10 con 0x7ff608101430 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.507+0000 7ff6077fe640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff5f00a32d0 0x7ff5f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.508+0000 7ff6077fe640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff5f00a32d0 0x7ff5f00a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.511+0000 7ff604ff9640 1 -- 172.21.15.16:0/955488839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5fc009de0 con 0x7ff608101430 2024-09-15T22:37:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.732+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff60810a040 con 0x7ff608101430 2024-09-15T22:37:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.733+0000 7ff604ff9640 1 -- 172.21.15.16:0/955488839 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff5fc09bf20 con 0x7ff608101430 2024-09-15T22:37:37.735 INFO:teuthology.orchestra.run.smithi016.stdout:77309411405 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff5f00a32d0 msgr2=0x7ff5f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff5f00a32d0 0x7ff5f00a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f80058d0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608101430 msgr2=0x7ff6081989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608101430 0x7ff6081989b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00d950 tx=0x7ff5fc00de10 comp rx=0 tx=0).stop 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 shutdown_connections 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7ff5f00a32d0 0x7ff5f00a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff608106a90 0x7ff60819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff608101430 0x7ff6081989b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 --2- 172.21.15.16:0/955488839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff608100a90 0x7ff608198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 >> 172.21.15.16:0/955488839 conn(0x7ff6080fc7d0 msgr2=0x7ff60806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:37.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 shutdown_connections 2024-09-15T22:37:37.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:37.735+0000 7ff60db87640 1 -- 172.21.15.16:0/955488839 wait complete. 2024-09-15T22:37:37.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:37 smithi059 ceph-mon[34872]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:37.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:37 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2333577393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:37:38.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:37 smithi130 ceph-mon[35742]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:38.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:37 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2333577393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:37:38.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:37 smithi016 ceph-mon[31492]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:38.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:37 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2333577393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:37:38.483 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411405 for osd.2 2024-09-15T22:37:38.484 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:38.889 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612638 for osd.11 2024-09-15T22:37:38.889 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:38.997 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:38 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/955488839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:37:39.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:38 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/955488839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 -- 172.21.15.16:0/1471831186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc1008d0 msgr2=0x7f83cc100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 --2- 172.21.15.16:0/1471831186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc1008d0 0x7f83cc100cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009900 tx=0x7f83bc02a410 comp rx=0 tx=0).stop 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 -- 172.21.15.16:0/1471831186 shutdown_connections 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 --2- 172.21.15.16:0/1471831186 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83cc10e480 0x7f83cc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 --2- 172.21.15.16:0/1471831186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83cc101280 0x7f83cc10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 --2- 172.21.15.16:0/1471831186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc1008d0 0x7f83cc100cb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 -- 172.21.15.16:0/1471831186 >> 172.21.15.16:0/1471831186 conn(0x7f83cc0fc7a0 msgr2=0x7f83cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 -- 172.21.15.16:0/1471831186 shutdown_connections 2024-09-15T22:37:39.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.090+0000 7f83d0c14640 1 -- 172.21.15.16:0/1471831186 wait complete. 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 Processor -- start 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 -- start start 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83cc1008d0 0x7f83cc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc101280 0x7f83cc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83cc10e480 0x7f83cc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc1141b0 con 0x7f83cc101280 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc114030 con 0x7f83cc1008d0 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83d0c14640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc114330 con 0x7f83cc10e480 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc101280 0x7f83cc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83c9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc101280 0x7f83cc19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39572/0 (socket says 172.21.15.16:39572) 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83c9d9b640 1 -- 172.21.15.16:0/4253297418 learned_addr learned my addr 172.21.15.16:0/4253297418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.091+0000 7f83ca59c640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83cc1008d0 0x7f83cc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83cad9d640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83cc10e480 0x7f83cc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c9d9b640 1 -- 172.21.15.16:0/4253297418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83cc10e480 msgr2=0x7f83cc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c9d9b640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83cc10e480 0x7f83cc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c9d9b640 1 -- 172.21.15.16:0/4253297418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83cc1008d0 msgr2=0x7f83cc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c9d9b640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83cc1008d0 0x7f83cc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c9d9b640 1 -- 172.21.15.16:0/4253297418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc19f600 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c9d9b640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc101280 0x7f83cc19ab90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f83b400d950 tx=0x7f83b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c37fe640 1 -- 172.21.15.16:0/4253297418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4017030 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc19f8f0 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c37fe640 1 -- 172.21.15.16:0/4253297418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b4004020 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83c37fe640 1 -- 172.21.15.16:0/4253297418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40047a0 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.092+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc1a71d0 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.093+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc100cb0 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.094+0000 7f83c37fe640 1 -- 172.21.15.16:0/4253297418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f83b40041d0 con 0x7f83cc101280 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.095+0000 7f83c37fe640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f83b80a32d0 0x7f83b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:39.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.095+0000 7f83ca59c640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f83b80a32d0 0x7f83b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.095+0000 7f83c37fe640 1 -- 172.21.15.16:0/4253297418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f83b40cf860 con 0x7f83cc101280 2024-09-15T22:37:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.096+0000 7f83ca59c640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f83b80a32d0 0x7f83b80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009230 tx=0x7f83bc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.099+0000 7f83c37fe640 1 -- 172.21.15.16:0/4253297418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83b409c060 con 0x7f83cc101280 2024-09-15T22:37:39.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:38 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/955488839' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:37:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.275+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f83cc02d7c0 con 0x7f83cc101280 2024-09-15T22:37:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.275+0000 7f83c37fe640 1 -- 172.21.15.16:0/4253297418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f83b409b960 con 0x7f83cc101280 2024-09-15T22:37:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:150323855421 2024-09-15T22:37:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f83b80a32d0 msgr2=0x7f83b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f83b80a32d0 0x7f83b80a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009230 tx=0x7f83bc0058d0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc101280 msgr2=0x7f83cc19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc101280 0x7f83cc19ab90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f83b400d950 tx=0x7f83b400de10 comp rx=0 tx=0).stop 2024-09-15T22:37:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 shutdown_connections 2024-09-15T22:37:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f83b80a32d0 0x7f83b80a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f83cc10e480 0x7f83cc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83cc101280 0x7f83cc19ab90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 --2- 172.21.15.16:0/4253297418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83cc1008d0 0x7f83cc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 >> 172.21.15.16:0/4253297418 conn(0x7f83cc0fc7a0 msgr2=0x7f83cc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 shutdown_connections 2024-09-15T22:37:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:39.277+0000 7f83d0c14640 1 -- 172.21.15.16:0/4253297418 wait complete. 2024-09-15T22:37:39.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:39 smithi016 ceph-mon[31492]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:39.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:39 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4253297418' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:37:39.991 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855418 got 150323855421 for osd.5 2024-09-15T22:37:39.991 DEBUG:teuthology.parallel:result is None 2024-09-15T22:37:39.991 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:37:39.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:37:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:39 smithi130 ceph-mon[35742]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:39 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4253297418' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:37:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:39 smithi059 ceph-mon[34872]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:40.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:39 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4253297418' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:37:40.254 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3293552495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 msgr2=0x7f4a68105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3293552495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a68105e70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a6402a410 comp rx=0 tx=0).stop 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3293552495 shutdown_connections 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3293552495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a68101bc0 0x7f4a6810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3293552495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a68101240 0x7f4a68101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3293552495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a68105e70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3293552495 >> 172.21.15.16:0/3293552495 conn(0x7f4a68077210 msgr2=0x7f4a680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3293552495 shutdown_connections 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.718+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3293552495 wait complete. 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 Processor -- start 2024-09-15T22:37:40.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 -- start start 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a68101240 0x7f4a68197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a68101bc0 0x7f4a68197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a6819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68110f30 con 0x7f4a68105a90 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68110db0 con 0x7f4a68101bc0 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6f3eb640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681110b0 con 0x7f4a68101240 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a6819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a6819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39590/0 (socket says 172.21.15.16:39590) 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 -- 172.21.15.16:0/3901227744 learned_addr learned my addr 172.21.15.16:0/3901227744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 -- 172.21.15.16:0/3901227744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a68101240 msgr2=0x7f4a68197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6c986640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a68101bc0 0x7f4a68197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:40.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d187640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a68101240 0x7f4a68197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a68101240 0x7f4a68197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 -- 172.21.15.16:0/3901227744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a68101bc0 msgr2=0x7f4a68197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a68101bc0 0x7f4a68197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d988640 1 -- 172.21.15.16:0/3901227744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6819c2e0 con 0x7f4a68105a90 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.719+0000 7f4a6d187640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a68101240 0x7f4a68197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.720+0000 7f4a6c986640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a68101bc0 0x7f4a68197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.720+0000 7f4a6d988640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a6819bc00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c970 tx=0x7f4a6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.720+0000 7f4a5a7fc640 1 -- 172.21.15.16:0/3901227744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60007bb0 con 0x7f4a68105a90 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.720+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6819c5d0 con 0x7f4a68105a90 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.720+0000 7f4a5a7fc640 1 -- 172.21.15.16:0/3901227744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a60007d50 con 0x7f4a68105a90 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.720+0000 7f4a5a7fc640 1 -- 172.21.15.16:0/3901227744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000f450 con 0x7f4a68105a90 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.720+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681a3eb0 con 0x7f4a68105a90 2024-09-15T22:37:40.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.721+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a68101680 con 0x7f4a68105a90 2024-09-15T22:37:40.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.722+0000 7f4a5a7fc640 1 -- 172.21.15.16:0/3901227744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f4a60004f60 con 0x7f4a68105a90 2024-09-15T22:37:40.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.723+0000 7f4a5a7fc640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a540a32d0 0x7f4a540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:40.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.723+0000 7f4a6d187640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a540a32d0 0x7f4a540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:40.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.723+0000 7f4a5a7fc640 1 -- 172.21.15.16:0/3901227744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f4a600cfc20 con 0x7f4a68105a90 2024-09-15T22:37:40.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.723+0000 7f4a6d187640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a540a32d0 0x7f4a540a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a64000c00 tx=0x7f4a640041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:40.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.727+0000 7f4a5a7fc640 1 -- 172.21.15.16:0/3901227744 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a60014030 con 0x7f4a68105a90 2024-09-15T22:37:40.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.900+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a681040a0 con 0x7f4a540a32d0 2024-09-15T22:37:40.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.908+0000 7f4a5a7fc640 1 -- 172.21.15.16:0/3901227744 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340355 (secure 0 0 0) 0x7f4a681040a0 con 0x7f4a540a32d0 2024-09-15T22:37:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:40.916 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a540a32d0 msgr2=0x7f4a540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a540a32d0 0x7f4a540a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a64000c00 tx=0x7f4a640041e0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 msgr2=0x7f4a6819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a6819bc00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c970 tx=0x7f4a6000ce30 comp rx=0 tx=0).stop 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 shutdown_connections 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f4a540a32d0 0x7f4a540a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a68105a90 0x7f4a6819bc00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a68101bc0 0x7f4a68197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 --2- 172.21.15.16:0/3901227744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4a68101240 0x7f4a68197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 >> 172.21.15.16:0/3901227744 conn(0x7f4a68077210 msgr2=0x7f4a68103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 shutdown_connections 2024-09-15T22:37:40.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:40.912+0000 7f4a6f3eb640 1 -- 172.21.15.16:0/3901227744 wait complete. 2024-09-15T22:37:41.731 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-15T22:37:40.605773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":477,"num_read_kb":360,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412952,"kb_used_data":13712,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124528872,"statfs":{"total":1151940427776,"available":1151517564928,"internally_reserved":0,"allocated":14041088,"data_stored":7849529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003015"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518743+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:37:12.518743+0000","last_peered":"2024-09-15T22:37:12.518743+0000","last_clean":"2024-09-15T22:37:12.518743+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:37:12.518743+0000","last_undegraded":"2024-09-15T22:37:12.518743+0000","last_fullsized":"2024-09-15T22:37:12.518743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598848+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:37:12.598848+0000","last_peered":"2024-09-15T22:37:12.598848+0000","last_clean":"2024-09-15T22:37:12.598848+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:37:12.598848+0000","last_undegraded":"2024-09-15T22:37:12.598848+0000","last_fullsized":"2024-09-15T22:37:12.598848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519759+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:37:12.519759+0000","last_peered":"2024-09-15T22:37:12.519759+0000","last_clean":"2024-09-15T22:37:12.519759+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:37:12.519759+0000","last_undegraded":"2024-09-15T22:37:12.519759+0000","last_fullsized":"2024-09-15T22:37:12.519759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599158+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:37:12.599158+0000","last_peered":"2024-09-15T22:37:12.599158+0000","last_clean":"2024-09-15T22:37:12.599158+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:37:12.599158+0000","last_undegraded":"2024-09-15T22:37:12.599158+0000","last_fullsized":"2024-09-15T22:37:12.599158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515426+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:37:12.515426+0000","last_peered":"2024-09-15T22:37:12.515426+0000","last_clean":"2024-09-15T22:37:12.515426+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:37:12.515426+0000","last_undegraded":"2024-09-15T22:37:12.515426+0000","last_fullsized":"2024-09-15T22:37:12.515426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600028+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:37:12.600028+0000","last_peered":"2024-09-15T22:37:12.600028+0000","last_clean":"2024-09-15T22:37:12.600028+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:37:12.600028+0000","last_undegraded":"2024-09-15T22:37:12.600028+0000","last_fullsized":"2024-09-15T22:37:12.600028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516022+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:37:12.516022+0000","last_peered":"2024-09-15T22:37:12.516022+0000","last_clean":"2024-09-15T22:37:12.516022+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:37:12.516022+0000","last_undegraded":"2024-09-15T22:37:12.516022+0000","last_fullsized":"2024-09-15T22:37:12.516022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519055+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:37:12.519055+0000","last_peered":"2024-09-15T22:37:12.519055+0000","last_clean":"2024-09-15T22:37:12.519055+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:37:12.519055+0000","last_undegraded":"2024-09-15T22:37:12.519055+0000","last_fullsized":"2024-09-15T22:37:12.519055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515480+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:37:12.515480+0000","last_peered":"2024-09-15T22:37:12.515480+0000","last_clean":"2024-09-15T22:37:12.515480+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:37:12.515480+0000","last_undegraded":"2024-09-15T22:37:12.515480+0000","last_fullsized":"2024-09-15T22:37:12.515480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515974+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:37:12.515974+0000","last_peered":"2024-09-15T22:37:12.515974+0000","last_clean":"2024-09-15T22:37:12.515974+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:37:12.515974+0000","last_undegraded":"2024-09-15T22:37:12.515974+0000","last_fullsized":"2024-09-15T22:37:12.515974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600276+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:37:12.600276+0000","last_peered":"2024-09-15T22:37:12.600276+0000","last_clean":"2024-09-15T22:37:12.600276+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:37:12.600276+0000","last_undegraded":"2024-09-15T22:37:12.600276+0000","last_fullsized":"2024-09-15T22:37:12.600276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515440+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:37:12.515440+0000","last_peered":"2024-09-15T22:37:12.515440+0000","last_clean":"2024-09-15T22:37:12.515440+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:37:12.515440+0000","last_undegraded":"2024-09-15T22:37:12.515440+0000","last_fullsized":"2024-09-15T22:37:12.515440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521970+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:37:12.521970+0000","last_peered":"2024-09-15T22:37:12.521970+0000","last_clean":"2024-09-15T22:37:12.521970+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:37:12.521970+0000","last_undegraded":"2024-09-15T22:37:12.521970+0000","last_fullsized":"2024-09-15T22:37:12.521970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.529608+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:37:12.529608+0000","last_peered":"2024-09-15T22:37:12.529608+0000","last_clean":"2024-09-15T22:37:12.529608+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:37:12.529608+0000","last_undegraded":"2024-09-15T22:37:12.529608+0000","last_fullsized":"2024-09-15T22:37:12.529608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516090+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:37:12.516090+0000","last_peered":"2024-09-15T22:37:12.516090+0000","last_clean":"2024-09-15T22:37:12.516090+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:37:12.516090+0000","last_undegraded":"2024-09-15T22:37:12.516090+0000","last_fullsized":"2024-09-15T22:37:12.516090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598455+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:37:12.598455+0000","last_peered":"2024-09-15T22:37:12.598455+0000","last_clean":"2024-09-15T22:37:12.598455+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:37:12.598455+0000","last_undegraded":"2024-09-15T22:37:12.598455+0000","last_fullsized":"2024-09-15T22:37:12.598455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519055+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:37:12.519055+0000","last_peered":"2024-09-15T22:37:12.519055+0000","last_clean":"2024-09-15T22:37:12.519055+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:37:12.519055+0000","last_undegraded":"2024-09-15T22:37:12.519055+0000","last_fullsized":"2024-09-15T22:37:12.519055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516034+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:37:12.516034+0000","last_peered":"2024-09-15T22:37:12.516034+0000","last_clean":"2024-09-15T22:37:12.516034+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:37:12.516034+0000","last_undegraded":"2024-09-15T22:37:12.516034+0000","last_fullsized":"2024-09-15T22:37:12.516034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518231+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:37:12.518231+0000","last_peered":"2024-09-15T22:37:12.518231+0000","last_clean":"2024-09-15T22:37:12.518231+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:37:12.518231+0000","last_undegraded":"2024-09-15T22:37:12.518231+0000","last_fullsized":"2024-09-15T22:37:12.518231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598647+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:37:12.598647+0000","last_peered":"2024-09-15T22:37:12.598647+0000","last_clean":"2024-09-15T22:37:12.598647+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:37:12.598647+0000","last_undegraded":"2024-09-15T22:37:12.598647+0000","last_fullsized":"2024-09-15T22:37:12.598647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518403+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:37:12.518403+0000","last_peered":"2024-09-15T22:37:12.518403+0000","last_clean":"2024-09-15T22:37:12.518403+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:37:12.518403+0000","last_undegraded":"2024-09-15T22:37:12.518403+0000","last_fullsized":"2024-09-15T22:37:12.518403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515452+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:37:12.515452+0000","last_peered":"2024-09-15T22:37:12.515452+0000","last_clean":"2024-09-15T22:37:12.515452+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:37:12.515452+0000","last_undegraded":"2024-09-15T22:37:12.515452+0000","last_fullsized":"2024-09-15T22:37:12.515452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518868+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:37:12.518868+0000","last_peered":"2024-09-15T22:37:12.518868+0000","last_clean":"2024-09-15T22:37:12.518868+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:37:12.518868+0000","last_undegraded":"2024-09-15T22:37:12.518868+0000","last_fullsized":"2024-09-15T22:37:12.518868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.602394+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:37:12.602394+0000","last_peered":"2024-09-15T22:37:12.602394+0000","last_clean":"2024-09-15T22:37:12.602394+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:37:12.602394+0000","last_undegraded":"2024-09-15T22:37:12.602394+0000","last_fullsized":"2024-09-15T22:37:12.602394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515127+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:37:12.515127+0000","last_peered":"2024-09-15T22:37:12.515127+0000","last_clean":"2024-09-15T22:37:12.515127+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:37:12.515127+0000","last_undegraded":"2024-09-15T22:37:12.515127+0000","last_fullsized":"2024-09-15T22:37:12.515127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516502+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:37:12.516502+0000","last_peered":"2024-09-15T22:37:12.516502+0000","last_clean":"2024-09-15T22:37:12.516502+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:37:12.516502+0000","last_undegraded":"2024-09-15T22:37:12.516502+0000","last_fullsized":"2024-09-15T22:37:12.516502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519963+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:37:12.519963+0000","last_peered":"2024-09-15T22:37:12.519963+0000","last_clean":"2024-09-15T22:37:12.519963+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:37:12.519963+0000","last_undegraded":"2024-09-15T22:37:12.519963+0000","last_fullsized":"2024-09-15T22:37:12.519963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599249+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:37:12.599249+0000","last_peered":"2024-09-15T22:37:12.599249+0000","last_clean":"2024-09-15T22:37:12.599249+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:37:12.599249+0000","last_undegraded":"2024-09-15T22:37:12.599249+0000","last_fullsized":"2024-09-15T22:37:12.599249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518816+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:37:12.518816+0000","last_peered":"2024-09-15T22:37:12.518816+0000","last_clean":"2024-09-15T22:37:12.518816+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:37:12.518816+0000","last_undegraded":"2024-09-15T22:37:12.518816+0000","last_fullsized":"2024-09-15T22:37:12.518816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598520+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:37:12.598520+0000","last_peered":"2024-09-15T22:37:12.598520+0000","last_clean":"2024-09-15T22:37:12.598520+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:37:12.598520+0000","last_undegraded":"2024-09-15T22:37:12.598520+0000","last_fullsized":"2024-09-15T22:37:12.598520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521045+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:37:12.521045+0000","last_peered":"2024-09-15T22:37:12.521045+0000","last_clean":"2024-09-15T22:37:12.521045+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:37:12.521045+0000","last_undegraded":"2024-09-15T22:37:12.521045+0000","last_fullsized":"2024-09-15T22:37:12.521045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516132+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:37:12.516132+0000","last_peered":"2024-09-15T22:37:12.516132+0000","last_clean":"2024-09-15T22:37:12.516132+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:37:12.516132+0000","last_undegraded":"2024-09-15T22:37:12.516132+0000","last_fullsized":"2024-09-15T22:37:12.516132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519131+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:37:12.519131+0000","last_peered":"2024-09-15T22:37:12.519131+0000","last_clean":"2024-09-15T22:37:12.519131+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:37:12.519131+0000","last_undegraded":"2024-09-15T22:37:12.519131+0000","last_fullsized":"2024-09-15T22:37:12.519131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601205+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:37:12.601205+0000","last_peered":"2024-09-15T22:37:12.601205+0000","last_clean":"2024-09-15T22:37:12.601205+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:37:12.601205+0000","last_undegraded":"2024-09-15T22:37:12.601205+0000","last_fullsized":"2024-09-15T22:37:12.601205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518808+0000","last_change":"2024-09-15T22:36:06.335992+0000","last_active":"2024-09-15T22:37:12.518808+0000","last_peered":"2024-09-15T22:37:12.518808+0000","last_clean":"2024-09-15T22:37:12.518808+0000","last_became_active":"2024-09-15T22:36:06.335660+0000","last_became_peered":"2024-09-15T22:36:06.335660+0000","last_unstale":"2024-09-15T22:37:12.518808+0000","last_undegraded":"2024-09-15T22:37:12.518808+0000","last_fullsized":"2024-09-15T22:37:12.518808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:10:12.935878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515459+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:37:12.515459+0000","last_peered":"2024-09-15T22:37:12.515459+0000","last_clean":"2024-09-15T22:37:12.515459+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:37:12.515459+0000","last_undegraded":"2024-09-15T22:37:12.515459+0000","last_fullsized":"2024-09-15T22:37:12.515459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601323+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:37:12.601323+0000","last_peered":"2024-09-15T22:37:12.601323+0000","last_clean":"2024-09-15T22:37:12.601323+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:37:12.601323+0000","last_undegraded":"2024-09-15T22:37:12.601323+0000","last_fullsized":"2024-09-15T22:37:12.601323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522122+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:37:12.522122+0000","last_peered":"2024-09-15T22:37:12.522122+0000","last_clean":"2024-09-15T22:37:12.522122+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:37:12.522122+0000","last_undegraded":"2024-09-15T22:37:12.522122+0000","last_fullsized":"2024-09-15T22:37:12.522122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520753+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:37:12.520753+0000","last_peered":"2024-09-15T22:37:12.520753+0000","last_clean":"2024-09-15T22:37:12.520753+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:37:12.520753+0000","last_undegraded":"2024-09-15T22:37:12.520753+0000","last_fullsized":"2024-09-15T22:37:12.520753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600398+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:37:12.600398+0000","last_peered":"2024-09-15T22:37:12.600398+0000","last_clean":"2024-09-15T22:37:12.600398+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:37:12.600398+0000","last_undegraded":"2024-09-15T22:37:12.600398+0000","last_fullsized":"2024-09-15T22:37:12.600398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:34.459074+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:37:34.459074+0000","last_peered":"2024-09-15T22:37:34.459074+0000","last_clean":"2024-09-15T22:37:34.459074+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:37:34.459074+0000","last_undegraded":"2024-09-15T22:37:34.459074+0000","last_fullsized":"2024-09-15T22:37:34.459074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520499+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:37:12.520499+0000","last_peered":"2024-09-15T22:37:12.520499+0000","last_clean":"2024-09-15T22:37:12.520499+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:37:12.520499+0000","last_undegraded":"2024-09-15T22:37:12.520499+0000","last_fullsized":"2024-09-15T22:37:12.520499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518474+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:37:12.518474+0000","last_peered":"2024-09-15T22:37:12.518474+0000","last_clean":"2024-09-15T22:37:12.518474+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:37:12.518474+0000","last_undegraded":"2024-09-15T22:37:12.518474+0000","last_fullsized":"2024-09-15T22:37:12.518474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521616+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:37:12.521616+0000","last_peered":"2024-09-15T22:37:12.521616+0000","last_clean":"2024-09-15T22:37:12.521616+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:37:12.521616+0000","last_undegraded":"2024-09-15T22:37:12.521616+0000","last_fullsized":"2024-09-15T22:37:12.521616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599978+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:37:12.599978+0000","last_peered":"2024-09-15T22:37:12.599978+0000","last_clean":"2024-09-15T22:37:12.599978+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:37:12.599978+0000","last_undegraded":"2024-09-15T22:37:12.599978+0000","last_fullsized":"2024-09-15T22:37:12.599978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598543+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:37:12.598543+0000","last_peered":"2024-09-15T22:37:12.598543+0000","last_clean":"2024-09-15T22:37:12.598543+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:37:12.598543+0000","last_undegraded":"2024-09-15T22:37:12.598543+0000","last_fullsized":"2024-09-15T22:37:12.598543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515829+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:37:12.515829+0000","last_peered":"2024-09-15T22:37:12.515829+0000","last_clean":"2024-09-15T22:37:12.515829+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:37:12.515829+0000","last_undegraded":"2024-09-15T22:37:12.515829+0000","last_fullsized":"2024-09-15T22:37:12.515829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521468+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:37:12.521468+0000","last_peered":"2024-09-15T22:37:12.521468+0000","last_clean":"2024-09-15T22:37:12.521468+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:37:12.521468+0000","last_undegraded":"2024-09-15T22:37:12.521468+0000","last_fullsized":"2024-09-15T22:37:12.521468+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521456+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:37:12.521456+0000","last_peered":"2024-09-15T22:37:12.521456+0000","last_clean":"2024-09-15T22:37:12.521456+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:37:12.521456+0000","last_undegraded":"2024-09-15T22:37:12.521456+0000","last_fullsized":"2024-09-15T22:37:12.521456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521629+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:37:12.521629+0000","last_peered":"2024-09-15T22:37:12.521629+0000","last_clean":"2024-09-15T22:37:12.521629+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:37:12.521629+0000","last_undegraded":"2024-09-15T22:37:12.521629+0000","last_fullsized":"2024-09-15T22:37:12.521629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520568+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:37:12.520568+0000","last_peered":"2024-09-15T22:37:12.520568+0000","last_clean":"2024-09-15T22:37:12.520568+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:37:12.520568+0000","last_undegraded":"2024-09-15T22:37:12.520568+0000","last_fullsized":"2024-09-15T22:37:12.520568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.603280+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:37:12.603280+0000","last_peered":"2024-09-15T22:37:12.603280+0000","last_clean":"2024-09-15T22:37:12.603280+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:37:12.603280+0000","last_undegraded":"2024-09-15T22:37:12.603280+0000","last_fullsized":"2024-09-15T22:37:12.603280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521424+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:37:12.521424+0000","last_peered":"2024-09-15T22:37:12.521424+0000","last_clean":"2024-09-15T22:37:12.521424+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:37:12.521424+0000","last_undegraded":"2024-09-15T22:37:12.521424+0000","last_fullsized":"2024-09-15T22:37:12.521424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600508+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:37:12.600508+0000","last_peered":"2024-09-15T22:37:12.600508+0000","last_clean":"2024-09-15T22:37:12.600508+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:37:12.600508+0000","last_undegraded":"2024-09-15T22:37:12.600508+0000","last_fullsized":"2024-09-15T22:37:12.600508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514748+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:37:12.514748+0000","last_peered":"2024-09-15T22:37:12.514748+0000","last_clean":"2024-09-15T22:37:12.514748+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:37:12.514748+0000","last_undegraded":"2024-09-15T22:37:12.514748+0000","last_fullsized":"2024-09-15T22:37:12.514748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521667+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:37:12.521667+0000","last_peered":"2024-09-15T22:37:12.521667+0000","last_clean":"2024-09-15T22:37:12.521667+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:37:12.521667+0000","last_undegraded":"2024-09-15T22:37:12.521667+0000","last_fullsized":"2024-09-15T22:37:12.521667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520299+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:37:12.520299+0000","last_peered":"2024-09-15T22:37:12.520299+0000","last_clean":"2024-09-15T22:37:12.520299+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:37:12.520299+0000","last_undegraded":"2024-09-15T22:37:12.520299+0000","last_fullsized":"2024-09-15T22:37:12.520299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599617+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:37:12.599617+0000","last_peered":"2024-09-15T22:37:12.599617+0000","last_clean":"2024-09-15T22:37:12.599617+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:37:12.599617+0000","last_undegraded":"2024-09-15T22:37:12.599617+0000","last_fullsized":"2024-09-15T22:37:12.599617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522385+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:37:12.522385+0000","last_peered":"2024-09-15T22:37:12.522385+0000","last_clean":"2024-09-15T22:37:12.522385+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:37:12.522385+0000","last_undegraded":"2024-09-15T22:37:12.522385+0000","last_fullsized":"2024-09-15T22:37:12.522385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516161+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:37:12.516161+0000","last_peered":"2024-09-15T22:37:12.516161+0000","last_clean":"2024-09-15T22:37:12.516161+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:37:12.516161+0000","last_undegraded":"2024-09-15T22:37:12.516161+0000","last_fullsized":"2024-09-15T22:37:12.516161+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599990+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:37:12.599990+0000","last_peered":"2024-09-15T22:37:12.599990+0000","last_clean":"2024-09-15T22:37:12.599990+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:37:12.599990+0000","last_undegraded":"2024-09-15T22:37:12.599990+0000","last_fullsized":"2024-09-15T22:37:12.599990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600191+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:37:12.600191+0000","last_peered":"2024-09-15T22:37:12.600191+0000","last_clean":"2024-09-15T22:37:12.600191+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:37:12.600191+0000","last_undegraded":"2024-09-15T22:37:12.600191+0000","last_fullsized":"2024-09-15T22:37:12.600191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.535528+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:37:12.535528+0000","last_peered":"2024-09-15T22:37:12.535528+0000","last_clean":"2024-09-15T22:37:12.535528+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:37:12.535528+0000","last_undegraded":"2024-09-15T22:37:12.535528+0000","last_fullsized":"2024-09-15T22:37:12.535528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600316+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:37:12.600316+0000","last_peered":"2024-09-15T22:37:12.600316+0000","last_clean":"2024-09-15T22:37:12.600316+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:37:12.600316+0000","last_undegraded":"2024-09-15T22:37:12.600316+0000","last_fullsized":"2024-09-15T22:37:12.600316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521246+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:37:12.521246+0000","last_peered":"2024-09-15T22:37:12.521246+0000","last_clean":"2024-09-15T22:37:12.521246+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:37:12.521246+0000","last_undegraded":"2024-09-15T22:37:12.521246+0000","last_fullsized":"2024-09-15T22:37:12.521246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:34.459869+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:37:34.459869+0000","last_peered":"2024-09-15T22:37:34.459869+0000","last_clean":"2024-09-15T22:37:34.459869+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:37:34.459869+0000","last_undegraded":"2024-09-15T22:37:34.459869+0000","last_fullsized":"2024-09-15T22:37:34.459869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518502+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:37:12.518502+0000","last_peered":"2024-09-15T22:37:12.518502+0000","last_clean":"2024-09-15T22:37:12.518502+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:37:12.518502+0000","last_undegraded":"2024-09-15T22:37:12.518502+0000","last_fullsized":"2024-09-15T22:37:12.518502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521473+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:37:12.521473+0000","last_peered":"2024-09-15T22:37:12.521473+0000","last_clean":"2024-09-15T22:37:12.521473+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:37:12.521473+0000","last_undegraded":"2024-09-15T22:37:12.521473+0000","last_fullsized":"2024-09-15T22:37:12.521473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600971+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:37:12.600971+0000","last_peered":"2024-09-15T22:37:12.600971+0000","last_clean":"2024-09-15T22:37:12.600971+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:37:12.600971+0000","last_undegraded":"2024-09-15T22:37:12.600971+0000","last_fullsized":"2024-09-15T22:37:12.600971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518349+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:37:12.518349+0000","last_peered":"2024-09-15T22:37:12.518349+0000","last_clean":"2024-09-15T22:37:12.518349+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:37:12.518349+0000","last_undegraded":"2024-09-15T22:37:12.518349+0000","last_fullsized":"2024-09-15T22:37:12.518349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519065+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:37:12.519065+0000","last_peered":"2024-09-15T22:37:12.519065+0000","last_clean":"2024-09-15T22:37:12.519065+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:37:12.519065+0000","last_undegraded":"2024-09-15T22:37:12.519065+0000","last_fullsized":"2024-09-15T22:37:12.519065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519534+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:37:12.519534+0000","last_peered":"2024-09-15T22:37:12.519534+0000","last_clean":"2024-09-15T22:37:12.519534+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:37:12.519534+0000","last_undegraded":"2024-09-15T22:37:12.519534+0000","last_fullsized":"2024-09-15T22:37:12.519534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600099+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:37:12.600099+0000","last_peered":"2024-09-15T22:37:12.600099+0000","last_clean":"2024-09-15T22:37:12.600099+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:37:12.600099+0000","last_undegraded":"2024-09-15T22:37:12.600099+0000","last_fullsized":"2024-09-15T22:37:12.600099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515885+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:37:12.515885+0000","last_peered":"2024-09-15T22:37:12.515885+0000","last_clean":"2024-09-15T22:37:12.515885+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:37:12.515885+0000","last_undegraded":"2024-09-15T22:37:12.515885+0000","last_fullsized":"2024-09-15T22:37:12.515885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:37.464725+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:37:37.464725+0000","last_peered":"2024-09-15T22:37:37.464725+0000","last_clean":"2024-09-15T22:37:37.464725+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:37:37.464725+0000","last_undegraded":"2024-09-15T22:37:37.464725+0000","last_fullsized":"2024-09-15T22:37:37.464725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":115,"num_read_kb":120,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518225+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:37:12.518225+0000","last_peered":"2024-09-15T22:37:12.518225+0000","last_clean":"2024-09-15T22:37:12.518225+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:37:12.518225+0000","last_undegraded":"2024-09-15T22:37:12.518225+0000","last_fullsized":"2024-09-15T22:37:12.518225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519045+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:37:12.519045+0000","last_peered":"2024-09-15T22:37:12.519045+0000","last_clean":"2024-09-15T22:37:12.519045+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:37:12.519045+0000","last_undegraded":"2024-09-15T22:37:12.519045+0000","last_fullsized":"2024-09-15T22:37:12.519045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600452+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:37:12.600452+0000","last_peered":"2024-09-15T22:37:12.600452+0000","last_clean":"2024-09-15T22:37:12.600452+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:37:12.600452+0000","last_undegraded":"2024-09-15T22:37:12.600452+0000","last_fullsized":"2024-09-15T22:37:12.600452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515684+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:37:12.515684+0000","last_peered":"2024-09-15T22:37:12.515684+0000","last_clean":"2024-09-15T22:37:12.515684+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:37:12.515684+0000","last_undegraded":"2024-09-15T22:37:12.515684+0000","last_fullsized":"2024-09-15T22:37:12.515684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515452+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:37:12.515452+0000","last_peered":"2024-09-15T22:37:12.515452+0000","last_clean":"2024-09-15T22:37:12.515452+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:37:12.515452+0000","last_undegraded":"2024-09-15T22:37:12.515452+0000","last_fullsized":"2024-09-15T22:37:12.515452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521901+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:37:12.521901+0000","last_peered":"2024-09-15T22:37:12.521901+0000","last_clean":"2024-09-15T22:37:12.521901+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:37:12.521901+0000","last_undegraded":"2024-09-15T22:37:12.521901+0000","last_fullsized":"2024-09-15T22:37:12.521901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.517877+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:37:12.517877+0000","last_peered":"2024-09-15T22:37:12.517877+0000","last_clean":"2024-09-15T22:37:12.517877+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:37:12.517877+0000","last_undegraded":"2024-09-15T22:37:12.517877+0000","last_fullsized":"2024-09-15T22:37:12.517877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519555+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:37:12.519555+0000","last_peered":"2024-09-15T22:37:12.519555+0000","last_clean":"2024-09-15T22:37:12.519555+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:37:12.519555+0000","last_undegraded":"2024-09-15T22:37:12.519555+0000","last_fullsized":"2024-09-15T22:37:12.519555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:36.461472+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:37:36.461472+0000","last_peered":"2024-09-15T22:37:36.461472+0000","last_clean":"2024-09-15T22:37:36.461472+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:37:36.461472+0000","last_undegraded":"2024-09-15T22:37:36.461472+0000","last_fullsized":"2024-09-15T22:37:36.461472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598378+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:37:12.598378+0000","last_peered":"2024-09-15T22:37:12.598378+0000","last_clean":"2024-09-15T22:37:12.598378+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:37:12.598378+0000","last_undegraded":"2024-09-15T22:37:12.598378+0000","last_fullsized":"2024-09-15T22:37:12.598378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514982+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:37:12.514982+0000","last_peered":"2024-09-15T22:37:12.514982+0000","last_clean":"2024-09-15T22:37:12.514982+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:37:12.514982+0000","last_undegraded":"2024-09-15T22:37:12.514982+0000","last_fullsized":"2024-09-15T22:37:12.514982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518316+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:37:12.518316+0000","last_peered":"2024-09-15T22:37:12.518316+0000","last_clean":"2024-09-15T22:37:12.518316+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:37:12.518316+0000","last_undegraded":"2024-09-15T22:37:12.518316+0000","last_fullsized":"2024-09-15T22:37:12.518316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600692+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:37:12.600692+0000","last_peered":"2024-09-15T22:37:12.600692+0000","last_clean":"2024-09-15T22:37:12.600692+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:37:12.600692+0000","last_undegraded":"2024-09-15T22:37:12.600692+0000","last_fullsized":"2024-09-15T22:37:12.600692+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518697+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:37:12.518697+0000","last_peered":"2024-09-15T22:37:12.518697+0000","last_clean":"2024-09-15T22:37:12.518697+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:37:12.518697+0000","last_undegraded":"2024-09-15T22:37:12.518697+0000","last_fullsized":"2024-09-15T22:37:12.518697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598830+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:37:12.598830+0000","last_peered":"2024-09-15T22:37:12.598830+0000","last_clean":"2024-09-15T22:37:12.598830+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:37:12.598830+0000","last_undegraded":"2024-09-15T22:37:12.598830+0000","last_fullsized":"2024-09-15T22:37:12.598830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518579+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:37:12.518579+0000","last_peered":"2024-09-15T22:37:12.518579+0000","last_clean":"2024-09-15T22:37:12.518579+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:37:12.518579+0000","last_undegraded":"2024-09-15T22:37:12.518579+0000","last_fullsized":"2024-09-15T22:37:12.518579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601419+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:37:12.601419+0000","last_peered":"2024-09-15T22:37:12.601419+0000","last_clean":"2024-09-15T22:37:12.601419+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:37:12.601419+0000","last_undegraded":"2024-09-15T22:37:12.601419+0000","last_fullsized":"2024-09-15T22:37:12.601419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601056+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:37:12.601056+0000","last_peered":"2024-09-15T22:37:12.601056+0000","last_clean":"2024-09-15T22:37:12.601056+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:37:12.601056+0000","last_undegraded":"2024-09-15T22:37:12.601056+0000","last_fullsized":"2024-09-15T22:37:12.601056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520233+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:37:12.520233+0000","last_peered":"2024-09-15T22:37:12.520233+0000","last_clean":"2024-09-15T22:37:12.520233+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:37:12.520233+0000","last_undegraded":"2024-09-15T22:37:12.520233+0000","last_fullsized":"2024-09-15T22:37:12.520233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:37.463460+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:37:37.463460+0000","last_peered":"2024-09-15T22:37:37.463460+0000","last_clean":"2024-09-15T22:37:37.463460+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:37:37.463460+0000","last_undegraded":"2024-09-15T22:37:37.463460+0000","last_fullsized":"2024-09-15T22:37:37.463460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515651+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:37:12.515651+0000","last_peered":"2024-09-15T22:37:12.515651+0000","last_clean":"2024-09-15T22:37:12.515651+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:37:12.515651+0000","last_undegraded":"2024-09-15T22:37:12.515651+0000","last_fullsized":"2024-09-15T22:37:12.515651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519783+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:37:12.519783+0000","last_peered":"2024-09-15T22:37:12.519783+0000","last_clean":"2024-09-15T22:37:12.519783+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:37:12.519783+0000","last_undegraded":"2024-09-15T22:37:12.519783+0000","last_fullsized":"2024-09-15T22:37:12.519783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518105+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:37:12.518105+0000","last_peered":"2024-09-15T22:37:12.518105+0000","last_clean":"2024-09-15T22:37:12.518105+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:37:12.518105+0000","last_undegraded":"2024-09-15T22:37:12.518105+0000","last_fullsized":"2024-09-15T22:37:12.518105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520193+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:37:12.520193+0000","last_peered":"2024-09-15T22:37:12.520193+0000","last_clean":"2024-09-15T22:37:12.520193+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:37:12.520193+0000","last_undegraded":"2024-09-15T22:37:12.520193+0000","last_fullsized":"2024-09-15T22:37:12.520193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518951+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:37:12.518951+0000","last_peered":"2024-09-15T22:37:12.518951+0000","last_clean":"2024-09-15T22:37:12.518951+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:37:12.518951+0000","last_undegraded":"2024-09-15T22:37:12.518951+0000","last_fullsized":"2024-09-15T22:37:12.518951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522337+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:37:12.522337+0000","last_peered":"2024-09-15T22:37:12.522337+0000","last_clean":"2024-09-15T22:37:12.522337+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:37:12.522337+0000","last_undegraded":"2024-09-15T22:37:12.522337+0000","last_fullsized":"2024-09-15T22:37:12.522337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520948+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:37:12.520948+0000","last_peered":"2024-09-15T22:37:12.520948+0000","last_clean":"2024-09-15T22:37:12.520948+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:37:12.520948+0000","last_undegraded":"2024-09-15T22:37:12.520948+0000","last_fullsized":"2024-09-15T22:37:12.520948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518797+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:37:12.518797+0000","last_peered":"2024-09-15T22:37:12.518797+0000","last_clean":"2024-09-15T22:37:12.518797+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:37:12.518797+0000","last_undegraded":"2024-09-15T22:37:12.518797+0000","last_fullsized":"2024-09-15T22:37:12.518797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518475+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:37:12.518475+0000","last_peered":"2024-09-15T22:37:12.518475+0000","last_clean":"2024-09-15T22:37:12.518475+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:37:12.518475+0000","last_undegraded":"2024-09-15T22:37:12.518475+0000","last_fullsized":"2024-09-15T22:37:12.518475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599230+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:37:12.599230+0000","last_peered":"2024-09-15T22:37:12.599230+0000","last_clean":"2024-09-15T22:37:12.599230+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:37:12.599230+0000","last_undegraded":"2024-09-15T22:37:12.599230+0000","last_fullsized":"2024-09-15T22:37:12.599230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599780+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:37:12.599780+0000","last_peered":"2024-09-15T22:37:12.599780+0000","last_clean":"2024-09-15T22:37:12.599780+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:37:12.599780+0000","last_undegraded":"2024-09-15T22:37:12.599780+0000","last_fullsized":"2024-09-15T22:37:12.599780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598540+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:37:12.598540+0000","last_peered":"2024-09-15T22:37:12.598540+0000","last_clean":"2024-09-15T22:37:12.598540+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:37:12.598540+0000","last_undegraded":"2024-09-15T22:37:12.598540+0000","last_fullsized":"2024-09-15T22:37:12.598540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519258+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:37:12.519258+0000","last_peered":"2024-09-15T22:37:12.519258+0000","last_clean":"2024-09-15T22:37:12.519258+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:37:12.519258+0000","last_undegraded":"2024-09-15T22:37:12.519258+0000","last_fullsized":"2024-09-15T22:37:12.519258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519737+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:37:12.519737+0000","last_peered":"2024-09-15T22:37:12.519737+0000","last_clean":"2024-09-15T22:37:12.519737+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:37:12.519737+0000","last_undegraded":"2024-09-15T22:37:12.519737+0000","last_fullsized":"2024-09-15T22:37:12.519737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522651+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:37:12.522651+0000","last_peered":"2024-09-15T22:37:12.522651+0000","last_clean":"2024-09-15T22:37:12.522651+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:37:12.522651+0000","last_undegraded":"2024-09-15T22:37:12.522651+0000","last_fullsized":"2024-09-15T22:37:12.522651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519993+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:37:12.519993+0000","last_peered":"2024-09-15T22:37:12.519993+0000","last_clean":"2024-09-15T22:37:12.519993+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:37:12.519993+0000","last_undegraded":"2024-09-15T22:37:12.519993+0000","last_fullsized":"2024-09-15T22:37:12.519993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518708+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:37:12.518708+0000","last_peered":"2024-09-15T22:37:12.518708+0000","last_clean":"2024-09-15T22:37:12.518708+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:37:12.518708+0000","last_undegraded":"2024-09-15T22:37:12.518708+0000","last_fullsized":"2024-09-15T22:37:12.518708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519335+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:37:12.519335+0000","last_peered":"2024-09-15T22:37:12.519335+0000","last_clean":"2024-09-15T22:37:12.519335+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:37:12.519335+0000","last_undegraded":"2024-09-15T22:37:12.519335+0000","last_fullsized":"2024-09-15T22:37:12.519335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518731+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:37:12.518731+0000","last_peered":"2024-09-15T22:37:12.518731+0000","last_clean":"2024-09-15T22:37:12.518731+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:37:12.518731+0000","last_undegraded":"2024-09-15T22:37:12.518731+0000","last_fullsized":"2024-09-15T22:37:12.518731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514579+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:37:12.514579+0000","last_peered":"2024-09-15T22:37:12.514579+0000","last_clean":"2024-09-15T22:37:12.514579+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:37:12.514579+0000","last_undegraded":"2024-09-15T22:37:12.514579+0000","last_fullsized":"2024-09-15T22:37:12.514579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514661+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:37:12.514661+0000","last_peered":"2024-09-15T22:37:12.514661+0000","last_clean":"2024-09-15T22:37:12.514661+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:37:12.514661+0000","last_undegraded":"2024-09-15T22:37:12.514661+0000","last_fullsized":"2024-09-15T22:37:12.514661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519641+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:37:12.519641+0000","last_peered":"2024-09-15T22:37:12.519641+0000","last_clean":"2024-09-15T22:37:12.519641+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:37:12.519641+0000","last_undegraded":"2024-09-15T22:37:12.519641+0000","last_fullsized":"2024-09-15T22:37:12.519641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516209+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:37:12.516209+0000","last_peered":"2024-09-15T22:37:12.516209+0000","last_clean":"2024-09-15T22:37:12.516209+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:37:12.516209+0000","last_undegraded":"2024-09-15T22:37:12.516209+0000","last_fullsized":"2024-09-15T22:37:12.516209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521586+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:37:12.521586+0000","last_peered":"2024-09-15T22:37:12.521586+0000","last_clean":"2024-09-15T22:37:12.521586+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:37:12.521586+0000","last_undegraded":"2024-09-15T22:37:12.521586+0000","last_fullsized":"2024-09-15T22:37:12.521586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.602044+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:37:12.602044+0000","last_peered":"2024-09-15T22:37:12.602044+0000","last_clean":"2024-09-15T22:37:12.602044+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:37:12.602044+0000","last_undegraded":"2024-09-15T22:37:12.602044+0000","last_fullsized":"2024-09-15T22:37:12.602044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519437+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:37:12.519437+0000","last_peered":"2024-09-15T22:37:12.519437+0000","last_clean":"2024-09-15T22:37:12.519437+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:37:12.519437+0000","last_undegraded":"2024-09-15T22:37:12.519437+0000","last_fullsized":"2024-09-15T22:37:12.519437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598698+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:37:12.598698+0000","last_peered":"2024-09-15T22:37:12.598698+0000","last_clean":"2024-09-15T22:37:12.598698+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:37:12.598698+0000","last_undegraded":"2024-09-15T22:37:12.598698+0000","last_fullsized":"2024-09-15T22:37:12.598698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521658+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:37:12.521658+0000","last_peered":"2024-09-15T22:37:12.521658+0000","last_clean":"2024-09-15T22:37:12.521658+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:37:12.521658+0000","last_undegraded":"2024-09-15T22:37:12.521658+0000","last_fullsized":"2024-09-15T22:37:12.521658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601631+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:37:12.601631+0000","last_peered":"2024-09-15T22:37:12.601631+0000","last_clean":"2024-09-15T22:37:12.601631+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:37:12.601631+0000","last_undegraded":"2024-09-15T22:37:12.601631+0000","last_fullsized":"2024-09-15T22:37:12.601631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515556+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:37:12.515556+0000","last_peered":"2024-09-15T22:37:12.515556+0000","last_clean":"2024-09-15T22:37:12.515556+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:37:12.515556+0000","last_undegraded":"2024-09-15T22:37:12.515556+0000","last_fullsized":"2024-09-15T22:37:12.515556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.524446+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:37:12.524446+0000","last_peered":"2024-09-15T22:37:12.524446+0000","last_clean":"2024-09-15T22:37:12.524446+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:37:12.524446+0000","last_undegraded":"2024-09-15T22:37:12.524446+0000","last_fullsized":"2024-09-15T22:37:12.524446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514846+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:37:12.514846+0000","last_peered":"2024-09-15T22:37:12.514846+0000","last_clean":"2024-09-15T22:37:12.514846+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:37:12.514846+0000","last_undegraded":"2024-09-15T22:37:12.514846+0000","last_fullsized":"2024-09-15T22:37:12.514846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515077+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:37:12.515077+0000","last_peered":"2024-09-15T22:37:12.515077+0000","last_clean":"2024-09-15T22:37:12.515077+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:37:12.515077+0000","last_undegraded":"2024-09-15T22:37:12.515077+0000","last_fullsized":"2024-09-15T22:37:12.515077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519295+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:37:12.519295+0000","last_peered":"2024-09-15T22:37:12.519295+0000","last_clean":"2024-09-15T22:37:12.519295+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:37:12.519295+0000","last_undegraded":"2024-09-15T22:37:12.519295+0000","last_fullsized":"2024-09-15T22:37:12.519295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":341,"num_read_kb":241,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612639,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":491466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1683456,"data_stored":1142798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005098,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1011712,"data_stored":495798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":491622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462964,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1601536,"data_stored":1087533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626489,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":987136,"data_stored":486878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855422,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1609728,"data_stored":1083121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986243,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":491640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":487072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1073152,"data_stored":552483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1056768,"data_stored":550852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1011712,"data_stored":488266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:37:41.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:37:41.995 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:42.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:41 smithi130 ceph-mon[35742]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:42.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:41 smithi130 ceph-mon[35742]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:42.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:41 smithi016 ceph-mon[31492]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:42.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:41 smithi016 ceph-mon[31492]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:41 smithi059 ceph-mon[34872]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:41 smithi059 ceph-mon[34872]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.461+0000 7f03ba9a9640 1 -- 172.21.15.16:0/3623897248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 msgr2=0x7f03b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.461+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/3623897248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b41021c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009900 tx=0x7f03a402a410 comp rx=0 tx=0).stop 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.461+0000 7f03ba9a9640 1 -- 172.21.15.16:0/3623897248 shutdown_connections 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/3623897248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03b4106f10 0x7f03b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/3623897248 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03b4102790 0x7f03b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/3623897248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b41021c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 -- 172.21.15.16:0/3623897248 >> 172.21.15.16:0/3623897248 conn(0x7f03b40fd8c0 msgr2=0x7f03b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 -- 172.21.15.16:0/3623897248 shutdown_connections 2024-09-15T22:37:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 -- 172.21.15.16:0/3623897248 wait complete. 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 Processor -- start 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.462+0000 7f03ba9a9640 1 -- start start 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03ba9a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03ba9a9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03b4102790 0x7f03b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03ba9a9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03b4106f10 0x7f03b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03ba9a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4111fc0 con 0x7f03b4101de0 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03ba9a9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4111e40 con 0x7f03b4106f10 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03ba9a9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4112140 con 0x7f03b4102790 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56348/0 (socket says 172.21.15.16:56348) 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 -- 172.21.15.16:0/2931477730 learned_addr learned my addr 172.21.15.16:0/2931477730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b8f46640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03b4106f10 0x7f03b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b37fe640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03b4102790 0x7f03b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:42.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 -- 172.21.15.16:0/2931477730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03b4102790 msgr2=0x7f03b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03b4102790 0x7f03b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 -- 172.21.15.16:0/2931477730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03b4106f10 msgr2=0x7f03b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03b4106f10 0x7f03b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 -- 172.21.15.16:0/2931477730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b41a6920 con 0x7f03b4101de0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b37fe640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03b4102790 0x7f03b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b3fff640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b40744f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03a4004770 tx=0x7f03a4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b17fa640 1 -- 172.21.15.16:0/2931477730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a403f070 con 0x7f03b4101de0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b17fa640 1 -- 172.21.15.16:0/2931477730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a4031ca0 con 0x7f03b4101de0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03b17fa640 1 -- 172.21.15.16:0/2931477730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4035030 con 0x7f03b4101de0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.463+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b41a6bb0 con 0x7f03b4101de0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.464+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b41a7090 con 0x7f03b4101de0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.465+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b41021c0 con 0x7f03b4101de0 2024-09-15T22:37:42.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.466+0000 7f03b17fa640 1 -- 172.21.15.16:0/2931477730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f03a4042070 con 0x7f03b4101de0 2024-09-15T22:37:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.466+0000 7f03b17fa640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f039c0a32d0 0x7f039c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.467+0000 7f03b17fa640 1 -- 172.21.15.16:0/2931477730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f03a40f0a50 con 0x7f03b4101de0 2024-09-15T22:37:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.467+0000 7f03b37fe640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f039c0a32d0 0x7f039c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.467+0000 7f03b37fe640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f039c0a32d0 0x7f039c0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f03b4075f80 tx=0x7f03a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.470+0000 7f03b17fa640 1 -- 172.21.15.16:0/2931477730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03a40bd370 con 0x7f03b4101de0 2024-09-15T22:37:42.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.644+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03b4062630 con 0x7f039c0a32d0 2024-09-15T22:37:42.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.652+0000 7f03b17fa640 1 -- 172.21.15.16:0/2931477730 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340355 (secure 0 0 0) 0x7f03b4062630 con 0x7f039c0a32d0 2024-09-15T22:37:42.657 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:42.659 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:37:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f039c0a32d0 msgr2=0x7f039c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f039c0a32d0 0x7f039c0a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f03b4075f80 tx=0x7f03a8005f90 comp rx=0 tx=0).stop 2024-09-15T22:37:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 msgr2=0x7f03b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b40744f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f03a4004770 tx=0x7f03a4003c10 comp rx=0 tx=0).stop 2024-09-15T22:37:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 shutdown_connections 2024-09-15T22:37:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f039c0a32d0 0x7f039c0a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03b4106f10 0x7f03b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03b4102790 0x7f03b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 --2- 172.21.15.16:0/2931477730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03b4101de0 0x7f03b40744f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 >> 172.21.15.16:0/2931477730 conn(0x7f03b40fd8c0 msgr2=0x7f03b40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 shutdown_connections 2024-09-15T22:37:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:42.655+0000 7f03ba9a9640 1 -- 172.21.15.16:0/2931477730 wait complete. 2024-09-15T22:37:43.510 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-15T22:37:42.606280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658669,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":487,"num_read_kb":365,"num_write":352,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":412952,"kb_used_data":13712,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124528872,"statfs":{"total":1151940427776,"available":1151517564928,"internally_reserved":0,"allocated":14041088,"data_stored":7849529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003061"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518743+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:37:12.518743+0000","last_peered":"2024-09-15T22:37:12.518743+0000","last_clean":"2024-09-15T22:37:12.518743+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:37:12.518743+0000","last_undegraded":"2024-09-15T22:37:12.518743+0000","last_fullsized":"2024-09-15T22:37:12.518743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598848+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:37:12.598848+0000","last_peered":"2024-09-15T22:37:12.598848+0000","last_clean":"2024-09-15T22:37:12.598848+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:37:12.598848+0000","last_undegraded":"2024-09-15T22:37:12.598848+0000","last_fullsized":"2024-09-15T22:37:12.598848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519759+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:37:12.519759+0000","last_peered":"2024-09-15T22:37:12.519759+0000","last_clean":"2024-09-15T22:37:12.519759+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:37:12.519759+0000","last_undegraded":"2024-09-15T22:37:12.519759+0000","last_fullsized":"2024-09-15T22:37:12.519759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599158+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:37:12.599158+0000","last_peered":"2024-09-15T22:37:12.599158+0000","last_clean":"2024-09-15T22:37:12.599158+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:37:12.599158+0000","last_undegraded":"2024-09-15T22:37:12.599158+0000","last_fullsized":"2024-09-15T22:37:12.599158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515426+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:37:12.515426+0000","last_peered":"2024-09-15T22:37:12.515426+0000","last_clean":"2024-09-15T22:37:12.515426+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:37:12.515426+0000","last_undegraded":"2024-09-15T22:37:12.515426+0000","last_fullsized":"2024-09-15T22:37:12.515426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600028+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:37:12.600028+0000","last_peered":"2024-09-15T22:37:12.600028+0000","last_clean":"2024-09-15T22:37:12.600028+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:37:12.600028+0000","last_undegraded":"2024-09-15T22:37:12.600028+0000","last_fullsized":"2024-09-15T22:37:12.600028+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516022+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:37:12.516022+0000","last_peered":"2024-09-15T22:37:12.516022+0000","last_clean":"2024-09-15T22:37:12.516022+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:37:12.516022+0000","last_undegraded":"2024-09-15T22:37:12.516022+0000","last_fullsized":"2024-09-15T22:37:12.516022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519055+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:37:12.519055+0000","last_peered":"2024-09-15T22:37:12.519055+0000","last_clean":"2024-09-15T22:37:12.519055+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:37:12.519055+0000","last_undegraded":"2024-09-15T22:37:12.519055+0000","last_fullsized":"2024-09-15T22:37:12.519055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515480+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:37:12.515480+0000","last_peered":"2024-09-15T22:37:12.515480+0000","last_clean":"2024-09-15T22:37:12.515480+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:37:12.515480+0000","last_undegraded":"2024-09-15T22:37:12.515480+0000","last_fullsized":"2024-09-15T22:37:12.515480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515974+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:37:12.515974+0000","last_peered":"2024-09-15T22:37:12.515974+0000","last_clean":"2024-09-15T22:37:12.515974+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:37:12.515974+0000","last_undegraded":"2024-09-15T22:37:12.515974+0000","last_fullsized":"2024-09-15T22:37:12.515974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600276+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:37:12.600276+0000","last_peered":"2024-09-15T22:37:12.600276+0000","last_clean":"2024-09-15T22:37:12.600276+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:37:12.600276+0000","last_undegraded":"2024-09-15T22:37:12.600276+0000","last_fullsized":"2024-09-15T22:37:12.600276+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515440+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:37:12.515440+0000","last_peered":"2024-09-15T22:37:12.515440+0000","last_clean":"2024-09-15T22:37:12.515440+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:37:12.515440+0000","last_undegraded":"2024-09-15T22:37:12.515440+0000","last_fullsized":"2024-09-15T22:37:12.515440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521970+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:37:12.521970+0000","last_peered":"2024-09-15T22:37:12.521970+0000","last_clean":"2024-09-15T22:37:12.521970+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:37:12.521970+0000","last_undegraded":"2024-09-15T22:37:12.521970+0000","last_fullsized":"2024-09-15T22:37:12.521970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.529608+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:37:12.529608+0000","last_peered":"2024-09-15T22:37:12.529608+0000","last_clean":"2024-09-15T22:37:12.529608+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:37:12.529608+0000","last_undegraded":"2024-09-15T22:37:12.529608+0000","last_fullsized":"2024-09-15T22:37:12.529608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516090+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:37:12.516090+0000","last_peered":"2024-09-15T22:37:12.516090+0000","last_clean":"2024-09-15T22:37:12.516090+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:37:12.516090+0000","last_undegraded":"2024-09-15T22:37:12.516090+0000","last_fullsized":"2024-09-15T22:37:12.516090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598455+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:37:12.598455+0000","last_peered":"2024-09-15T22:37:12.598455+0000","last_clean":"2024-09-15T22:37:12.598455+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:37:12.598455+0000","last_undegraded":"2024-09-15T22:37:12.598455+0000","last_fullsized":"2024-09-15T22:37:12.598455+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519055+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:37:12.519055+0000","last_peered":"2024-09-15T22:37:12.519055+0000","last_clean":"2024-09-15T22:37:12.519055+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:37:12.519055+0000","last_undegraded":"2024-09-15T22:37:12.519055+0000","last_fullsized":"2024-09-15T22:37:12.519055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516034+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:37:12.516034+0000","last_peered":"2024-09-15T22:37:12.516034+0000","last_clean":"2024-09-15T22:37:12.516034+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:37:12.516034+0000","last_undegraded":"2024-09-15T22:37:12.516034+0000","last_fullsized":"2024-09-15T22:37:12.516034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518231+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:37:12.518231+0000","last_peered":"2024-09-15T22:37:12.518231+0000","last_clean":"2024-09-15T22:37:12.518231+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:37:12.518231+0000","last_undegraded":"2024-09-15T22:37:12.518231+0000","last_fullsized":"2024-09-15T22:37:12.518231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598647+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:37:12.598647+0000","last_peered":"2024-09-15T22:37:12.598647+0000","last_clean":"2024-09-15T22:37:12.598647+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:37:12.598647+0000","last_undegraded":"2024-09-15T22:37:12.598647+0000","last_fullsized":"2024-09-15T22:37:12.598647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518403+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:37:12.518403+0000","last_peered":"2024-09-15T22:37:12.518403+0000","last_clean":"2024-09-15T22:37:12.518403+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:37:12.518403+0000","last_undegraded":"2024-09-15T22:37:12.518403+0000","last_fullsized":"2024-09-15T22:37:12.518403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515452+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:37:12.515452+0000","last_peered":"2024-09-15T22:37:12.515452+0000","last_clean":"2024-09-15T22:37:12.515452+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:37:12.515452+0000","last_undegraded":"2024-09-15T22:37:12.515452+0000","last_fullsized":"2024-09-15T22:37:12.515452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518868+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:37:12.518868+0000","last_peered":"2024-09-15T22:37:12.518868+0000","last_clean":"2024-09-15T22:37:12.518868+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:37:12.518868+0000","last_undegraded":"2024-09-15T22:37:12.518868+0000","last_fullsized":"2024-09-15T22:37:12.518868+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.602394+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:37:12.602394+0000","last_peered":"2024-09-15T22:37:12.602394+0000","last_clean":"2024-09-15T22:37:12.602394+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:37:12.602394+0000","last_undegraded":"2024-09-15T22:37:12.602394+0000","last_fullsized":"2024-09-15T22:37:12.602394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515127+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:37:12.515127+0000","last_peered":"2024-09-15T22:37:12.515127+0000","last_clean":"2024-09-15T22:37:12.515127+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:37:12.515127+0000","last_undegraded":"2024-09-15T22:37:12.515127+0000","last_fullsized":"2024-09-15T22:37:12.515127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516502+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:37:12.516502+0000","last_peered":"2024-09-15T22:37:12.516502+0000","last_clean":"2024-09-15T22:37:12.516502+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:37:12.516502+0000","last_undegraded":"2024-09-15T22:37:12.516502+0000","last_fullsized":"2024-09-15T22:37:12.516502+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519963+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:37:12.519963+0000","last_peered":"2024-09-15T22:37:12.519963+0000","last_clean":"2024-09-15T22:37:12.519963+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:37:12.519963+0000","last_undegraded":"2024-09-15T22:37:12.519963+0000","last_fullsized":"2024-09-15T22:37:12.519963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599249+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:37:12.599249+0000","last_peered":"2024-09-15T22:37:12.599249+0000","last_clean":"2024-09-15T22:37:12.599249+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:37:12.599249+0000","last_undegraded":"2024-09-15T22:37:12.599249+0000","last_fullsized":"2024-09-15T22:37:12.599249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518816+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:37:12.518816+0000","last_peered":"2024-09-15T22:37:12.518816+0000","last_clean":"2024-09-15T22:37:12.518816+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:37:12.518816+0000","last_undegraded":"2024-09-15T22:37:12.518816+0000","last_fullsized":"2024-09-15T22:37:12.518816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598520+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:37:12.598520+0000","last_peered":"2024-09-15T22:37:12.598520+0000","last_clean":"2024-09-15T22:37:12.598520+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:37:12.598520+0000","last_undegraded":"2024-09-15T22:37:12.598520+0000","last_fullsized":"2024-09-15T22:37:12.598520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521045+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:37:12.521045+0000","last_peered":"2024-09-15T22:37:12.521045+0000","last_clean":"2024-09-15T22:37:12.521045+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:37:12.521045+0000","last_undegraded":"2024-09-15T22:37:12.521045+0000","last_fullsized":"2024-09-15T22:37:12.521045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516132+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:37:12.516132+0000","last_peered":"2024-09-15T22:37:12.516132+0000","last_clean":"2024-09-15T22:37:12.516132+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:37:12.516132+0000","last_undegraded":"2024-09-15T22:37:12.516132+0000","last_fullsized":"2024-09-15T22:37:12.516132+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519131+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:37:12.519131+0000","last_peered":"2024-09-15T22:37:12.519131+0000","last_clean":"2024-09-15T22:37:12.519131+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:37:12.519131+0000","last_undegraded":"2024-09-15T22:37:12.519131+0000","last_fullsized":"2024-09-15T22:37:12.519131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601205+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:37:12.601205+0000","last_peered":"2024-09-15T22:37:12.601205+0000","last_clean":"2024-09-15T22:37:12.601205+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:37:12.601205+0000","last_undegraded":"2024-09-15T22:37:12.601205+0000","last_fullsized":"2024-09-15T22:37:12.601205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518808+0000","last_change":"2024-09-15T22:36:06.335992+0000","last_active":"2024-09-15T22:37:12.518808+0000","last_peered":"2024-09-15T22:37:12.518808+0000","last_clean":"2024-09-15T22:37:12.518808+0000","last_became_active":"2024-09-15T22:36:06.335660+0000","last_became_peered":"2024-09-15T22:36:06.335660+0000","last_unstale":"2024-09-15T22:37:12.518808+0000","last_undegraded":"2024-09-15T22:37:12.518808+0000","last_fullsized":"2024-09-15T22:37:12.518808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:10:12.935878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515459+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:37:12.515459+0000","last_peered":"2024-09-15T22:37:12.515459+0000","last_clean":"2024-09-15T22:37:12.515459+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:37:12.515459+0000","last_undegraded":"2024-09-15T22:37:12.515459+0000","last_fullsized":"2024-09-15T22:37:12.515459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601323+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:37:12.601323+0000","last_peered":"2024-09-15T22:37:12.601323+0000","last_clean":"2024-09-15T22:37:12.601323+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:37:12.601323+0000","last_undegraded":"2024-09-15T22:37:12.601323+0000","last_fullsized":"2024-09-15T22:37:12.601323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522122+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:37:12.522122+0000","last_peered":"2024-09-15T22:37:12.522122+0000","last_clean":"2024-09-15T22:37:12.522122+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:37:12.522122+0000","last_undegraded":"2024-09-15T22:37:12.522122+0000","last_fullsized":"2024-09-15T22:37:12.522122+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520753+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:37:12.520753+0000","last_peered":"2024-09-15T22:37:12.520753+0000","last_clean":"2024-09-15T22:37:12.520753+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:37:12.520753+0000","last_undegraded":"2024-09-15T22:37:12.520753+0000","last_fullsized":"2024-09-15T22:37:12.520753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600398+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:37:12.600398+0000","last_peered":"2024-09-15T22:37:12.600398+0000","last_clean":"2024-09-15T22:37:12.600398+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:37:12.600398+0000","last_undegraded":"2024-09-15T22:37:12.600398+0000","last_fullsized":"2024-09-15T22:37:12.600398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:39.466609+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:37:39.466609+0000","last_peered":"2024-09-15T22:37:39.466609+0000","last_clean":"2024-09-15T22:37:39.466609+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:37:39.466609+0000","last_undegraded":"2024-09-15T22:37:39.466609+0000","last_fullsized":"2024-09-15T22:37:39.466609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520499+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:37:12.520499+0000","last_peered":"2024-09-15T22:37:12.520499+0000","last_clean":"2024-09-15T22:37:12.520499+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:37:12.520499+0000","last_undegraded":"2024-09-15T22:37:12.520499+0000","last_fullsized":"2024-09-15T22:37:12.520499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518474+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:37:12.518474+0000","last_peered":"2024-09-15T22:37:12.518474+0000","last_clean":"2024-09-15T22:37:12.518474+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:37:12.518474+0000","last_undegraded":"2024-09-15T22:37:12.518474+0000","last_fullsized":"2024-09-15T22:37:12.518474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521616+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:37:12.521616+0000","last_peered":"2024-09-15T22:37:12.521616+0000","last_clean":"2024-09-15T22:37:12.521616+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:37:12.521616+0000","last_undegraded":"2024-09-15T22:37:12.521616+0000","last_fullsized":"2024-09-15T22:37:12.521616+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599978+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:37:12.599978+0000","last_peered":"2024-09-15T22:37:12.599978+0000","last_clean":"2024-09-15T22:37:12.599978+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:37:12.599978+0000","last_undegraded":"2024-09-15T22:37:12.599978+0000","last_fullsized":"2024-09-15T22:37:12.599978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598543+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:37:12.598543+0000","last_peered":"2024-09-15T22:37:12.598543+0000","last_clean":"2024-09-15T22:37:12.598543+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:37:12.598543+0000","last_undegraded":"2024-09-15T22:37:12.598543+0000","last_fullsized":"2024-09-15T22:37:12.598543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515829+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:37:12.515829+0000","last_peered":"2024-09-15T22:37:12.515829+0000","last_clean":"2024-09-15T22:37:12.515829+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:37:12.515829+0000","last_undegraded":"2024-09-15T22:37:12.515829+0000","last_fullsized":"2024-09-15T22:37:12.515829+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521519+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:37:12.521519+0000","last_peered":"2024-09-15T22:37:12.521519+0000","last_clean":"2024-09-15T22:37:12.521519+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:37:12.521519+0000","last_undegraded":"2024-09-15T22:37:12.521519+0000","last_fullsized":"2024-09-15T22:37:12.521519+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521456+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:37:12.521456+0000","last_peered":"2024-09-15T22:37:12.521456+0000","last_clean":"2024-09-15T22:37:12.521456+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:37:12.521456+0000","last_undegraded":"2024-09-15T22:37:12.521456+0000","last_fullsized":"2024-09-15T22:37:12.521456+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521629+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:37:12.521629+0000","last_peered":"2024-09-15T22:37:12.521629+0000","last_clean":"2024-09-15T22:37:12.521629+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:37:12.521629+0000","last_undegraded":"2024-09-15T22:37:12.521629+0000","last_fullsized":"2024-09-15T22:37:12.521629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520568+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:37:12.520568+0000","last_peered":"2024-09-15T22:37:12.520568+0000","last_clean":"2024-09-15T22:37:12.520568+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:37:12.520568+0000","last_undegraded":"2024-09-15T22:37:12.520568+0000","last_fullsized":"2024-09-15T22:37:12.520568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.603280+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:37:12.603280+0000","last_peered":"2024-09-15T22:37:12.603280+0000","last_clean":"2024-09-15T22:37:12.603280+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:37:12.603280+0000","last_undegraded":"2024-09-15T22:37:12.603280+0000","last_fullsized":"2024-09-15T22:37:12.603280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521424+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:37:12.521424+0000","last_peered":"2024-09-15T22:37:12.521424+0000","last_clean":"2024-09-15T22:37:12.521424+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:37:12.521424+0000","last_undegraded":"2024-09-15T22:37:12.521424+0000","last_fullsized":"2024-09-15T22:37:12.521424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600508+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:37:12.600508+0000","last_peered":"2024-09-15T22:37:12.600508+0000","last_clean":"2024-09-15T22:37:12.600508+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:37:12.600508+0000","last_undegraded":"2024-09-15T22:37:12.600508+0000","last_fullsized":"2024-09-15T22:37:12.600508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514748+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:37:12.514748+0000","last_peered":"2024-09-15T22:37:12.514748+0000","last_clean":"2024-09-15T22:37:12.514748+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:37:12.514748+0000","last_undegraded":"2024-09-15T22:37:12.514748+0000","last_fullsized":"2024-09-15T22:37:12.514748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521667+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:37:12.521667+0000","last_peered":"2024-09-15T22:37:12.521667+0000","last_clean":"2024-09-15T22:37:12.521667+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:37:12.521667+0000","last_undegraded":"2024-09-15T22:37:12.521667+0000","last_fullsized":"2024-09-15T22:37:12.521667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520299+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:37:12.520299+0000","last_peered":"2024-09-15T22:37:12.520299+0000","last_clean":"2024-09-15T22:37:12.520299+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:37:12.520299+0000","last_undegraded":"2024-09-15T22:37:12.520299+0000","last_fullsized":"2024-09-15T22:37:12.520299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599617+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:37:12.599617+0000","last_peered":"2024-09-15T22:37:12.599617+0000","last_clean":"2024-09-15T22:37:12.599617+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:37:12.599617+0000","last_undegraded":"2024-09-15T22:37:12.599617+0000","last_fullsized":"2024-09-15T22:37:12.599617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522385+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:37:12.522385+0000","last_peered":"2024-09-15T22:37:12.522385+0000","last_clean":"2024-09-15T22:37:12.522385+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:37:12.522385+0000","last_undegraded":"2024-09-15T22:37:12.522385+0000","last_fullsized":"2024-09-15T22:37:12.522385+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516161+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:37:12.516161+0000","last_peered":"2024-09-15T22:37:12.516161+0000","last_clean":"2024-09-15T22:37:12.516161+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:37:12.516161+0000","last_undegraded":"2024-09-15T22:37:12.516161+0000","last_fullsized":"2024-09-15T22:37:12.516161+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599990+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:37:12.599990+0000","last_peered":"2024-09-15T22:37:12.599990+0000","last_clean":"2024-09-15T22:37:12.599990+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:37:12.599990+0000","last_undegraded":"2024-09-15T22:37:12.599990+0000","last_fullsized":"2024-09-15T22:37:12.599990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600191+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:37:12.600191+0000","last_peered":"2024-09-15T22:37:12.600191+0000","last_clean":"2024-09-15T22:37:12.600191+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:37:12.600191+0000","last_undegraded":"2024-09-15T22:37:12.600191+0000","last_fullsized":"2024-09-15T22:37:12.600191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.535528+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:37:12.535528+0000","last_peered":"2024-09-15T22:37:12.535528+0000","last_clean":"2024-09-15T22:37:12.535528+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:37:12.535528+0000","last_undegraded":"2024-09-15T22:37:12.535528+0000","last_fullsized":"2024-09-15T22:37:12.535528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600316+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:37:12.600316+0000","last_peered":"2024-09-15T22:37:12.600316+0000","last_clean":"2024-09-15T22:37:12.600316+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:37:12.600316+0000","last_undegraded":"2024-09-15T22:37:12.600316+0000","last_fullsized":"2024-09-15T22:37:12.600316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521246+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:37:12.521246+0000","last_peered":"2024-09-15T22:37:12.521246+0000","last_clean":"2024-09-15T22:37:12.521246+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:37:12.521246+0000","last_undegraded":"2024-09-15T22:37:12.521246+0000","last_fullsized":"2024-09-15T22:37:12.521246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:39.467147+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:37:39.467147+0000","last_peered":"2024-09-15T22:37:39.467147+0000","last_clean":"2024-09-15T22:37:39.467147+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:37:39.467147+0000","last_undegraded":"2024-09-15T22:37:39.467147+0000","last_fullsized":"2024-09-15T22:37:39.467147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518502+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:37:12.518502+0000","last_peered":"2024-09-15T22:37:12.518502+0000","last_clean":"2024-09-15T22:37:12.518502+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:37:12.518502+0000","last_undegraded":"2024-09-15T22:37:12.518502+0000","last_fullsized":"2024-09-15T22:37:12.518502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521473+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:37:12.521473+0000","last_peered":"2024-09-15T22:37:12.521473+0000","last_clean":"2024-09-15T22:37:12.521473+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:37:12.521473+0000","last_undegraded":"2024-09-15T22:37:12.521473+0000","last_fullsized":"2024-09-15T22:37:12.521473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600971+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:37:12.600971+0000","last_peered":"2024-09-15T22:37:12.600971+0000","last_clean":"2024-09-15T22:37:12.600971+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:37:12.600971+0000","last_undegraded":"2024-09-15T22:37:12.600971+0000","last_fullsized":"2024-09-15T22:37:12.600971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518349+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:37:12.518349+0000","last_peered":"2024-09-15T22:37:12.518349+0000","last_clean":"2024-09-15T22:37:12.518349+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:37:12.518349+0000","last_undegraded":"2024-09-15T22:37:12.518349+0000","last_fullsized":"2024-09-15T22:37:12.518349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519065+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:37:12.519065+0000","last_peered":"2024-09-15T22:37:12.519065+0000","last_clean":"2024-09-15T22:37:12.519065+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:37:12.519065+0000","last_undegraded":"2024-09-15T22:37:12.519065+0000","last_fullsized":"2024-09-15T22:37:12.519065+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519534+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:37:12.519534+0000","last_peered":"2024-09-15T22:37:12.519534+0000","last_clean":"2024-09-15T22:37:12.519534+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:37:12.519534+0000","last_undegraded":"2024-09-15T22:37:12.519534+0000","last_fullsized":"2024-09-15T22:37:12.519534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600099+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:37:12.600099+0000","last_peered":"2024-09-15T22:37:12.600099+0000","last_clean":"2024-09-15T22:37:12.600099+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:37:12.600099+0000","last_undegraded":"2024-09-15T22:37:12.600099+0000","last_fullsized":"2024-09-15T22:37:12.600099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515885+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:37:12.515885+0000","last_peered":"2024-09-15T22:37:12.515885+0000","last_clean":"2024-09-15T22:37:12.515885+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:37:12.515885+0000","last_undegraded":"2024-09-15T22:37:12.515885+0000","last_fullsized":"2024-09-15T22:37:12.515885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"85'33","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:37.464725+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:37:37.464725+0000","last_peered":"2024-09-15T22:37:37.464725+0000","last_clean":"2024-09-15T22:37:37.464725+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:37:37.464725+0000","last_undegraded":"2024-09-15T22:37:37.464725+0000","last_fullsized":"2024-09-15T22:37:37.464725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64064,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":115,"num_read_kb":120,"num_write":34,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518225+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:37:12.518225+0000","last_peered":"2024-09-15T22:37:12.518225+0000","last_clean":"2024-09-15T22:37:12.518225+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:37:12.518225+0000","last_undegraded":"2024-09-15T22:37:12.518225+0000","last_fullsized":"2024-09-15T22:37:12.518225+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519045+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:37:12.519045+0000","last_peered":"2024-09-15T22:37:12.519045+0000","last_clean":"2024-09-15T22:37:12.519045+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:37:12.519045+0000","last_undegraded":"2024-09-15T22:37:12.519045+0000","last_fullsized":"2024-09-15T22:37:12.519045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600452+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:37:12.600452+0000","last_peered":"2024-09-15T22:37:12.600452+0000","last_clean":"2024-09-15T22:37:12.600452+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:37:12.600452+0000","last_undegraded":"2024-09-15T22:37:12.600452+0000","last_fullsized":"2024-09-15T22:37:12.600452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515684+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:37:12.515684+0000","last_peered":"2024-09-15T22:37:12.515684+0000","last_clean":"2024-09-15T22:37:12.515684+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:37:12.515684+0000","last_undegraded":"2024-09-15T22:37:12.515684+0000","last_fullsized":"2024-09-15T22:37:12.515684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515452+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:37:12.515452+0000","last_peered":"2024-09-15T22:37:12.515452+0000","last_clean":"2024-09-15T22:37:12.515452+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:37:12.515452+0000","last_undegraded":"2024-09-15T22:37:12.515452+0000","last_fullsized":"2024-09-15T22:37:12.515452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521901+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:37:12.521901+0000","last_peered":"2024-09-15T22:37:12.521901+0000","last_clean":"2024-09-15T22:37:12.521901+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:37:12.521901+0000","last_undegraded":"2024-09-15T22:37:12.521901+0000","last_fullsized":"2024-09-15T22:37:12.521901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.517877+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:37:12.517877+0000","last_peered":"2024-09-15T22:37:12.517877+0000","last_clean":"2024-09-15T22:37:12.517877+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:37:12.517877+0000","last_undegraded":"2024-09-15T22:37:12.517877+0000","last_fullsized":"2024-09-15T22:37:12.517877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519555+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:37:12.519555+0000","last_peered":"2024-09-15T22:37:12.519555+0000","last_clean":"2024-09-15T22:37:12.519555+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:37:12.519555+0000","last_undegraded":"2024-09-15T22:37:12.519555+0000","last_fullsized":"2024-09-15T22:37:12.519555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:36.461472+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:37:36.461472+0000","last_peered":"2024-09-15T22:37:36.461472+0000","last_clean":"2024-09-15T22:37:36.461472+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:37:36.461472+0000","last_undegraded":"2024-09-15T22:37:36.461472+0000","last_fullsized":"2024-09-15T22:37:36.461472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598378+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:37:12.598378+0000","last_peered":"2024-09-15T22:37:12.598378+0000","last_clean":"2024-09-15T22:37:12.598378+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:37:12.598378+0000","last_undegraded":"2024-09-15T22:37:12.598378+0000","last_fullsized":"2024-09-15T22:37:12.598378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514982+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:37:12.514982+0000","last_peered":"2024-09-15T22:37:12.514982+0000","last_clean":"2024-09-15T22:37:12.514982+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:37:12.514982+0000","last_undegraded":"2024-09-15T22:37:12.514982+0000","last_fullsized":"2024-09-15T22:37:12.514982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518316+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:37:12.518316+0000","last_peered":"2024-09-15T22:37:12.518316+0000","last_clean":"2024-09-15T22:37:12.518316+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:37:12.518316+0000","last_undegraded":"2024-09-15T22:37:12.518316+0000","last_fullsized":"2024-09-15T22:37:12.518316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.600692+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:37:12.600692+0000","last_peered":"2024-09-15T22:37:12.600692+0000","last_clean":"2024-09-15T22:37:12.600692+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:37:12.600692+0000","last_undegraded":"2024-09-15T22:37:12.600692+0000","last_fullsized":"2024-09-15T22:37:12.600692+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518697+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:37:12.518697+0000","last_peered":"2024-09-15T22:37:12.518697+0000","last_clean":"2024-09-15T22:37:12.518697+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:37:12.518697+0000","last_undegraded":"2024-09-15T22:37:12.518697+0000","last_fullsized":"2024-09-15T22:37:12.518697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598830+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:37:12.598830+0000","last_peered":"2024-09-15T22:37:12.598830+0000","last_clean":"2024-09-15T22:37:12.598830+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:37:12.598830+0000","last_undegraded":"2024-09-15T22:37:12.598830+0000","last_fullsized":"2024-09-15T22:37:12.598830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518579+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:37:12.518579+0000","last_peered":"2024-09-15T22:37:12.518579+0000","last_clean":"2024-09-15T22:37:12.518579+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:37:12.518579+0000","last_undegraded":"2024-09-15T22:37:12.518579+0000","last_fullsized":"2024-09-15T22:37:12.518579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601419+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:37:12.601419+0000","last_peered":"2024-09-15T22:37:12.601419+0000","last_clean":"2024-09-15T22:37:12.601419+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:37:12.601419+0000","last_undegraded":"2024-09-15T22:37:12.601419+0000","last_fullsized":"2024-09-15T22:37:12.601419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601056+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:37:12.601056+0000","last_peered":"2024-09-15T22:37:12.601056+0000","last_clean":"2024-09-15T22:37:12.601056+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:37:12.601056+0000","last_undegraded":"2024-09-15T22:37:12.601056+0000","last_fullsized":"2024-09-15T22:37:12.601056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520233+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:37:12.520233+0000","last_peered":"2024-09-15T22:37:12.520233+0000","last_clean":"2024-09-15T22:37:12.520233+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:37:12.520233+0000","last_undegraded":"2024-09-15T22:37:12.520233+0000","last_fullsized":"2024-09-15T22:37:12.520233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:37.463460+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:37:37.463460+0000","last_peered":"2024-09-15T22:37:37.463460+0000","last_clean":"2024-09-15T22:37:37.463460+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:37:37.463460+0000","last_undegraded":"2024-09-15T22:37:37.463460+0000","last_fullsized":"2024-09-15T22:37:37.463460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515651+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:37:12.515651+0000","last_peered":"2024-09-15T22:37:12.515651+0000","last_clean":"2024-09-15T22:37:12.515651+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:37:12.515651+0000","last_undegraded":"2024-09-15T22:37:12.515651+0000","last_fullsized":"2024-09-15T22:37:12.515651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519783+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:37:12.519783+0000","last_peered":"2024-09-15T22:37:12.519783+0000","last_clean":"2024-09-15T22:37:12.519783+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:37:12.519783+0000","last_undegraded":"2024-09-15T22:37:12.519783+0000","last_fullsized":"2024-09-15T22:37:12.519783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518105+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:37:12.518105+0000","last_peered":"2024-09-15T22:37:12.518105+0000","last_clean":"2024-09-15T22:37:12.518105+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:37:12.518105+0000","last_undegraded":"2024-09-15T22:37:12.518105+0000","last_fullsized":"2024-09-15T22:37:12.518105+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520193+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:37:12.520193+0000","last_peered":"2024-09-15T22:37:12.520193+0000","last_clean":"2024-09-15T22:37:12.520193+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:37:12.520193+0000","last_undegraded":"2024-09-15T22:37:12.520193+0000","last_fullsized":"2024-09-15T22:37:12.520193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518951+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:37:12.518951+0000","last_peered":"2024-09-15T22:37:12.518951+0000","last_clean":"2024-09-15T22:37:12.518951+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:37:12.518951+0000","last_undegraded":"2024-09-15T22:37:12.518951+0000","last_fullsized":"2024-09-15T22:37:12.518951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522337+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:37:12.522337+0000","last_peered":"2024-09-15T22:37:12.522337+0000","last_clean":"2024-09-15T22:37:12.522337+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:37:12.522337+0000","last_undegraded":"2024-09-15T22:37:12.522337+0000","last_fullsized":"2024-09-15T22:37:12.522337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.520948+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:37:12.520948+0000","last_peered":"2024-09-15T22:37:12.520948+0000","last_clean":"2024-09-15T22:37:12.520948+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:37:12.520948+0000","last_undegraded":"2024-09-15T22:37:12.520948+0000","last_fullsized":"2024-09-15T22:37:12.520948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518797+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:37:12.518797+0000","last_peered":"2024-09-15T22:37:12.518797+0000","last_clean":"2024-09-15T22:37:12.518797+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:37:12.518797+0000","last_undegraded":"2024-09-15T22:37:12.518797+0000","last_fullsized":"2024-09-15T22:37:12.518797+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518475+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:37:12.518475+0000","last_peered":"2024-09-15T22:37:12.518475+0000","last_clean":"2024-09-15T22:37:12.518475+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:37:12.518475+0000","last_undegraded":"2024-09-15T22:37:12.518475+0000","last_fullsized":"2024-09-15T22:37:12.518475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599230+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:37:12.599230+0000","last_peered":"2024-09-15T22:37:12.599230+0000","last_clean":"2024-09-15T22:37:12.599230+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:37:12.599230+0000","last_undegraded":"2024-09-15T22:37:12.599230+0000","last_fullsized":"2024-09-15T22:37:12.599230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.599780+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:37:12.599780+0000","last_peered":"2024-09-15T22:37:12.599780+0000","last_clean":"2024-09-15T22:37:12.599780+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:37:12.599780+0000","last_undegraded":"2024-09-15T22:37:12.599780+0000","last_fullsized":"2024-09-15T22:37:12.599780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598540+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:37:12.598540+0000","last_peered":"2024-09-15T22:37:12.598540+0000","last_clean":"2024-09-15T22:37:12.598540+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:37:12.598540+0000","last_undegraded":"2024-09-15T22:37:12.598540+0000","last_fullsized":"2024-09-15T22:37:12.598540+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519258+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:37:12.519258+0000","last_peered":"2024-09-15T22:37:12.519258+0000","last_clean":"2024-09-15T22:37:12.519258+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:37:12.519258+0000","last_undegraded":"2024-09-15T22:37:12.519258+0000","last_fullsized":"2024-09-15T22:37:12.519258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519737+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:37:12.519737+0000","last_peered":"2024-09-15T22:37:12.519737+0000","last_clean":"2024-09-15T22:37:12.519737+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:37:12.519737+0000","last_undegraded":"2024-09-15T22:37:12.519737+0000","last_fullsized":"2024-09-15T22:37:12.519737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.522651+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:37:12.522651+0000","last_peered":"2024-09-15T22:37:12.522651+0000","last_clean":"2024-09-15T22:37:12.522651+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:37:12.522651+0000","last_undegraded":"2024-09-15T22:37:12.522651+0000","last_fullsized":"2024-09-15T22:37:12.522651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519993+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:37:12.519993+0000","last_peered":"2024-09-15T22:37:12.519993+0000","last_clean":"2024-09-15T22:37:12.519993+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:37:12.519993+0000","last_undegraded":"2024-09-15T22:37:12.519993+0000","last_fullsized":"2024-09-15T22:37:12.519993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518708+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:37:12.518708+0000","last_peered":"2024-09-15T22:37:12.518708+0000","last_clean":"2024-09-15T22:37:12.518708+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:37:12.518708+0000","last_undegraded":"2024-09-15T22:37:12.518708+0000","last_fullsized":"2024-09-15T22:37:12.518708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519335+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:37:12.519335+0000","last_peered":"2024-09-15T22:37:12.519335+0000","last_clean":"2024-09-15T22:37:12.519335+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:37:12.519335+0000","last_undegraded":"2024-09-15T22:37:12.519335+0000","last_fullsized":"2024-09-15T22:37:12.519335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.518731+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:37:12.518731+0000","last_peered":"2024-09-15T22:37:12.518731+0000","last_clean":"2024-09-15T22:37:12.518731+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:37:12.518731+0000","last_undegraded":"2024-09-15T22:37:12.518731+0000","last_fullsized":"2024-09-15T22:37:12.518731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514579+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:37:12.514579+0000","last_peered":"2024-09-15T22:37:12.514579+0000","last_clean":"2024-09-15T22:37:12.514579+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:37:12.514579+0000","last_undegraded":"2024-09-15T22:37:12.514579+0000","last_fullsized":"2024-09-15T22:37:12.514579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514661+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:37:12.514661+0000","last_peered":"2024-09-15T22:37:12.514661+0000","last_clean":"2024-09-15T22:37:12.514661+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:37:12.514661+0000","last_undegraded":"2024-09-15T22:37:12.514661+0000","last_fullsized":"2024-09-15T22:37:12.514661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519641+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:37:12.519641+0000","last_peered":"2024-09-15T22:37:12.519641+0000","last_clean":"2024-09-15T22:37:12.519641+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:37:12.519641+0000","last_undegraded":"2024-09-15T22:37:12.519641+0000","last_fullsized":"2024-09-15T22:37:12.519641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.516209+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:37:12.516209+0000","last_peered":"2024-09-15T22:37:12.516209+0000","last_clean":"2024-09-15T22:37:12.516209+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:37:12.516209+0000","last_undegraded":"2024-09-15T22:37:12.516209+0000","last_fullsized":"2024-09-15T22:37:12.516209+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521586+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:37:12.521586+0000","last_peered":"2024-09-15T22:37:12.521586+0000","last_clean":"2024-09-15T22:37:12.521586+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:37:12.521586+0000","last_undegraded":"2024-09-15T22:37:12.521586+0000","last_fullsized":"2024-09-15T22:37:12.521586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.602044+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:37:12.602044+0000","last_peered":"2024-09-15T22:37:12.602044+0000","last_clean":"2024-09-15T22:37:12.602044+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:37:12.602044+0000","last_undegraded":"2024-09-15T22:37:12.602044+0000","last_fullsized":"2024-09-15T22:37:12.602044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519437+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:37:12.519437+0000","last_peered":"2024-09-15T22:37:12.519437+0000","last_clean":"2024-09-15T22:37:12.519437+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:37:12.519437+0000","last_undegraded":"2024-09-15T22:37:12.519437+0000","last_fullsized":"2024-09-15T22:37:12.519437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.598698+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:37:12.598698+0000","last_peered":"2024-09-15T22:37:12.598698+0000","last_clean":"2024-09-15T22:37:12.598698+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:37:12.598698+0000","last_undegraded":"2024-09-15T22:37:12.598698+0000","last_fullsized":"2024-09-15T22:37:12.598698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.521658+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:37:12.521658+0000","last_peered":"2024-09-15T22:37:12.521658+0000","last_clean":"2024-09-15T22:37:12.521658+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:37:12.521658+0000","last_undegraded":"2024-09-15T22:37:12.521658+0000","last_fullsized":"2024-09-15T22:37:12.521658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.601631+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:37:12.601631+0000","last_peered":"2024-09-15T22:37:12.601631+0000","last_clean":"2024-09-15T22:37:12.601631+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:37:12.601631+0000","last_undegraded":"2024-09-15T22:37:12.601631+0000","last_fullsized":"2024-09-15T22:37:12.601631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515556+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:37:12.515556+0000","last_peered":"2024-09-15T22:37:12.515556+0000","last_clean":"2024-09-15T22:37:12.515556+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:37:12.515556+0000","last_undegraded":"2024-09-15T22:37:12.515556+0000","last_fullsized":"2024-09-15T22:37:12.515556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.524446+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:37:12.524446+0000","last_peered":"2024-09-15T22:37:12.524446+0000","last_clean":"2024-09-15T22:37:12.524446+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:37:12.524446+0000","last_undegraded":"2024-09-15T22:37:12.524446+0000","last_fullsized":"2024-09-15T22:37:12.524446+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.514846+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:37:12.514846+0000","last_peered":"2024-09-15T22:37:12.514846+0000","last_clean":"2024-09-15T22:37:12.514846+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:37:12.514846+0000","last_undegraded":"2024-09-15T22:37:12.514846+0000","last_fullsized":"2024-09-15T22:37:12.514846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.515077+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:37:12.515077+0000","last_peered":"2024-09-15T22:37:12.515077+0000","last_clean":"2024-09-15T22:37:12.515077+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:37:12.515077+0000","last_undegraded":"2024-09-15T22:37:12.515077+0000","last_fullsized":"2024-09-15T22:37:12.515077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:37:12.519295+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:37:12.519295+0000","last_peered":"2024-09-15T22:37:12.519295+0000","last_clean":"2024-09-15T22:37:12.519295+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:37:12.519295+0000","last_undegraded":"2024-09-15T22:37:12.519295+0000","last_fullsized":"2024-09-15T22:37:12.519295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68067,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":351,"num_read_kb":246,"num_write":169,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":204201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612639,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":491466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841572,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1683456,"data_stored":1142798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005098,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1011712,"data_stored":495798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234031,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":491622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462965,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1601536,"data_stored":1087533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626490,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":987136,"data_stored":486878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855422,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1609728,"data_stored":1083121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":491640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":487072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1073152,"data_stored":552483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1056768,"data_stored":550852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1011712,"data_stored":488266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":65695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:37:43.516 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:37:43.516 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T22:37:43.516 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-15T22:37:43.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-15T22:37:43.777 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:44.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:43 smithi016 ceph-mon[31492]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:44.021 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:43 smithi016 ceph-mon[31492]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:44.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:43 smithi130 ceph-mon[35742]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:44.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:43 smithi130 ceph-mon[35742]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:44.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:43 smithi059 ceph-mon[34872]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:44.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:43 smithi059 ceph-mon[34872]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.281+0000 7f50dfb83640 1 -- 172.21.15.16:0/1169549443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8101420 msgr2=0x7f50d810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.281+0000 7f50dfb83640 1 --2- 172.21.15.16:0/1169549443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8101420 0x7f50d810d950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009cd0 tx=0x7f50d002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.282+0000 7f50dfb83640 1 -- 172.21.15.16:0/1169549443 shutdown_connections 2024-09-15T22:37:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.282+0000 7f50dfb83640 1 --2- 172.21.15.16:0/1169549443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8101420 0x7f50d810d950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.282+0000 7f50dfb83640 1 --2- 172.21.15.16:0/1169549443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d8100a80 0x7f50d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.282+0000 7f50dfb83640 1 --2- 172.21.15.16:0/1169549443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50d8106a80 0x7f50d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.282+0000 7f50dfb83640 1 -- 172.21.15.16:0/1169549443 >> 172.21.15.16:0/1169549443 conn(0x7f50d80fc7a0 msgr2=0x7f50d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.282+0000 7f50dfb83640 1 -- 172.21.15.16:0/1169549443 shutdown_connections 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.282+0000 7f50dfb83640 1 -- 172.21.15.16:0/1169549443 wait complete. 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 Processor -- start 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 -- start start 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8100a80 0x7f50d81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d8101420 0x7f50d81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50d8106a80 0x7f50d81a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8110ed0 con 0x7f50d8100a80 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8110d50 con 0x7f50d8101420 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dfb83640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8111050 con 0x7f50d8106a80 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8100a80 0x7f50d81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8100a80 0x7f50d81992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56362/0 (socket says 172.21.15.16:56362) 2024-09-15T22:37:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 -- 172.21.15.16:0/263200196 learned_addr learned my addr 172.21.15.16:0/263200196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50de120640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50d8106a80 0x7f50d81a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd11e640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d8101420 0x7f50d81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 -- 172.21.15.16:0/263200196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50d8106a80 msgr2=0x7f50d81a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50d8106a80 0x7f50d81a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 -- 172.21.15.16:0/263200196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d8101420 msgr2=0x7f50d81997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d8101420 0x7f50d81997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd91f640 1 -- 172.21.15.16:0/263200196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d81a4860 con 0x7f50d8100a80 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.283+0000 7f50dd11e640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d8101420 0x7f50d81997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.284+0000 7f50dd91f640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8100a80 0x7f50d81992a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f50d400d770 tx=0x7f50d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.284+0000 7f50ceffd640 1 -- 172.21.15.16:0/263200196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4017070 con 0x7f50d8100a80 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.284+0000 7f50ceffd640 1 -- 172.21.15.16:0/263200196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d4005500 con 0x7f50d8100a80 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.284+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d81a4a30 con 0x7f50d8100a80 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.284+0000 7f50ceffd640 1 -- 172.21.15.16:0/263200196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d4004e00 con 0x7f50d8100a80 2024-09-15T22:37:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.284+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d81a4f40 con 0x7f50d8100a80 2024-09-15T22:37:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.285+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d81021c0 con 0x7f50d8100a80 2024-09-15T22:37:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.286+0000 7f50ceffd640 1 -- 172.21.15.16:0/263200196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f50d4004fa0 con 0x7f50d8100a80 2024-09-15T22:37:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.287+0000 7f50ceffd640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50c40a3350 0x7f50c40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.287+0000 7f50ceffd640 1 -- 172.21.15.16:0/263200196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f50d40cfbe0 con 0x7f50d8100a80 2024-09-15T22:37:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.287+0000 7f50dd11e640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50c40a3350 0x7f50c40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.287+0000 7f50dd11e640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50c40a3350 0x7f50c40a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50d806b2e0 tx=0x7f50c800b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.290+0000 7f50ceffd640 1 -- 172.21.15.16:0/263200196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50d409c4c0 con 0x7f50d8100a80 2024-09-15T22:37:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.506+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f50d81a5230 con 0x7f50d8100a80 2024-09-15T22:37:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.507+0000 7f50ceffd640 1 -- 172.21.15.16:0/263200196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f50d409bdc0 con 0x7f50d8100a80 2024-09-15T22:37:44.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:44.510 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T22:37:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50c40a3350 msgr2=0x7f50c40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50c40a3350 0x7f50c40a5810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50d806b2e0 tx=0x7f50c800b5f0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8100a80 msgr2=0x7f50d81992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8100a80 0x7f50d81992a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f50d400d770 tx=0x7f50d400dc30 comp rx=0 tx=0).stop 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 shutdown_connections 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f50c40a3350 0x7f50c40a5810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f50d8106a80 0x7f50d81a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50d8101420 0x7f50d81997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 --2- 172.21.15.16:0/263200196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50d8100a80 0x7f50d81992a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 >> 172.21.15.16:0/263200196 conn(0x7f50d80fc7a0 msgr2=0x7f50d80fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 shutdown_connections 2024-09-15T22:37:44.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:44.509+0000 7f50dfb83640 1 -- 172.21.15.16:0/263200196 wait complete. 2024-09-15T22:37:45.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:44 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/263200196' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:37:45.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:44 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/263200196' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:37:45.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:44 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/263200196' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:37:45.263 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-15T22:37:45.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:37:45.522 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:45.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:45 smithi016 ceph-mon[31492]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:46.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 -- 172.21.15.16:0/2985063806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1082a0 msgr2=0x7f18ec10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:46.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 --2- 172.21.15.16:0/2985063806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1082a0 0x7f18ec10ec40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009cd0 tx=0x7f18e402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 -- 172.21.15.16:0/2985063806 shutdown_connections 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 --2- 172.21.15.16:0/2985063806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1082a0 0x7f18ec10ec40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 --2- 172.21.15.16:0/2985063806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18ec1036e0 0x7f18ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 --2- 172.21.15.16:0/2985063806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18ec102d30 0x7f18ec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 -- 172.21.15.16:0/2985063806 >> 172.21.15.16:0/2985063806 conn(0x7f18ec0fc7d0 msgr2=0x7f18ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 -- 172.21.15.16:0/2985063806 shutdown_connections 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 -- 172.21.15.16:0/2985063806 wait complete. 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 Processor -- start 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.989+0000 7f18f226f640 1 -- start start 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18f226f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18ec102d30 0x7f18ec100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:46.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18f226f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1036e0 0x7f18ec100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18f226f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18ec1082a0 0x7f18ec101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18f226f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec112100 con 0x7f18ec1036e0 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18f226f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec111f80 con 0x7f18ec1082a0 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18f226f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18ec112280 con 0x7f18ec102d30 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1036e0 0x7f18ec100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1036e0 0x7f18ec100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56376/0 (socket says 172.21.15.16:56376) 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 -- 172.21.15.16:0/3470601814 learned_addr learned my addr 172.21.15.16:0/3470601814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18ebfff640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18ec102d30 0x7f18ec100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18f080c640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18ec1082a0 0x7f18ec101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 -- 172.21.15.16:0/3470601814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18ec102d30 msgr2=0x7f18ec100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18ec102d30 0x7f18ec100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 -- 172.21.15.16:0/3470601814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18ec1082a0 msgr2=0x7f18ec101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:46.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18ec1082a0 0x7f18ec101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 -- 172.21.15.16:0/3470601814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18ec1a6a80 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.990+0000 7f18eb7fe640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1036e0 0x7f18ec100ed0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c970 tx=0x7f18e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.991+0000 7f18e97fa640 1 -- 172.21.15.16:0/3470601814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0007bb0 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.991+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18ec1a6d70 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.991+0000 7f18e97fa640 1 -- 172.21.15.16:0/3470601814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e0007d50 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.991+0000 7f18e97fa640 1 -- 172.21.15.16:0/3470601814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e000f450 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.991+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18ec1a7220 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.992+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18ec103110 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.993+0000 7f18e97fa640 1 -- 172.21.15.16:0/3470601814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f18e0004f60 con 0x7f18ec1036e0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.994+0000 7f18e97fa640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18d40a32d0 0x7f18d40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.994+0000 7f18ebfff640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18d40a32d0 0x7f18d40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:46.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.994+0000 7f18e97fa640 1 -- 172.21.15.16:0/3470601814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f18e00cfba0 con 0x7f18ec1036e0 2024-09-15T22:37:46.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.994+0000 7f18ebfff640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18d40a32d0 0x7f18d40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009890 tx=0x7f18dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:46.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:45.998+0000 7f18e97fa640 1 -- 172.21.15.16:0/3470601814 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18e0014030 con 0x7f18ec1036e0 2024-09-15T22:37:46.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:45 smithi130 ceph-mon[35742]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:46.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:45 smithi059 ceph-mon[34872]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:46.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.212+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18ec10a2b0 con 0x7f18ec1036e0 2024-09-15T22:37:46.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.212+0000 7f18e97fa640 1 -- 172.21.15.16:0/3470601814 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f18e009c400 con 0x7f18ec1036e0 2024-09-15T22:37:46.215 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:46.215 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-15T22:36:47:380763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:47.103123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24469,24479,24500]},"id":1}]} 2024-09-15T22:37:46.215 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-15T22:37:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.214+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18d40a32d0 msgr2=0x7f18d40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.214+0000 7f18f226f640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18d40a32d0 0x7f18d40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009890 tx=0x7f18dc008040 comp rx=0 tx=0).stop 2024-09-15T22:37:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.214+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1036e0 msgr2=0x7f18ec100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.214+0000 7f18f226f640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1036e0 0x7f18ec100ed0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c970 tx=0x7f18e000ce30 comp rx=0 tx=0).stop 2024-09-15T22:37:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 shutdown_connections 2024-09-15T22:37:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f18d40a32d0 0x7f18d40a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f18ec1082a0 0x7f18ec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1036e0 0x7f18ec100ed0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 --2- 172.21.15.16:0/3470601814 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f18ec102d30 0x7f18ec100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 >> 172.21.15.16:0/3470601814 conn(0x7f18ec0fc7d0 msgr2=0x7f18ec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 shutdown_connections 2024-09-15T22:37:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:46.215+0000 7f18f226f640 1 -- 172.21.15.16:0/3470601814 wait complete. 2024-09-15T22:37:46.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:46 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3470601814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:46.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:37:47.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:46 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3470601814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:47.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:46 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3470601814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:47.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 -- 172.21.15.16:0/2689342650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20101420 msgr2=0x7fef2010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 --2- 172.21.15.16:0/2689342650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20101420 0x7fef2010d950 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fef18009cd0 tx=0x7fef1802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 -- 172.21.15.16:0/2689342650 shutdown_connections 2024-09-15T22:37:47.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 --2- 172.21.15.16:0/2689342650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20101420 0x7fef2010d950 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 --2- 172.21.15.16:0/2689342650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef20100a80 0x7fef20100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 --2- 172.21.15.16:0/2689342650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20106a80 0x7fef20106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 -- 172.21.15.16:0/2689342650 >> 172.21.15.16:0/2689342650 conn(0x7fef200fc7a0 msgr2=0x7fef200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:47.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 -- 172.21.15.16:0/2689342650 shutdown_connections 2024-09-15T22:37:47.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.605+0000 7fef27610640 1 -- 172.21.15.16:0/2689342650 wait complete. 2024-09-15T22:37:47.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.606+0000 7fef27610640 1 Processor -- start 2024-09-15T22:37:47.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.606+0000 7fef27610640 1 -- start start 2024-09-15T22:37:47.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.606+0000 7fef27610640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef20100a80 0x7fef20197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef27610640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20101420 0x7fef20197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef27610640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20106a80 0x7fef2019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef27610640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef20110e80 con 0x7fef20106a80 2024-09-15T22:37:47.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef27610640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fef20110d00 con 0x7fef20100a80 2024-09-15T22:37:47.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef27610640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fef20111000 con 0x7fef20101420 2024-09-15T22:37:47.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20106a80 0x7fef2019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20106a80 0x7fef2019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56392/0 (socket says 172.21.15.16:56392) 2024-09-15T22:37:47.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 -- 172.21.15.16:0/599746034 learned_addr learned my addr 172.21.15.16:0/599746034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:47.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef253ac640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef20100a80 0x7fef20197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef24bab640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20101420 0x7fef20197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 -- 172.21.15.16:0/599746034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20101420 msgr2=0x7fef20197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20101420 0x7fef20197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 -- 172.21.15.16:0/599746034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef20100a80 msgr2=0x7fef20197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef20100a80 0x7fef20197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 -- 172.21.15.16:0/599746034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2019c2d0 con 0x7fef20106a80 2024-09-15T22:37:47.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef25bad640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20106a80 0x7fef2019bbf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fef18005cc0 tx=0x7fef18003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:47.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef167fc640 1 -- 172.21.15.16:0/599746034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18004040 con 0x7fef20106a80 2024-09-15T22:37:47.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2019c560 con 0x7fef20106a80 2024-09-15T22:37:47.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef167fc640 1 -- 172.21.15.16:0/599746034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef180046c0 con 0x7fef20106a80 2024-09-15T22:37:47.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.607+0000 7fef167fc640 1 -- 172.21.15.16:0/599746034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18046480 con 0x7fef20106a80 2024-09-15T22:37:47.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.608+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef201a3ea0 con 0x7fef20106a80 2024-09-15T22:37:47.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.608+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef20100ee0 con 0x7fef20106a80 2024-09-15T22:37:47.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.610+0000 7fef167fc640 1 -- 172.21.15.16:0/599746034 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fef180305a0 con 0x7fef20106a80 2024-09-15T22:37:47.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.610+0000 7fef167fc640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef0c0a32d0 0x7fef0c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.610+0000 7fef253ac640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef0c0a32d0 0x7fef0c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.610+0000 7fef167fc640 1 -- 172.21.15.16:0/599746034 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fef1802e030 con 0x7fef20106a80 2024-09-15T22:37:47.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.611+0000 7fef253ac640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef0c0a32d0 0x7fef0c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00bb60 tx=0x7fef1c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:47.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.613+0000 7fef167fc640 1 -- 172.21.15.16:0/599746034 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef1804b060 con 0x7fef20106a80 2024-09-15T22:37:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.830+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef20109050 con 0x7fef20106a80 2024-09-15T22:37:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.831+0000 7fef167fc640 1 -- 172.21.15.16:0/599746034 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7fef180bdb90 con 0x7fef20106a80 2024-09-15T22:37:47.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-15T22:36:47:380763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:47.103123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24469,24479,24500]},"id":1}]} 2024-09-15T22:37:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-15T22:37:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.833+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef0c0a32d0 msgr2=0x7fef0c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.833+0000 7fef27610640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef0c0a32d0 0x7fef0c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00bb60 tx=0x7fef1c00b390 comp rx=0 tx=0).stop 2024-09-15T22:37:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20106a80 msgr2=0x7fef2019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20106a80 0x7fef2019bbf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fef18005cc0 tx=0x7fef18003970 comp rx=0 tx=0).stop 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 shutdown_connections 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fef0c0a32d0 0x7fef0c0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef20106a80 0x7fef2019bbf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fef20101420 0x7fef20197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 --2- 172.21.15.16:0/599746034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef20100a80 0x7fef20197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 >> 172.21.15.16:0/599746034 conn(0x7fef200fc7a0 msgr2=0x7fef200fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 shutdown_connections 2024-09-15T22:37:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:47.834+0000 7fef27610640 1 -- 172.21.15.16:0/599746034 wait complete. 2024-09-15T22:37:48.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:47 smithi130 ceph-mon[35742]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:48.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:47 smithi016 ceph-mon[31492]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:48.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:47 smithi059 ceph-mon[34872]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:48.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:37:48.767 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:49.012 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:48 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/599746034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:48 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/599746034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:49.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:48 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/599746034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 -- 172.21.15.16:0/1977858174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9864102d30 msgr2=0x7f9864103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 --2- 172.21.15.16:0/1977858174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9864102d30 0x7f9864103110 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9854009930 tx=0x7f985402a420 comp rx=0 tx=0).stop 2024-09-15T22:37:49.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 -- 172.21.15.16:0/1977858174 shutdown_connections 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 --2- 172.21.15.16:0/1977858174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98641082a0 0x7f986410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 --2- 172.21.15.16:0/1977858174 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98641036e0 0x7f9864107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 --2- 172.21.15.16:0/1977858174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9864102d30 0x7f9864103110 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 -- 172.21.15.16:0/1977858174 >> 172.21.15.16:0/1977858174 conn(0x7f98640fc7d0 msgr2=0x7f98640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 -- 172.21.15.16:0/1977858174 shutdown_connections 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.236+0000 7f986aa17640 1 -- 172.21.15.16:0/1977858174 wait complete. 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 Processor -- start 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 -- start start 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9864102d30 0x7f9864198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98641036e0 0x7f9864198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98641082a0 0x7f986419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:49.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9864112100 con 0x7f98641082a0 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9864111f80 con 0x7f9864102d30 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.237+0000 7f986aa17640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9864112280 con 0x7f98641036e0 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98641082a0 0x7f986419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9863fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9864102d30 0x7f9864198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98641082a0 0x7f986419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56406/0 (socket says 172.21.15.16:56406) 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 -- 172.21.15.16:0/1046404912 learned_addr learned my addr 172.21.15.16:0/1046404912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f98637fe640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98641036e0 0x7f9864198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 -- 172.21.15.16:0/1046404912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98641036e0 msgr2=0x7f9864198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:49.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98641036e0 0x7f9864198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 -- 172.21.15.16:0/1046404912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9864102d30 msgr2=0x7f9864198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9864102d30 0x7f9864198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 -- 172.21.15.16:0/1046404912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986419d5b0 con 0x7f98641082a0 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f9868fb4640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98641082a0 0x7f986419ced0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f985c00b970 tx=0x7f985c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f98617fa640 1 -- 172.21.15.16:0/1046404912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c019070 con 0x7f98641082a0 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f98617fa640 1 -- 172.21.15.16:0/1046404912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f985c00cb40 con 0x7f98641082a0 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f98617fa640 1 -- 172.21.15.16:0/1046404912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c007650 con 0x7f98641082a0 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986419d8a0 con 0x7f98641082a0 2024-09-15T22:37:49.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.238+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641a5180 con 0x7f98641082a0 2024-09-15T22:37:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.239+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986406a850 con 0x7f98641082a0 2024-09-15T22:37:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.241+0000 7f98617fa640 1 -- 172.21.15.16:0/1046404912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f985c01c070 con 0x7f98641082a0 2024-09-15T22:37:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.241+0000 7f98617fa640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f984c0a3350 0x7f984c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.242+0000 7f9863fff640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f984c0a3350 0x7f984c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.242+0000 7f98617fa640 1 -- 172.21.15.16:0/1046404912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f985c0cf6e0 con 0x7f98641082a0 2024-09-15T22:37:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.242+0000 7f9863fff640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f984c0a3350 0x7f984c0a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9854009900 tx=0x7f98540058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:49.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.245+0000 7f98617fa640 1 -- 172.21.15.16:0/1046404912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f985c09ba00 con 0x7f98641082a0 2024-09-15T22:37:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.422+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9864103110 con 0x7f98641082a0 2024-09-15T22:37:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.424+0000 7f98617fa640 1 -- 172.21.15.16:0/1046404912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24717 (secure 0 0 0) 0x7f9864103110 con 0x7f98641082a0 2024-09-15T22:37:49.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:49.427 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":87,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:37:12.511711+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:37:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f984c0a3350 msgr2=0x7f984c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f984c0a3350 0x7f984c0a5810 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9854009900 tx=0x7f98540058d0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98641082a0 msgr2=0x7f986419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98641082a0 0x7f986419ced0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f985c00b970 tx=0x7f985c00be30 comp rx=0 tx=0).stop 2024-09-15T22:37:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 shutdown_connections 2024-09-15T22:37:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f984c0a3350 0x7f984c0a5810 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98641082a0 0x7f986419ced0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f98641036e0 0x7f9864198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 --2- 172.21.15.16:0/1046404912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9864102d30 0x7f9864198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 >> 172.21.15.16:0/1046404912 conn(0x7f98640fc7d0 msgr2=0x7f9864105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 shutdown_connections 2024-09-15T22:37:49.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:49.426+0000 7f986aa17640 1 -- 172.21.15.16:0/1046404912 wait complete. 2024-09-15T22:37:50.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:49 smithi130 ceph-mon[35742]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:50.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:49 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1046404912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:50.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:49 smithi016 ceph-mon[31492]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:50.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:49 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1046404912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:50.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:37:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:49 smithi059 ceph-mon[34872]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:50.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:49 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1046404912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:37:50.412 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.851+0000 7f73ac218640 1 -- 172.21.15.16:0/2525653495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101de0 msgr2=0x7f73a41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.851+0000 7f73ac218640 1 --2- 172.21.15.16:0/2525653495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101de0 0x7f73a41021c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7394009ca0 tx=0x7f739402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.851+0000 7f73ac218640 1 -- 172.21.15.16:0/2525653495 shutdown_connections 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.851+0000 7f73ac218640 1 --2- 172.21.15.16:0/2525653495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a4106f10 0x7f73a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.851+0000 7f73ac218640 1 --2- 172.21.15.16:0/2525653495 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73a4102790 0x7f73a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.851+0000 7f73ac218640 1 --2- 172.21.15.16:0/2525653495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4101de0 0x7f73a41021c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.851+0000 7f73ac218640 1 -- 172.21.15.16:0/2525653495 >> 172.21.15.16:0/2525653495 conn(0x7f73a40fd8c0 msgr2=0x7f73a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 -- 172.21.15.16:0/2525653495 shutdown_connections 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 -- 172.21.15.16:0/2525653495 wait complete. 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 Processor -- start 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 -- start start 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a4101de0 0x7f73a41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4102790 0x7f73a4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73a4106f10 0x7f73a419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111f70 con 0x7f73a4102790 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f73a4111df0 con 0x7f73a4101de0 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.852+0000 7f73ac218640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f73a41120f0 con 0x7f73a4106f10 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4102790 0x7f73a4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4102790 0x7f73a4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56432/0 (socket says 172.21.15.16:56432) 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 -- 172.21.15.16:0/1091795525 learned_addr learned my addr 172.21.15.16:0/1091795525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a9fb4640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a4101de0 0x7f73a41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73aa7b5640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73a4106f10 0x7f73a419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 -- 172.21.15.16:0/1091795525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73a4106f10 msgr2=0x7f73a419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73a4106f10 0x7f73a419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 -- 172.21.15.16:0/1091795525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a4101de0 msgr2=0x7f73a41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a4101de0 0x7f73a41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 -- 172.21.15.16:0/1091795525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a419d380 con 0x7f73a4102790 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a97b3640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4102790 0x7f73a4198910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f739800d950 tx=0x7f739800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a2ffd640 1 -- 172.21.15.16:0/1091795525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398017070 con 0x7f73a4102790 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a2ffd640 1 -- 172.21.15.16:0/1091795525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7398009c30 con 0x7f73a4102790 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a419d670 con 0x7f73a4102790 2024-09-15T22:37:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73a2ffd640 1 -- 172.21.15.16:0/1091795525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739800f6b0 con 0x7f73a4102790 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.853+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a41a4f50 con 0x7f73a4102790 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.854+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a41021c0 con 0x7f73a4102790 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.855+0000 7f73a2ffd640 1 -- 172.21.15.16:0/1091795525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f73980109d0 con 0x7f73a4102790 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.856+0000 7f73a2ffd640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f738c0a3350 0x7f738c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.856+0000 7f73a2ffd640 1 -- 172.21.15.16:0/1091795525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f73980cfaa0 con 0x7f73a4102790 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.856+0000 7f73a9fb4640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f738c0a3350 0x7f738c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.857+0000 7f73a9fb4640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f738c0a3350 0x7f738c0a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7394000f80 tx=0x7f73940093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:50.860+0000 7f73a2ffd640 1 -- 172.21.15.16:0/1091795525 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7398009de0 con 0x7f73a4102790 2024-09-15T22:37:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.075+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73a4199730 con 0x7f73a4102790 2024-09-15T22:37:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.076+0000 7f73a2ffd640 1 -- 172.21.15.16:0/1091795525 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f739809c300 con 0x7f73a4102790 2024-09-15T22:37:51.077 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:37:51.077 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-15T22:36:47:380763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:47.103123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24469,24479,24500]},"id":1}]} 2024-09-15T22:37:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f738c0a3350 msgr2=0x7f738c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f738c0a3350 0x7f738c0a5810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7394000f80 tx=0x7f73940093f0 comp rx=0 tx=0).stop 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4102790 msgr2=0x7f73a4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4102790 0x7f73a4198910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f739800d950 tx=0x7f739800de10 comp rx=0 tx=0).stop 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 shutdown_connections 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f738c0a3350 0x7f738c0a5810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f73a4106f10 0x7f73a419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73a4102790 0x7f73a4198910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 --2- 172.21.15.16:0/1091795525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73a4101de0 0x7f73a41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 >> 172.21.15.16:0/1091795525 conn(0x7f73a40fd8c0 msgr2=0x7f73a40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 shutdown_connections 2024-09-15T22:37:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:51.078+0000 7f73ac218640 1 -- 172.21.15.16:0/1091795525 wait complete. 2024-09-15T22:37:51.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:36:08.871593+0000', 'modified': '2024-09-15T22:36:47.103123+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24500, 'mds_1': 24479, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/2738921095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 2738921095}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 2738921095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6835/2202482741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2202482741}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2202482741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/1773539140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 1773539140}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 1773539140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.130:6839/1236682624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 1236682624}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 1236682624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.59:6837/2084675816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 2084675816}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2084675816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6839/384228279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 384228279}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 384228279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24469, 24479, 24500]} 2024-09-15T22:37:51.954 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:37:51.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:37:52.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:51 smithi130 ceph-mon[35742]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:52.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:51 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1091795525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:52.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:51 smithi016 ceph-mon[31492]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:52.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:51 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1091795525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:51 smithi059 ceph-mon[34872]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:37:52.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:51 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1091795525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:37:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 -- 172.21.15.16:0/2285985900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 msgr2=0x7fddc010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 --2- 172.21.15.16:0/2285985900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc010ea90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009ca0 tx=0x7fddb002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 -- 172.21.15.16:0/2285985900 shutdown_connections 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 --2- 172.21.15.16:0/2285985900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc010ea90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 --2- 172.21.15.16:0/2285985900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc0102790 0x7fddc01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 --2- 172.21.15.16:0/2285985900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc0101de0 0x7fddc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 -- 172.21.15.16:0/2285985900 >> 172.21.15.16:0/2285985900 conn(0x7fddc00fd8c0 msgr2=0x7fddc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 -- 172.21.15.16:0/2285985900 shutdown_connections 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 -- 172.21.15.16:0/2285985900 wait complete. 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 Processor -- start 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.711+0000 7fddc82f4640 1 -- start start 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc82f4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc0101de0 0x7fddc00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc82f4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc0102790 0x7fddc0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc82f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc82f4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0111fb0 con 0x7fddc0106f10 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc82f4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0111e30 con 0x7fddc0102790 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc82f4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0112130 con 0x7fddc0101de0 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45232/0 (socket says 172.21.15.16:45232) 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 -- 172.21.15.16:0/458628961 learned_addr learned my addr 172.21.15.16:0/458628961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6090640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc0101de0 0x7fddc00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc588f640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc0102790 0x7fddc0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 -- 172.21.15.16:0/458628961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc0101de0 msgr2=0x7fddc00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc0101de0 0x7fddc00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 -- 172.21.15.16:0/458628961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc0102790 msgr2=0x7fddc0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc0102790 0x7fddc0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 -- 172.21.15.16:0/458628961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc01a69e0 con 0x7fddc0106f10 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddc6891640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc0074f70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fddb0031a20 tx=0x7fddb0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddbf7fe640 1 -- 172.21.15.16:0/458628961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0003b70 con 0x7fddc0106f10 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.712+0000 7fddbf7fe640 1 -- 172.21.15.16:0/458628961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb0003d10 con 0x7fddc0106f10 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.713+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc01a6c70 con 0x7fddc0106f10 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.713+0000 7fddbf7fe640 1 -- 172.21.15.16:0/458628961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb002fc80 con 0x7fddc0106f10 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.713+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc01a7200 con 0x7fddc0106f10 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.715+0000 7fddbf7fe640 1 -- 172.21.15.16:0/458628961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fddb00046a0 con 0x7fddc0106f10 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.715+0000 7fddca6cd740 1 Processor -- start 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.715+0000 7fddca6cd740 1 -- start start 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.715+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.715+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8dd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddca6cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556add839090 con 0x556add8ddbb0 2024-09-15T22:37:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddca6cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556add7e0a70 con 0x556add8d32f0 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddca6cd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556add835050 con 0x556add8d29d0 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddbf7fe640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdda80a3350 0x7fdda80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc588f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc588f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45244/0 (socket says 172.21.15.16:45244) 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc588f640 1 -- 172.21.15.16:0/1995786622 learned_addr learned my addr 172.21.15.16:0/1995786622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6090640 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc588f640 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6891640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdda80a3350 0x7fdda80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddbf7fe640 1 -- 172.21.15.16:0/458628961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fddb00f0eb0 con 0x7fddc0106f10 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6090640 1 -- 172.21.15.16:0/1995786622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 msgr2=0x556add8d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6090640 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6090640 1 -- 172.21.15.16:0/1995786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 msgr2=0x556add8dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6090640 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6090640 1 -- 172.21.15.16:0/1995786622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556add7ed130 con 0x556add8d32f0 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc588f640 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6090640 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8dd670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddb8004630 tx=0x7fddb800bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef797cd464029ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.716+0000 7fddc6891640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdda80a3350 0x7fdda80a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fddb00c5800 tx=0x7fddb002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddbcdea640 1 -- 172.21.15.16:0/1995786622 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8003bd0 con 0x556add8d32f0 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddbcdea640 1 -- 172.21.15.16:0/1995786622 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8015070 con 0x556add8d32f0 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddbcdea640 1 -- 172.21.15.16:0/1995786622 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8004d20 con 0x556add8d32f0 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddca6cd740 1 -- 172.21.15.16:0/1995786622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 msgr2=0x556add8dd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8dd670 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fddb8004630 tx=0x7fddb800bb20 comp rx=0 tx=0).stop 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddca6cd740 1 -- 172.21.15.16:0/1995786622 shutdown_connections 2024-09-15T22:37:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8dd670 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1995786622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.717+0000 7fddca6cd740 1 -- 172.21.15.16:0/1995786622 >> 172.21.15.16:0/1995786622 conn(0x556add8cef60 msgr2=0x556add8cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 -- 172.21.15.16:0/1995786622 shutdown_connections 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 -- 172.21.15.16:0/1995786622 wait complete. 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 -- start start 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 Processor -- start 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556add8e0780 con 0x556add8ddbb0 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556add7f3f20 con 0x556add8d32f0 2024-09-15T22:37:52.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556add8e0900 con 0x556add8d29d0 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddca6cd740 10 client.0 _ll_register_callbacks cb 0x556add7bd030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddc588f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddc588f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45258/0 (socket says 172.21.15.16:45258) 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.718+0000 7fddc588f640 1 -- 172.21.15.16:0/4235538809 learned_addr learned my addr 172.21.15.16:0/4235538809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc6090640 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc588f640 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc588f640 1 -- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 msgr2=0x556add8dd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc588f640 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8dd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc588f640 1 -- 172.21.15.16:0/4235538809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 msgr2=0x556add8da6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc588f640 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8da6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc588f640 1 -- 172.21.15.16:0/4235538809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556add5aa2b0 con 0x556add8ddbb0 2024-09-15T22:37:52.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddc588f640 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dac00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fddb4018c20 tx=0x7fddb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5036e3b346968a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fdda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fdda5ffb640 1 -- 172.21.15.16:0/4235538809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40163d0 con 0x556add8ddbb0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb8015070 con 0x556add8ddbb0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fdda5ffb640 1 -- 172.21.15.16:0/4235538809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fddb400f040 con 0x556add8ddbb0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fdda5ffb640 1 -- 172.21.15.16:0/4235538809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddb402a050 con 0x556add8ddbb0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddca6cd740 10 client.15057 fetch_fsmap learned FSMap version 25 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556add8ec040 con 0x556add8ddbb0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.719+0000 7fdda5ffb640 1 -- 172.21.15.16:0/4235538809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8015070 con 0x556add8ddbb0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fdda5ffb640 1 -- 172.21.15.16:0/4235538809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fddb40154f0 con 0x556add8ddbb0 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddca6cd740 10 client.15057 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddca6cd740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddca6cd740 20 client.15057 populate_metadata read hostname 'smithi016' 2024-09-15T22:37:52.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddca6cd740 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add8ee630 0x556add8efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddca6cd740 4 client.15057 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556add846600 con 0x556add8ee630 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddc6090640 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add8ee630 0x556add8efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fdda5ffb640 10 client.15057 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.720+0000 7fddc6090640 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add8ee630 0x556add8efaa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52a60d7ddce8bbcc server_cookie=8031ed659aebdee1 in_seq=0 out_seq=0 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fdda5ffb640 1 -- 172.21.15.16:0/4235538809 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556add846600 con 0x556add8ee630 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fdda5ffb640 10 client.15057 handle_command_reply: tid=0 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 client.15057 shutdown 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 msgr2=0x556add8dac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dac00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fddb4018c20 tx=0x7fddb40046c0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 shutdown_connections 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add8ee630 0x556add8efaa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8ddbb0 0x556add8dac00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8da6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fdda5ffb640 0 client.15057 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:37:52.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 --2- 172.21.15.16:0/4235538809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8dd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 >> 172.21.15.16:0/4235538809 conn(0x556add8cef60 msgr2=0x556add8d4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 shutdown_connections 2024-09-15T22:37:52.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.721+0000 7fddca6cd740 1 -- 172.21.15.16:0/4235538809 wait complete. 2024-09-15T22:37:52.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.722+0000 7fddca6cd740 20 client.15057 trim_cache size 0 max 16384 2024-09-15T22:37:52.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 Processor -- start 2024-09-15T22:37:52.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 -- start start 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d32f0 0x556add8dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8ddbb0 0x556add8d5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556add8edde0 con 0x556add8d32f0 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556add8ee4b0 con 0x556add8ddbb0 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddca6cd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556add8eeb80 con 0x556add8d29d0 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddc6090640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d32f0 0x556add8dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddc6090640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d32f0 0x556add8dac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45262/0 (socket says 172.21.15.16:45262) 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.740+0000 7fddc6090640 1 -- 172.21.15.16:0/809061111 learned_addr learned my addr 172.21.15.16:0/809061111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:52.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc588f640 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8ddbb0 0x556add8d5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc588f640 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc6090640 1 -- 172.21.15.16:0/809061111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 msgr2=0x556add8efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc6090640 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc6090640 1 -- 172.21.15.16:0/809061111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8ddbb0 msgr2=0x556add8d5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc6090640 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8ddbb0 0x556add8d5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc6090640 1 -- 172.21.15.16:0/809061111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556add8dcf90 con 0x556add8d32f0 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc588f640 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fddc6090640 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d32f0 0x556add8dac10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fddb8004630 tx=0x7fddb8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc524bf9ccae56c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fdda7fff640 1 -- 172.21.15.16:0/809061111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8003bd0 con 0x556add8d32f0 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fdda7fff640 1 -- 172.21.15.16:0/809061111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8015030 con 0x556add8d32f0 2024-09-15T22:37:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.741+0000 7fdda7fff640 1 -- 172.21.15.16:0/809061111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8027d40 con 0x556add8d32f0 2024-09-15T22:37:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 -- 172.21.15.16:0/809061111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d32f0 msgr2=0x556add8dac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d32f0 0x556add8dac10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fddb8004630 tx=0x7fddb8013040 comp rx=0 tx=0).stop 2024-09-15T22:37:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 -- 172.21.15.16:0/809061111 shutdown_connections 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8ddbb0 0x556add8d5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d32f0 0x556add8dac10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 --2- 172.21.15.16:0/809061111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8d29d0 0x556add8efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 -- 172.21.15.16:0/809061111 >> 172.21.15.16:0/809061111 conn(0x556add8cef60 msgr2=0x556add8d5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 -- 172.21.15.16:0/809061111 shutdown_connections 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 -- 172.21.15.16:0/809061111 wait complete. 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 -- start start 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 Processor -- start 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d29d0 0x556add8f8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.742+0000 7fddca6cd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8ddbb0 0x556add8d7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddca6cd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556add8ef920 con 0x556add8d29d0 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddca6cd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556add8ef250 con 0x556add8d32f0 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddca6cd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556add8efff0 con 0x556add8ddbb0 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddca6cd740 10 client.0 _ll_register_callbacks cb 0x7fddb4009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d29d0 0x556add8f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d29d0 0x556add8f8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45276/0 (socket says 172.21.15.16:45276) 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 -- 172.21.15.16:0/1400542971 learned_addr learned my addr 172.21.15.16:0/1400542971 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc6090640 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8ddbb0 0x556add8d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 -- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8ddbb0 msgr2=0x556add8d7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8ddbb0 0x556add8d7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 -- 172.21.15.16:0/1400542971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 msgr2=0x556add8d72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8d72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 -- 172.21.15.16:0/1400542971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556add8d7ef0 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddc588f640 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d29d0 0x556add8f8370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009b20 tx=0x7fddb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1733ef044852be41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fdda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fdda5ffb640 1 -- 172.21.15.16:0/1400542971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4018450 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb8015030 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.743+0000 7fdda5ffb640 1 -- 172.21.15.16:0/1400542971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fddb4015e20 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fdda5ffb640 1 -- 172.21.15.16:0/1400542971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddb40155e0 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 10 client.15063 fetch_fsmap learned FSMap version 25 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556add8f6c80 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fdda5ffb640 1 -- 172.21.15.16:0/1400542971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb8015030 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fdda5ffb640 1 -- 172.21.15.16:0/1400542971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fddb402ea30 con 0x556add8d29d0 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 10 client.15063 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:37:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 20 client.15063 populate_metadata read hostname 'smithi016' 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add95d6e0 0x556add95c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 4 client.15063 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "status"}) -- 0x556add840b10 con 0x556add95d6e0 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.744+0000 7fddc6090640 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add95d6e0 0x556add95c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddc6090640 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add95d6e0 0x556add95c3f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38ffe57fec6994c4 server_cookie=6cc1308b4bb0e0fa in_seq=0 out_seq=0 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fdda5ffb640 10 client.15063 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fdda5ffb640 1 -- 172.21.15.16:0/1400542971 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556add840b10 con 0x556add95d6e0 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fdda5ffb640 10 client.15063 handle_command_reply: tid=0 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 client.15063 shutdown 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d29d0 msgr2=0x556add8f8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d29d0 0x556add8f8370 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009b20 tx=0x7fddb40046c0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 shutdown_connections 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x556add95d6e0 0x556add95c3f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556add8ddbb0 0x556add8d7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556add8d32f0 0x556add8d72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 --2- 172.21.15.16:0/1400542971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556add8d29d0 0x556add8f8370 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 >> 172.21.15.16:0/1400542971 conn(0x556add8cef60 msgr2=0x556add8d5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fdda5ffb640 0 client.15063 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 shutdown_connections 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.745+0000 7fddca6cd740 1 -- 172.21.15.16:0/1400542971 wait complete. 2024-09-15T22:37:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddca6cd740 20 client.15063 trim_cache size 0 max 16384 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24469, 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 68.369355057999996, 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 87, 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 85, 2024-09-15T22:37:52.754 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 123.553923994 2024-09-15T22:37:52.755 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:37:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdda80a3350 msgr2=0x7fdda80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdda80a3350 0x7fdda80a5810 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fddb00c5800 tx=0x7fddb002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 msgr2=0x7fddc0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc0074f70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fddb0031a20 tx=0x7fddb0031a50 comp rx=0 tx=0).stop 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 shutdown_connections 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fdda80a3350 0x7fdda80a5810 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc0106f10 0x7fddc0074f70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fddc0102790 0x7fddc0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 --2- 172.21.15.16:0/458628961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fddc0101de0 0x7fddc00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 >> 172.21.15.16:0/458628961 conn(0x7fddc00fd8c0 msgr2=0x7fddc00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 shutdown_connections 2024-09-15T22:37:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:52.746+0000 7fddc82f4640 1 -- 172.21.15.16:0/458628961 wait complete. 2024-09-15T22:37:53.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:37:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:54.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:53 smithi130 ceph-mon[35742]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:54.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:53 smithi016 ceph-mon[31492]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:54.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:53 smithi059 ceph-mon[34872]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 -- 172.21.15.16:0/438045629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 msgr2=0x7f665410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 --2- 172.21.15.16:0/438045629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665410ea90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f664c009ca0 tx=0x7f664c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 -- 172.21.15.16:0/438045629 shutdown_connections 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 --2- 172.21.15.16:0/438045629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665410ea90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 --2- 172.21.15.16:0/438045629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654102790 0x7f66541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 --2- 172.21.15.16:0/438045629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6654101de0 0x7f66541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 -- 172.21.15.16:0/438045629 >> 172.21.15.16:0/438045629 conn(0x7f66540fd8c0 msgr2=0x7f66540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 -- 172.21.15.16:0/438045629 shutdown_connections 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 -- 172.21.15.16:0/438045629 wait complete. 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 Processor -- start 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.319+0000 7f6659d01640 1 -- start start 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6659d01640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654101de0 0x7f6654198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6659d01640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6654102790 0x7f6654198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6659d01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6659d01640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6654111fb0 con 0x7f6654106f10 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6659d01640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6654111e30 con 0x7f6654101de0 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6659d01640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6654112130 con 0x7f6654102790 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45296/0 (socket says 172.21.15.16:45296) 2024-09-15T22:37:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 -- 172.21.15.16:0/1892074122 learned_addr learned my addr 172.21.15.16:0/1892074122 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6652ffd640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6654102790 0x7f6654198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f66537fe640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654101de0 0x7f6654198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 -- 172.21.15.16:0/1892074122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6654102790 msgr2=0x7f6654198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6654102790 0x7f6654198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 -- 172.21.15.16:0/1892074122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654101de0 msgr2=0x7f6654198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654101de0 0x7f6654198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.320+0000 7f6653fff640 1 -- 172.21.15.16:0/1892074122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665419d520 con 0x7f6654106f10 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.321+0000 7f6653fff640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665419ce40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f664c031a20 tx=0x7f664c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.321+0000 7f6650ff9640 1 -- 172.21.15.16:0/1892074122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c003b70 con 0x7f6654106f10 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.321+0000 7f6650ff9640 1 -- 172.21.15.16:0/1892074122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f664c003d10 con 0x7f6654106f10 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.321+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665419d7b0 con 0x7f6654106f10 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.321+0000 7f6650ff9640 1 -- 172.21.15.16:0/1892074122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c02fc80 con 0x7f6654106f10 2024-09-15T22:37:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.321+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66541a50f0 con 0x7f6654106f10 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.323+0000 7f6650ff9640 1 -- 172.21.15.16:0/1892074122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f664c0046a0 con 0x7f6654106f10 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 Processor -- start 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 -- start start 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd25db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd262f0 0x55ab8cd30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd32fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cc8c090 con 0x55ab8cd259d0 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cc33a70 con 0x55ab8cd30bb0 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f665c0da740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cc88050 con 0x55ab8cd262f0 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6650ff9640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f663c0a3350 0x7f663c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd25db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd25db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45302/0 (socket says 172.21.15.16:45302) 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 -- 172.21.15.16:0/849679629 learned_addr learned my addr 172.21.15.16:0/849679629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6653fff640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f663c0a3350 0x7f663c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f66537fe640 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd262f0 0x55ab8cd30670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6650ff9640 1 -- 172.21.15.16:0/1892074122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f664c0f0e40 con 0x7f6654106f10 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd32fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 -- 172.21.15.16:0/849679629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd262f0 msgr2=0x55ab8cd30670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd262f0 0x55ab8cd30670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 -- 172.21.15.16:0/849679629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd30bb0 msgr2=0x55ab8cd32fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd32fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.324+0000 7f6652ffd640 1 -- 172.21.15.16:0/849679629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab8cc40130 con 0x55ab8cd259d0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.325+0000 7f6652ffd640 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd25db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6648006aa0 tx=0x7f66480095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cdd5e089f37994 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.325+0000 7f6653fff640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f663c0a3350 0x7f663c0a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f664c0b9b10 tx=0x7f664c08cdb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.325+0000 7f6639ffb640 1 -- 172.21.15.16:0/849679629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648017610 con 0x55ab8cd259d0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.325+0000 7f6639ffb640 1 -- 172.21.15.16:0/849679629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f664801c070 con 0x55ab8cd259d0 2024-09-15T22:37:54.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.325+0000 7f6639ffb640 1 -- 172.21.15.16:0/849679629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66480159d0 con 0x55ab8cd259d0 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 -- 172.21.15.16:0/849679629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 msgr2=0x55ab8cd25db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd25db0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6648006aa0 tx=0x7f66480095d0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 -- 172.21.15.16:0/849679629 shutdown_connections 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd32fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd262f0 0x55ab8cd30670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 --2- 172.21.15.16:0/849679629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd25db0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 -- 172.21.15.16:0/849679629 >> 172.21.15.16:0/849679629 conn(0x55ab8cd21f60 msgr2=0x55ab8cd22340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 -- 172.21.15.16:0/849679629 shutdown_connections 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 -- 172.21.15.16:0/849679629 wait complete. 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 -- start start 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 Processor -- start 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd30440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.326+0000 7f665c0da740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f665c0da740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd2dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f665c0da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd33780 con 0x55ab8cd259d0 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f665c0da740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cc46f20 con 0x55ab8cd262f0 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f665c0da740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd33900 con 0x55ab8cd30bb0 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f665c0da740 10 client.0 _ll_register_callbacks cb 0x55ab8cc10030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd30440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd30440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45318/0 (socket says 172.21.15.16:45318) 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 -- 172.21.15.16:0/2169456938 learned_addr learned my addr 172.21.15.16:0/2169456938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f66537fe640 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd2dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 -- 172.21.15.16:0/2169456938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 msgr2=0x55ab8cd2dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd2dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 -- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd262f0 msgr2=0x55ab8cd2d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 -- 172.21.15.16:0/2169456938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab8c9fd2b0 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6652ffd640 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd30440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f66480156b0 tx=0x7f66480047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168651f1bec2deb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6632ffd640 1 -- 172.21.15.16:0/2169456938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648015b90 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f6632ffd640 1 -- 172.21.15.16:0/2169456938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6648015d30 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.327+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f664801c070 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f6632ffd640 1 -- 172.21.15.16:0/2169456938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6648017410 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 10 client.15075 fetch_fsmap learned FSMap version 25 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab8cd3f090 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f6632ffd640 1 -- 172.21.15.16:0/2169456938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f664801c070 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f6632ffd640 1 -- 172.21.15.16:0/2169456938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6648020680 con 0x55ab8cd259d0 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 10 client.15075 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 10 client.15075 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:37:54.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 20 client.15075 populate_metadata read hostname 'smithi016' 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cd412f0 0x55ab8cd42760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 4 client.15075 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab8cc99600 con 0x55ab8cd412f0 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.328+0000 7f66537fe640 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cd412f0 0x55ab8cd42760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.329+0000 7f66537fe640 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cd412f0 0x55ab8cd42760 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b60beb7238f06e86 server_cookie=2a28365b4370f5d8 in_seq=0 out_seq=0 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.329+0000 7f6632ffd640 10 client.15075 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f6632ffd640 1 -- 172.21.15.16:0/2169456938 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab8cc99600 con 0x55ab8cd412f0 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f6632ffd640 10 client.15075 handle_command_reply: tid=0 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 client.15075 shutdown 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 msgr2=0x55ab8cd30440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd30440 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f66480156b0 tx=0x7f66480047d0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 shutdown_connections 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cd412f0 0x55ab8cd42760 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd2dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f6632ffd640 0 client.15075 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:37:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 --2- 172.21.15.16:0/2169456938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd259d0 0x55ab8cd30440 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 >> 172.21.15.16:0/2169456938 conn(0x55ab8cd21f60 msgr2=0x55ab8cd27c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 shutdown_connections 2024-09-15T22:37:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 1 -- 172.21.15.16:0/2169456938 wait complete. 2024-09-15T22:37:54.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.330+0000 7f665c0da740 20 client.15075 trim_cache size 0 max 16384 2024-09-15T22:37:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 Processor -- start 2024-09-15T22:37:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 -- start start 2024-09-15T22:37:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd40c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd4aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd40aa0 con 0x55ab8cd262f0 2024-09-15T22:37:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd41170 con 0x55ab8cd259d0 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.348+0000 7f665c0da740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd41840 con 0x55ab8cd30bb0 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45328/0 (socket says 172.21.15.16:45328) 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 -- 172.21.15.16:0/2918267623 learned_addr learned my addr 172.21.15.16:0/2918267623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f6652ffd640 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd40c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f6652ffd640 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd4aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 -- 172.21.15.16:0/2918267623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 msgr2=0x55ab8cd4aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd4aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 -- 172.21.15.16:0/2918267623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 msgr2=0x55ab8cd40c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd40c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 -- 172.21.15.16:0/2918267623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab8cd26120 con 0x55ab8cd262f0 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f6652ffd640 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd40c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:54.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66537fe640 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2c0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6644004280 tx=0x7f664400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827638c9b53dd81c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66397fa640 1 -- 172.21.15.16:0/2918267623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644022070 con 0x55ab8cd262f0 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66397fa640 1 -- 172.21.15.16:0/2918267623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6644011920 con 0x55ab8cd262f0 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.349+0000 7f66397fa640 1 -- 172.21.15.16:0/2918267623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664401c920 con 0x55ab8cd262f0 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 -- 172.21.15.16:0/2918267623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 msgr2=0x55ab8cd2c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2c0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6644004280 tx=0x7f664400be60 comp rx=0 tx=0).stop 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 -- 172.21.15.16:0/2918267623 shutdown_connections 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd4aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd2c0a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 --2- 172.21.15.16:0/2918267623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd40c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 -- 172.21.15.16:0/2918267623 >> 172.21.15.16:0/2918267623 conn(0x55ab8cd21f60 msgr2=0x55ab8cd1cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 -- 172.21.15.16:0/2918267623 shutdown_connections 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 -- 172.21.15.16:0/2918267623 wait complete. 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.350+0000 7f665c0da740 1 -- start start 2024-09-15T22:37:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 1 Processor -- start 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd4af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd44390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd425e0 con 0x55ab8cd262f0 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd41f10 con 0x55ab8cd259d0 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ab8cd42cb0 con 0x55ab8cd30bb0 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f665c0da740 10 client.0 _ll_register_callbacks cb 0x7f6648009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f66537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd44390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f66537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd44390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45338/0 (socket says 172.21.15.16:45338) 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f66537fe640 1 -- 172.21.15.16:0/561422648 learned_addr learned my addr 172.21.15.16:0/561422648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f6652ffd640 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd4af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f6652ffd640 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f66537fe640 1 -- 172.21.15.16:0/561422648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 msgr2=0x55ab8cd448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.351+0000 7f66537fe640 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f66537fe640 1 -- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 msgr2=0x55ab8cd4af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f66537fe640 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd4af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f66537fe640 1 -- 172.21.15.16:0/561422648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab8cd44f40 con 0x55ab8cd262f0 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f66537fe640 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd44390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6644004360 tx=0x7f664401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3d703e7833d5db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f6632ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f6632ffd640 1 -- 172.21.15.16:0/561422648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664402a070 con 0x55ab8cd262f0 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f6632ffd640 1 -- 172.21.15.16:0/561422648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6644011ad0 con 0x55ab8cd262f0 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6644011920 con 0x55ab8cd262f0 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f6632ffd640 1 -- 172.21.15.16:0/561422648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6644022040 con 0x55ab8cd262f0 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f665c0da740 10 client.15081 fetch_fsmap learned FSMap version 25 2024-09-15T22:37:54.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab8cd49870 con 0x55ab8cd262f0 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.352+0000 7f6632ffd640 1 -- 172.21.15.16:0/561422648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6644011920 con 0x55ab8cd262f0 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f6632ffd640 1 -- 172.21.15.16:0/561422648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6644004970 con 0x55ab8cd262f0 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f665c0da740 10 client.15081 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f665c0da740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f665c0da740 20 client.15081 populate_metadata read hostname 'smithi016' 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f665c0da740 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cdaff90 0x55ab8cdaeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f665c0da740 4 client.15081 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "status"}) -- 0x55ab8cc93b10 con 0x55ab8cdaff90 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.353+0000 7f6652ffd640 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cdaff90 0x55ab8cdaeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f6632ffd640 10 client.15081 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f6652ffd640 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cdaff90 0x55ab8cdaeca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1561d2753d26703 server_cookie=3c3a8b2ade23fc82 in_seq=0 out_seq=0 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f6632ffd640 1 -- 172.21.15.16:0/561422648 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ab8cc93b10 con 0x55ab8cdaff90 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f6632ffd640 10 client.15081 handle_command_reply: tid=0 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 client.15081 shutdown 2024-09-15T22:37:54.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 msgr2=0x55ab8cd44390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd44390 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6644004360 tx=0x7f664401c580 comp rx=0 tx=0).stop 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 shutdown_connections 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55ab8cdaff90 0x55ab8cdaeca0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ab8cd30bb0 0x55ab8cd448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ab8cd262f0 0x55ab8cd44390 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f6632ffd640 0 client.15081 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 --2- 172.21.15.16:0/561422648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ab8cd259d0 0x55ab8cd4af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 >> 172.21.15.16:0/561422648 conn(0x55ab8cd21f60 msgr2=0x55ab8cd22340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 shutdown_connections 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 1 -- 172.21.15.16:0/561422648 wait complete. 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.354+0000 7f665c0da740 20 client.15081 trim_cache size 0 max 16384 2024-09-15T22:37:54.367 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24479, 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 70.982886221000001, 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 87, 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 85, 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 127.701213809 2024-09-15T22:37:54.368 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:37:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f663c0a3350 msgr2=0x7f663c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f663c0a3350 0x7f663c0a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f664c0b9b10 tx=0x7f664c08cdb0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 msgr2=0x7f665419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665419ce40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f664c031a20 tx=0x7f664c031a50 comp rx=0 tx=0).stop 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 shutdown_connections 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f663c0a3350 0x7f663c0a5810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6654106f10 0x7f665419ce40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6654102790 0x7f6654198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 --2- 172.21.15.16:0/1892074122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6654101de0 0x7f6654198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 >> 172.21.15.16:0/1892074122 conn(0x7f66540fd8c0 msgr2=0x7f66540fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 shutdown_connections 2024-09-15T22:37:54.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:54.355+0000 7f6659d01640 1 -- 172.21.15.16:0/1892074122 wait complete. 2024-09-15T22:37:55.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:37:55.417 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:55.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:55 smithi016 ceph-mon[31492]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:55.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 -- 172.21.15.16:0/602230166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810103660 msgr2=0x7f781010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 --2- 172.21.15.16:0/602230166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810103660 0x7f781010bb50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009900 tx=0x7f77f802a480 comp rx=0 tx=0).stop 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 -- 172.21.15.16:0/602230166 shutdown_connections 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 --2- 172.21.15.16:0/602230166 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f781010c170 0x7f781010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 --2- 172.21.15.16:0/602230166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810103660 0x7f781010bb50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 --2- 172.21.15.16:0/602230166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810102d40 0x7f7810103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 -- 172.21.15.16:0/602230166 >> 172.21.15.16:0/602230166 conn(0x7f78100fc7a0 msgr2=0x7f78100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 -- 172.21.15.16:0/602230166 shutdown_connections 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.896+0000 7f7815227640 1 -- 172.21.15.16:0/602230166 wait complete. 2024-09-15T22:37:55.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 Processor -- start 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 -- start start 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810102d40 0x7f78101009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810103660 0x7f7810100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f781010c170 0x7f7810101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7810111f70 con 0x7f7810102d40 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7810111df0 con 0x7f7810103660 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7815227640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f78101120f0 con 0x7f781010c170 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810102d40 0x7f78101009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810102d40 0x7f78101009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45356/0 (socket says 172.21.15.16:45356) 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 -- 172.21.15.16:0/1799530597 learned_addr learned my addr 172.21.15.16:0/1799530597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7807fff640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810103660 0x7f7810100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 -- 172.21.15.16:0/1799530597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f781010c170 msgr2=0x7f7810101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780f59e640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f781010c170 0x7f7810101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f781010c170 0x7f7810101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 -- 172.21.15.16:0/1799530597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810103660 msgr2=0x7f7810100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810103660 0x7f7810100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780ed9d640 1 -- 172.21.15.16:0/1799530597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78101a6890 con 0x7f7810102d40 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f7807fff640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810103660 0x7f7810100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.897+0000 7f780f59e640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f781010c170 0x7f7810101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.898+0000 7f780ed9d640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810102d40 0x7f78101009a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f780000c920 tx=0x7f780000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.898+0000 7f780cd99640 1 -- 172.21.15.16:0/1799530597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800007c70 con 0x7f7810102d40 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.898+0000 7f780cd99640 1 -- 172.21.15.16:0/1799530597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7800007e10 con 0x7f7810102d40 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.898+0000 7f780cd99640 1 -- 172.21.15.16:0/1799530597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800018890 con 0x7f7810102d40 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.898+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78101a6b80 con 0x7f7810102d40 2024-09-15T22:37:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.898+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101a7110 con 0x7f7810102d40 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.900+0000 7f780cd99640 1 -- 172.21.15.16:0/1799530597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7800018a70 con 0x7f7810102d40 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.900+0000 7f7817600740 1 Processor -- start 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.900+0000 7f7817600740 1 -- start start 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f7817600740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b4482cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f7817600740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b44837670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f7817600740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44839fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f7817600740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559b44793090 con 0x559b4482c9d0 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f7817600740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b4473aa70 con 0x559b4482d2f0 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f7817600740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559b4478f050 con 0x559b44837bb0 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780cd99640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f77fc0a3350 0x7f77fc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b4482cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b4482cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45358/0 (socket says 172.21.15.16:45358) 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 -- 172.21.15.16:0/594760792 learned_addr learned my addr 172.21.15.16:0/594760792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b44837670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f77fc0a3350 0x7f77fc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f7807fff640 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44839fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780cd99640 1 -- 172.21.15.16:0/1799530597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f78000cff60 con 0x7f7810102d40 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 -- 172.21.15.16:0/594760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 msgr2=0x559b44839fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44839fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 -- 172.21.15.16:0/594760792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 msgr2=0x559b44837670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b44837670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780f59e640 1 -- 172.21.15.16:0/594760792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b44747130 con 0x559b4482c9d0 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.901+0000 7f780ed9d640 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b44837670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.902+0000 7f780f59e640 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b4482cdb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7808014df0 tx=0x7f78080117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be267348a4262fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.902+0000 7f780f59e640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f77fc0a3350 0x7f77fc0a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7808011cb0 tx=0x7f780801b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.902+0000 7f78057fa640 1 -- 172.21.15.16:0/594760792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808021070 con 0x559b4482c9d0 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.902+0000 7f78057fa640 1 -- 172.21.15.16:0/594760792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780800fd40 con 0x559b4482c9d0 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.902+0000 7f78057fa640 1 -- 172.21.15.16:0/594760792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808013aa0 con 0x559b4482c9d0 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 -- 172.21.15.16:0/594760792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 msgr2=0x559b4482cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b4482cdb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7808014df0 tx=0x7f78080117b0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 -- 172.21.15.16:0/594760792 shutdown_connections 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44839fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b44837670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 --2- 172.21.15.16:0/594760792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b4482cdb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 -- 172.21.15.16:0/594760792 >> 172.21.15.16:0/594760792 conn(0x559b44828f60 msgr2=0x559b44829340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 -- 172.21.15.16:0/594760792 shutdown_connections 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 -- 172.21.15.16:0/594760792 wait complete. 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 -- start start 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.903+0000 7f7817600740 1 Processor -- start 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7817600740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b448374a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7817600740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482d2f0 0x559b448347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7817600740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b44837bb0 0x559b44834d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7817600740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559b4483a780 con 0x559b4482c9d0 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7817600740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b4474df20 con 0x559b44837bb0 2024-09-15T22:37:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7817600740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559b4483a900 con 0x559b4482d2f0 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7817600740 10 client.0 _ll_register_callbacks cb 0x559b44717030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b448374a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b448374a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45370/0 (socket says 172.21.15.16:45370) 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 -- 172.21.15.16:0/2865819535 learned_addr learned my addr 172.21.15.16:0/2865819535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f7807fff640 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482d2f0 0x559b448347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780f59e640 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b44837bb0 0x559b44834d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 -- 172.21.15.16:0/2865819535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482d2f0 msgr2=0x559b448347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482d2f0 0x559b448347f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 -- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b44837bb0 msgr2=0x559b44834d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b44837bb0 0x559b44834d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 -- 172.21.15.16:0/2865819535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b445042b0 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f780ed9d640 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b448374a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7800007590 tx=0x7f780006bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4a1d5fb8766890 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.904+0000 7f77f67fc640 1 -- 172.21.15.16:0/2865819535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000a4d30 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f77f67fc640 1 -- 172.21.15.16:0/2865819535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f780004d1b0 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780800fd40 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f77f67fc640 1 -- 172.21.15.16:0/2865819535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f78000cd800 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 10 client.15093 fetch_fsmap learned FSMap version 25 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b448461a0 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f77f67fc640 1 -- 172.21.15.16:0/2865819535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f780800fd40 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f77f67fc640 1 -- 172.21.15.16:0/2865819535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7800066ac0 con 0x559b4482c9d0 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 10 client.15093 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:37:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 20 client.15093 populate_metadata read hostname 'smithi016' 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b44848690 0x559b44849b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 4 client.15093 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b447a0600 con 0x559b44848690 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.905+0000 7f7807fff640 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b44848690 0x559b44849b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.906+0000 7f7807fff640 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b44848690 0x559b44849b00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13617b9cfc6f67ea server_cookie=b37400d83bf9cd59 in_seq=0 out_seq=0 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.906+0000 7f77f67fc640 10 client.15093 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.906+0000 7f77f67fc640 1 -- 172.21.15.16:0/2865819535 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b447a0600 con 0x559b44848690 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.906+0000 7f77f67fc640 10 client.15093 handle_command_reply: tid=0 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.906+0000 7f7817600740 1 client.15093 shutdown 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 msgr2=0x559b448374a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b448374a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7800007590 tx=0x7f780006bd30 comp rx=0 tx=0).stop 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 shutdown_connections 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b44848690 0x559b44849b00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b44837bb0 0x559b44834d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482d2f0 0x559b448347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f77f67fc640 0 client.15093 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 --2- 172.21.15.16:0/2865819535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482c9d0 0x559b448374a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 >> 172.21.15.16:0/2865819535 conn(0x559b44828f60 msgr2=0x559b4482ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 shutdown_connections 2024-09-15T22:37:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 1 -- 172.21.15.16:0/2865819535 wait complete. 2024-09-15T22:37:55.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.907+0000 7f7817600740 20 client.15093 trim_cache size 0 max 16384 2024-09-15T22:37:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 Processor -- start 2024-09-15T22:37:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 -- start start 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482c9d0 0x559b448340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482d2f0 0x559b44833100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44831ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559b44847e40 con 0x559b4482d2f0 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b44848510 con 0x559b4482c9d0 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7817600740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559b44848be0 con 0x559b44837bb0 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482d2f0 0x559b44833100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7807fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482d2f0 0x559b44833100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45376/0 (socket says 172.21.15.16:45376) 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.925+0000 7f7807fff640 1 -- 172.21.15.16:0/984369673 learned_addr learned my addr 172.21.15.16:0/984369673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f780f59e640 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44831ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f780ed9d640 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482c9d0 0x559b448340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7807fff640 1 -- 172.21.15.16:0/984369673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 msgr2=0x559b44831ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7807fff640 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44831ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7807fff640 1 -- 172.21.15.16:0/984369673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482c9d0 msgr2=0x559b448340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7807fff640 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482c9d0 0x559b448340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7807fff640 1 -- 172.21.15.16:0/984369673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4483a890 con 0x559b4482d2f0 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f780ed9d640 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482c9d0 0x559b448340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7807fff640 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482d2f0 0x559b44833100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005940 tx=0x7f77f8031780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5c0ec5e42ceb22 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7804ff9640 1 -- 172.21.15.16:0/984369673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8035040 con 0x559b4482d2f0 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7804ff9640 1 -- 172.21.15.16:0/984369673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f8005500 con 0x559b4482d2f0 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.926+0000 7f7804ff9640 1 -- 172.21.15.16:0/984369673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8004d60 con 0x559b4482d2f0 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 -- 172.21.15.16:0/984369673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482d2f0 msgr2=0x559b44833100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482d2f0 0x559b44833100 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005940 tx=0x7f77f8031780 comp rx=0 tx=0).stop 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 -- 172.21.15.16:0/984369673 shutdown_connections 2024-09-15T22:37:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b44837bb0 0x559b44831ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b4482d2f0 0x559b44833100 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 --2- 172.21.15.16:0/984369673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482c9d0 0x559b448340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 -- 172.21.15.16:0/984369673 >> 172.21.15.16:0/984369673 conn(0x559b44828f60 msgr2=0x559b44829340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 -- 172.21.15.16:0/984369673 shutdown_connections 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 -- 172.21.15.16:0/984369673 wait complete. 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.927+0000 7f7817600740 1 -- start start 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 1 Processor -- start 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482c9d0 0x559b44852340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b4484b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b44837bb0 0x559b4484bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559b44849980 con 0x559b44837bb0 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559b448492b0 con 0x559b4482d2f0 2024-09-15T22:37:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559b4484a050 con 0x559b4482c9d0 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7817600740 10 client.0 _ll_register_callbacks cb 0x7f780009bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b44837bb0 0x559b4484bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b44837bb0 0x559b4484bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45380/0 (socket says 172.21.15.16:45380) 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 -- 172.21.15.16:0/843000231 learned_addr learned my addr 172.21.15.16:0/843000231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780ed9d640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482c9d0 0x559b44852340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f7807fff640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b4484b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 -- 172.21.15.16:0/843000231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482c9d0 msgr2=0x559b44852340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482c9d0 0x559b44852340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 -- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 msgr2=0x559b4484b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b4484b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780f59e640 1 -- 172.21.15.16:0/843000231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4484c1f0 con 0x559b44837bb0 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.928+0000 7f780ed9d640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482c9d0 0x559b44852340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f780f59e640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b44837bb0 0x559b4484bbb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7808015010 tx=0x7f7808011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcbd2d5d5d1e5ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f77f67fc640 1 -- 172.21.15.16:0/843000231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808035020 con 0x559b44837bb0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f77f67fc640 1 -- 172.21.15.16:0/843000231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f780800fd40 con 0x559b44837bb0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f8005500 con 0x559b44837bb0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f77f67fc640 1 -- 172.21.15.16:0/843000231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f780803d050 con 0x559b44837bb0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 10 client.15099 fetch_fsmap learned FSMap version 25 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b44850c00 con 0x559b44837bb0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f77f67fc640 1 -- 172.21.15.16:0/843000231 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f8005500 con 0x559b44837bb0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f77f67fc640 1 -- 172.21.15.16:0/843000231 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7808030070 con 0x559b44837bb0 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 10 client.15099 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 20 client.15099 populate_metadata read hostname 'smithi016' 2024-09-15T22:37:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b448b74a0 0x559b448b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:55.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 4 client.15099 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:37:55.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.929+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "status"}) -- 0x559b4479ab10 con 0x559b448b74a0 2024-09-15T22:37:55.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.930+0000 7f780ed9d640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b448b74a0 0x559b448b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:55.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.930+0000 7f780ed9d640 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b448b74a0 0x559b448b61b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11add655112673b server_cookie=6320e3b8fc0b2c67 in_seq=0 out_seq=0 2024-09-15T22:37:55.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.930+0000 7f77f67fc640 10 client.15099 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:37:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.930+0000 7f77f67fc640 1 -- 172.21.15.16:0/843000231 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559b4479ab10 con 0x559b448b74a0 2024-09-15T22:37:55.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.930+0000 7f77f67fc640 10 client.15099 handle_command_reply: tid=0 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 client.15099 shutdown 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b44837bb0 msgr2=0x559b4484bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b44837bb0 0x559b4484bbb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7808015010 tx=0x7f7808011f80 comp rx=0 tx=0).stop 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 shutdown_connections 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559b448b74a0 0x559b448b61b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559b44837bb0 0x559b4484bbb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559b4482d2f0 0x559b4484b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 --2- 172.21.15.16:0/843000231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559b4482c9d0 0x559b44852340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 >> 172.21.15.16:0/843000231 conn(0x559b44828f60 msgr2=0x559b44832d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f77f67fc640 0 client.15099 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:37:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 shutdown_connections 2024-09-15T22:37:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 1 -- 172.21.15.16:0/843000231 wait complete. 2024-09-15T22:37:55.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7817600740 20 client.15099 trim_cache size 0 max 16384 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 106.680084248, 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 87, 2024-09-15T22:37:55.943 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 85, 2024-09-15T22:37:55.944 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 117.531238532 2024-09-15T22:37:55.944 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:37:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f77fc0a3350 msgr2=0x7f77fc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f77fc0a3350 0x7f77fc0a5810 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7808011cb0 tx=0x7f780801b000 comp rx=0 tx=0).stop 2024-09-15T22:37:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810102d40 msgr2=0x7f78101009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810102d40 0x7f78101009a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f780000c920 tx=0x7f780000cde0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 shutdown_connections 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f77fc0a3350 0x7f77fc0a5810 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f781010c170 0x7f7810101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7810103660 0x7f7810100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 --2- 172.21.15.16:0/1799530597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7810102d40 0x7f78101009a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.931+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 >> 172.21.15.16:0/1799530597 conn(0x7f78100fc7a0 msgr2=0x7f78100fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.932+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 shutdown_connections 2024-09-15T22:37:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:55.932+0000 7f7815227640 1 -- 172.21.15.16:0/1799530597 wait complete. 2024-09-15T22:37:56.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:55 smithi130 ceph-mon[35742]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:56.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:55 smithi059 ceph-mon[34872]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:37:56.769 INFO:tasks.cephadm:Setup complete, yielding 2024-09-15T22:37:56.769 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:37:56.781 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:37:56.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-15T22:37:57.042 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:57.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 -- 172.21.15.16:0/685881759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f81037c0 msgr2=0x7f90f810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 --2- 172.21.15.16:0/685881759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f81037c0 0x7f90f810a050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009ca0 tx=0x7f90f002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 -- 172.21.15.16:0/685881759 shutdown_connections 2024-09-15T22:37:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 --2- 172.21.15.16:0/685881759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f81037c0 0x7f90f810a050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 --2- 172.21.15.16:0/685881759 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f8102e00 0x7f90f8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 --2- 172.21.15.16:0/685881759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f8101c00 0x7f90f8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 -- 172.21.15.16:0/685881759 >> 172.21.15.16:0/685881759 conn(0x7f90f80fd390 msgr2=0x7f90f80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 -- 172.21.15.16:0/685881759 shutdown_connections 2024-09-15T22:37:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.556+0000 7f90fe70b640 1 -- 172.21.15.16:0/685881759 wait complete. 2024-09-15T22:37:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.557+0000 7f90fe70b640 1 Processor -- start 2024-09-15T22:37:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.557+0000 7f90fe70b640 1 -- start start 2024-09-15T22:37:57.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90fe70b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f8101c00 0x7f90f8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:57.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90fe70b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f8102e00 0x7f90f80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:57.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90fe70b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f81037c0 0x7f90f8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:57.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90fe70b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90f810d8b0 con 0x7f90f8101c00 2024-09-15T22:37:57.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90fe70b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90f810d730 con 0x7f90f8102e00 2024-09-15T22:37:57.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90fe70b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90f810da30 con 0x7f90f81037c0 2024-09-15T22:37:57.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f8101c00 0x7f90f8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:57.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f8101c00 0x7f90f8075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45394/0 (socket says 172.21.15.16:45394) 2024-09-15T22:37:57.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 -- 172.21.15.16:0/4167917674 learned_addr learned my addr 172.21.15.16:0/4167917674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:57.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f77fe640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f8102e00 0x7f90f80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:57.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90fcca8640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f81037c0 0x7f90f8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:57.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 -- 172.21.15.16:0/4167917674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f81037c0 msgr2=0x7f90f8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f81037c0 0x7f90f8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 -- 172.21.15.16:0/4167917674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f8102e00 msgr2=0x7f90f80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f8102e00 0x7f90f80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 -- 172.21.15.16:0/4167917674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f80753e0 con 0x7f90f8101c00 2024-09-15T22:37:57.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f7fff640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f8101c00 0x7f90f8075f90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90e800d770 tx=0x7f90e800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.558+0000 7f90f57fa640 1 -- 172.21.15.16:0/4167917674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8017070 con 0x7f90f8101c00 2024-09-15T22:37:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.559+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f81a1f50 con 0x7f90f8101c00 2024-09-15T22:37:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.559+0000 7f90f57fa640 1 -- 172.21.15.16:0/4167917674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e8009d00 con 0x7f90f8101c00 2024-09-15T22:37:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.559+0000 7f90f57fa640 1 -- 172.21.15.16:0/4167917674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8003d30 con 0x7f90f8101c00 2024-09-15T22:37:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.559+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f81a2340 con 0x7f90f8101c00 2024-09-15T22:37:57.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.559+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f8108ed0 con 0x7f90f8101c00 2024-09-15T22:37:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.561+0000 7f90f57fa640 1 -- 172.21.15.16:0/4167917674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f90e8009820 con 0x7f90f8101c00 2024-09-15T22:37:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.561+0000 7f90f57fa640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90e00a32d0 0x7f90e00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.562+0000 7f90f77fe640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90e00a32d0 0x7f90e00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.562+0000 7f90f57fa640 1 -- 172.21.15.16:0/4167917674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f90e80cfb20 con 0x7f90f8101c00 2024-09-15T22:37:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.562+0000 7f90f77fe640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90e00a32d0 0x7f90e00a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90f8075d10 tx=0x7f90ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:57.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.565+0000 7f90f57fa640 1 -- 172.21.15.16:0/4167917674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90e809c320 con 0x7f90f8101c00 2024-09-15T22:37:57.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.755+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f90f8062630 con 0x7f90e00a32d0 2024-09-15T22:37:57.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.756+0000 7f90f57fa640 1 -- 172.21.15.16:0/4167917674 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f90f8062630 con 0x7f90e00a32d0 2024-09-15T22:37:57.757 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-09-15T22:37:57.758 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-09-15T22:37:57.758 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90e00a32d0 msgr2=0x7f90e00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90e00a32d0 0x7f90e00a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90f8075d10 tx=0x7f90ec00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f8101c00 msgr2=0x7f90f8075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f8101c00 0x7f90f8075f90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f90e800d770 tx=0x7f90e800db70 comp rx=0 tx=0).stop 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 shutdown_connections 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f90e00a32d0 0x7f90e00a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f90f81037c0 0x7f90f8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f8102e00 0x7f90f80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 --2- 172.21.15.16:0/4167917674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90f8101c00 0x7f90f8075f90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 >> 172.21.15.16:0/4167917674 conn(0x7f90f80fd390 msgr2=0x7f90f81060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:57.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 shutdown_connections 2024-09-15T22:37:57.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:57.758+0000 7f90fe70b640 1 -- 172.21.15.16:0/4167917674 wait complete. 2024-09-15T22:37:58.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:57 smithi130 ceph-mon[35742]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:58.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:57 smithi016 ceph-mon[31492]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:58.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:57 smithi059 ceph-mon[34872]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:37:58.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-15T22:37:58.780 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:59.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:58 smithi016 ceph-mon[31492]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:59.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:58 smithi130 ceph-mon[35742]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:59.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:58 smithi059 ceph-mon[34872]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 -- 172.21.15.16:0/2622922827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78101d10 msgr2=0x7f6d78102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 --2- 172.21.15.16:0/2622922827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78101d10 0x7f6d78102190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009900 tx=0x7f6d6002a410 comp rx=0 tx=0).stop 2024-09-15T22:37:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 -- 172.21.15.16:0/2622922827 shutdown_connections 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 --2- 172.21.15.16:0/2622922827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d781026d0 0x7f6d78108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 --2- 172.21.15.16:0/2622922827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78101d10 0x7f6d78102190 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 --2- 172.21.15.16:0/2622922827 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d78100b10 0x7f6d78100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 -- 172.21.15.16:0/2622922827 >> 172.21.15.16:0/2622922827 conn(0x7f6d780fc2a0 msgr2=0x7f6d780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 -- 172.21.15.16:0/2622922827 shutdown_connections 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.231+0000 7f6d7e001640 1 -- 172.21.15.16:0/2622922827 wait complete. 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 Processor -- start 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 -- start start 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78100b10 0x7f6d7810b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d78101d10 0x7f6d78109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d781026d0 0x7f6d7810a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7810d8c0 con 0x7f6d78100b10 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7810d740 con 0x7f6d78101d10 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d7e001640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7810da40 con 0x7f6d781026d0 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78100b10 0x7f6d7810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78100b10 0x7f6d7810b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45410/0 (socket says 172.21.15.16:45410) 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.232+0000 7f6d777fe640 1 -- 172.21.15.16:0/1071175888 learned_addr learned my addr 172.21.15.16:0/1071175888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d77fff640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d781026d0 0x7f6d7810a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d6effd640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d78101d10 0x7f6d78109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d777fe640 1 -- 172.21.15.16:0/1071175888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d781026d0 msgr2=0x7f6d7810a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d777fe640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d781026d0 0x7f6d7810a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d777fe640 1 -- 172.21.15.16:0/1071175888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d78101d10 msgr2=0x7f6d78109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d777fe640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d78101d10 0x7f6d78109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d777fe640 1 -- 172.21.15.16:0/1071175888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7810a8e0 con 0x7f6d78100b10 2024-09-15T22:37:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d77fff640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d781026d0 0x7f6d7810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d777fe640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78100b10 0x7f6d7810b490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800e9c0 tx=0x7f6d6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d757fa640 1 -- 172.21.15.16:0/1071175888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6800cd80 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d757fa640 1 -- 172.21.15.16:0/1071175888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d680035f0 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d781a2070 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d757fa640 1 -- 172.21.15.16:0/1071175888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d68010690 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.233+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d781a2530 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.234+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d78107eb0 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.236+0000 7f6d757fa640 1 -- 172.21.15.16:0/1071175888 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6d680037a0 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.236+0000 7f6d757fa640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6d640a32d0 0x7f6d640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.237+0000 7f6d6effd640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6d640a32d0 0x7f6d640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.237+0000 7f6d757fa640 1 -- 172.21.15.16:0/1071175888 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6d680d02a0 con 0x7f6d78100b10 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.237+0000 7f6d6effd640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6d640a32d0 0x7f6d640a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d600314e0 tx=0x7f6d60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.239+0000 7f6d757fa640 1 -- 172.21.15.16:0/1071175888 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d6809cb00 con 0x7f6d78100b10 2024-09-15T22:37:59.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.427+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6d78062630 con 0x7f6d640a32d0 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.439+0000 7f6d757fa640 1 -- 172.21.15.16:0/1071175888 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7f6d78062630 con 0x7f6d640a32d0 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (2m) 52s ago 2m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c f678fa8d9cad 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi059 running (2m) 53s ago 2m 26.5M - 19.1.1-326-g8921c8ef f1455bd0219c 6d8fe8545d83 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi130 running (2m) 54s ago 2m 17.0M - 19.1.1-326-g8921c8ef f1455bd0219c 830d2a77a2bc 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (2m) 52s ago 2m 14.3M - 19.1.1-326-g8921c8ef f1455bd0219c 0d040ce8a246 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi059 running (2m) 53s ago 2m 15.9M - 19.1.1-326-g8921c8ef f1455bd0219c 02dbb5fb6282 2024-09-15T22:37:59.441 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi130 running (2m) 54s ago 2m 15.6M - 19.1.1-326-g8921c8ef f1455bd0219c ef1819beb068 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (2m) 52s ago 2m 14.2M - 19.1.1-326-g8921c8ef f1455bd0219c 7f183bd1c09f 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi059 running (2m) 53s ago 2m 16.3M - 19.1.1-326-g8921c8ef f1455bd0219c 55618dd24db2 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi130 running (2m) 54s ago 2m 18.4M - 19.1.1-326-g8921c8ef f1455bd0219c e5b968143979 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (2m) 52s ago 2m 15.9M - 19.1.1-326-g8921c8ef f1455bd0219c d1044db680af 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi059 running (2m) 53s ago 2m 14.3M - 19.1.1-326-g8921c8ef f1455bd0219c d51f617d1f26 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi130 running (2m) 54s ago 2m 16.0M - 19.1.1-326-g8921c8ef f1455bd0219c ab862784bd7a 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (9m) 52s ago 9m 509M - 19.1.1-326-g8921c8ef f1455bd0219c c77097c54c1f 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi059 *:8443,8765 running (7m) 53s ago 7m 441M - 19.1.1-326-g8921c8ef f1455bd0219c ac1b67580f65 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi130 *:8443,8765 running (7m) 54s ago 7m 444M - 19.1.1-326-g8921c8ef f1455bd0219c b016d80211b6 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (9m) 52s ago 9m 49.3M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 7ea2261f5ae7 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi059 running (8m) 53s ago 8m 42.8M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 69143470fcd6 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi130 running (8m) 54s ago 8m 40.5M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 4afbc3ac33c3 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (7m) 52s ago 7m 53.8M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 3752ea441a60 2024-09-15T22:37:59.442 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi059 running (7m) 53s ago 7m 56.6M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 4322a1cd0886 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi130 running (6m) 54s ago 6m 57.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 3bee4528cb3f 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (6m) 52s ago 6m 53.3M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 48b6a25f6d03 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi059 running (5m) 53s ago 5m 48.9M 4273M 19.1.1-326-g8921c8ef f1455bd0219c e2d27bc2b985 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi130 running (5m) 54s ago 5m 44.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5a8f98d84e1f 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (4m) 52s ago 4m 48.3M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 841debfe2272 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi059 running (4m) 53s ago 4m 52.0M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 5b647a19f940 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi130 running (4m) 54s ago 4m 52.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c d0e784dd317a 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 52s ago 3m 49.6M 4273M 19.1.1-326-g8921c8ef f1455bd0219c b17d9657ce21 2024-09-15T22:37:59.443 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi059 running (3m) 53s ago 3m 42.3M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 83570af48b2f 2024-09-15T22:37:59.444 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi130 running (2m) 54s ago 2m 41.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c e9d309afaa3a 2024-09-15T22:37:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6d640a32d0 msgr2=0x7f6d640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6d640a32d0 0x7f6d640a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d600314e0 tx=0x7f6d60000bc0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78100b10 msgr2=0x7f6d7810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78100b10 0x7f6d7810b490 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6d6800e9c0 tx=0x7f6d6800ee80 comp rx=0 tx=0).stop 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 shutdown_connections 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f6d640a32d0 0x7f6d640a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6d781026d0 0x7f6d7810a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d78101d10 0x7f6d78109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 --2- 172.21.15.16:0/1071175888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d78100b10 0x7f6d7810b490 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 >> 172.21.15.16:0/1071175888 conn(0x7f6d780fc2a0 msgr2=0x7f6d78105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 shutdown_connections 2024-09-15T22:37:59.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:37:59.442+0000 7f6d7e001640 1 -- 172.21.15.16:0/1071175888 wait complete. 2024-09-15T22:38:00.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:59 smithi130 ceph-mon[35742]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:00.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:37:59 smithi130 ceph-mon[35742]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:00.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:59 smithi016 ceph-mon[31492]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:00.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:37:59 smithi016 ceph-mon[31492]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:00.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-15T22:38:00.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:59 smithi059 ceph-mon[34872]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:00.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:37:59 smithi059 ceph-mon[34872]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 -- 172.21.15.16:0/739331612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c104ff0 msgr2=0x7f261c1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 --2- 172.21.15.16:0/739331612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c104ff0 0x7f261c1073e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f260c009900 tx=0x7f260c02a410 comp rx=0 tx=0).stop 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 -- 172.21.15.16:0/739331612 shutdown_connections 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 --2- 172.21.15.16:0/739331612 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261c107920 0x7f261c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 --2- 172.21.15.16:0/739331612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c104ff0 0x7f261c1073e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 --2- 172.21.15.16:0/739331612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f261c1026c0 0x7f261c104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 -- 172.21.15.16:0/739331612 >> 172.21.15.16:0/739331612 conn(0x7f261c0fc280 msgr2=0x7f261c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 -- 172.21.15.16:0/739331612 shutdown_connections 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 -- 172.21.15.16:0/739331612 wait complete. 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 Processor -- start 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.898+0000 7f262323c640 1 -- start start 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f262323c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261c1026c0 0x7f261c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f262323c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f261c104ff0 0x7f261c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f262323c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c107920 0x7f261c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f262323c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10d850 con 0x7f261c107920 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f262323c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10d6d0 con 0x7f261c104ff0 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f262323c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f261c10d9d0 con 0x7f261c1026c0 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c107920 0x7f261c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c107920 0x7f261c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45424/0 (socket says 172.21.15.16:45424) 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 -- 172.21.15.16:0/2970018535 learned_addr learned my addr 172.21.15.16:0/2970018535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f2620fd8640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261c1026c0 0x7f261c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f261bfff640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f261c104ff0 0x7f261c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 -- 172.21.15.16:0/2970018535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261c1026c0 msgr2=0x7f261c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261c1026c0 0x7f261c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 -- 172.21.15.16:0/2970018535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f261c104ff0 msgr2=0x7f261c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f261c104ff0 0x7f261c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 -- 172.21.15.16:0/2970018535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c19faa0 con 0x7f261c107920 2024-09-15T22:38:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f26217d9640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c107920 0x7f261c19f430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f261400eaa0 tx=0x7f261400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f2619ffb640 1 -- 172.21.15.16:0/2970018535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614004d20 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f2619ffb640 1 -- 172.21.15.16:0/2970018535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2614005020 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f2619ffb640 1 -- 172.21.15.16:0/2970018535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26140116d0 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.899+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261c19fd90 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.900+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261c1a01f0 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.901+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261c0fdd70 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.902+0000 7f2619ffb640 1 -- 172.21.15.16:0/2970018535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f26140051d0 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.902+0000 7f2619ffb640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f26080a3380 0x7f26080a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.903+0000 7f2620fd8640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f26080a3380 0x7f26080a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.903+0000 7f2619ffb640 1 -- 172.21.15.16:0/2970018535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f26140cff60 con 0x7f261c107920 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.903+0000 7f2620fd8640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f26080a3380 0x7f26080a5840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f260400abe0 tx=0x7f260400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:00.907+0000 7f2619ffb640 1 -- 172.21.15.16:0/2970018535 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2614016030 con 0x7f261c107920 2024-09-15T22:38:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.089+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f261c062630 con 0x7f26080a3380 2024-09-15T22:38:01.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.095+0000 7f2619ffb640 1 -- 172.21.15.16:0/2970018535 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f261c062630 con 0x7f26080a3380 2024-09-15T22:38:01.097 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-15T22:38:01.097 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 55s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi059=b;smithi059=e;smithi059=h;smithi059=k;smithi130=c;smithi130=f;smithi130=i;smithi130=l;count:12 2024-09-15T22:38:01.097 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 55s ago 7m smithi016=x;smithi059=y;smithi130=z;count:3 2024-09-15T22:38:01.097 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 55s ago 8m smithi016:172.21.15.16=a;smithi059:172.21.15.59=b;smithi130:172.21.15.130=c;count:3 2024-09-15T22:38:01.097 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 55s ago - 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f26080a3380 msgr2=0x7f26080a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f26080a3380 0x7f26080a5840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f260400abe0 tx=0x7f260400a2d0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c107920 msgr2=0x7f261c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c107920 0x7f261c19f430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f261400eaa0 tx=0x7f261400eea0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 shutdown_connections 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f26080a3380 0x7f26080a5840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261c107920 0x7f261c19f430 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f261c104ff0 0x7f261c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 --2- 172.21.15.16:0/2970018535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f261c1026c0 0x7f261c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 >> 172.21.15.16:0/2970018535 conn(0x7f261c0fc280 msgr2=0x7f261c103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.097+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 shutdown_connections 2024-09-15T22:38:01.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:01.098+0000 7f262323c640 1 -- 172.21.15.16:0/2970018535 wait complete. 2024-09-15T22:38:01.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-15T22:38:02.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:01 smithi130 ceph-mon[35742]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:38:02.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:01 smithi130 ceph-mon[35742]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:02.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:01 smithi016 ceph-mon[31492]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:38:02.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:01 smithi016 ceph-mon[31492]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:02.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:01 smithi059 ceph-mon[34872]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:38:02.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:01 smithi059 ceph-mon[34872]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 -- 172.21.15.16:0/3762291943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62281037c0 msgr2=0x7f622810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 --2- 172.21.15.16:0/3762291943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62281037c0 0x7f622810a050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6220009ca0 tx=0x7f622002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 -- 172.21.15.16:0/3762291943 shutdown_connections 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 --2- 172.21.15.16:0/3762291943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62281037c0 0x7f622810a050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 --2- 172.21.15.16:0/3762291943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6228102e00 0x7f6228103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 --2- 172.21.15.16:0/3762291943 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6228101c00 0x7f6228102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 -- 172.21.15.16:0/3762291943 >> 172.21.15.16:0/3762291943 conn(0x7f62280fd390 msgr2=0x7f62280ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 -- 172.21.15.16:0/3762291943 shutdown_connections 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 -- 172.21.15.16:0/3762291943 wait complete. 2024-09-15T22:38:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 Processor -- start 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.659+0000 7f622edcd640 1 -- start start 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622edcd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6228101c00 0x7f6228197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622edcd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6228102e00 0x7f62281983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622edcd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62281037c0 0x7f622819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622edcd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f622810d880 con 0x7f6228102e00 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622edcd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f622810d700 con 0x7f6228101c00 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622edcd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f622810da00 con 0x7f62281037c0 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6228102e00 0x7f62281983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6228102e00 0x7f62281983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49658/0 (socket says 172.21.15.16:49658) 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 -- 172.21.15.16:0/3254461258 learned_addr learned my addr 172.21.15.16:0/3254461258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622cb69640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6228101c00 0x7f6228197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622d36a640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62281037c0 0x7f622819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 -- 172.21.15.16:0/3254461258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62281037c0 msgr2=0x7f622819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62281037c0 0x7f622819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 -- 172.21.15.16:0/3254461258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6228101c00 msgr2=0x7f6228197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6228101c00 0x7f6228197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 -- 172.21.15.16:0/3254461258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622819fad0 con 0x7f6228102e00 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f622d36a640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62281037c0 0x7f622819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6227fff640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6228102e00 0x7f62281983e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f621400d950 tx=0x7f621400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.660+0000 7f6225ffb640 1 -- 172.21.15.16:0/3254461258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214017030 con 0x7f6228102e00 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.661+0000 7f6225ffb640 1 -- 172.21.15.16:0/3254461258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6214004020 con 0x7f6228102e00 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.661+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622819fdc0 con 0x7f6228102e00 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.661+0000 7f6225ffb640 1 -- 172.21.15.16:0/3254461258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140047a0 con 0x7f6228102e00 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.661+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281a0270 con 0x7f6228102e00 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.662+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6228109000 con 0x7f6228102e00 2024-09-15T22:38:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.663+0000 7f6225ffb640 1 -- 172.21.15.16:0/3254461258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f62140041d0 con 0x7f6228102e00 2024-09-15T22:38:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.663+0000 7f6225ffb640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62180a3300 0x7f62180a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.664+0000 7f622cb69640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62180a3300 0x7f62180a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.664+0000 7f6225ffb640 1 -- 172.21.15.16:0/3254461258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f62140cf7c0 con 0x7f6228102e00 2024-09-15T22:38:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.664+0000 7f622cb69640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62180a3300 0x7f62180a57c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6228102c60 tx=0x7f621c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:02.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.668+0000 7f6225ffb640 1 -- 172.21.15.16:0/3254461258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f621409c020 con 0x7f6228102e00 2024-09-15T22:38:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.850+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f6228062630 con 0x7f62180a3300 2024-09-15T22:38:02.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.851+0000 7f6225ffb640 1 -- 172.21.15.16:0/3254461258 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f6228062630 con 0x7f62180a3300 2024-09-15T22:38:02.853 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-09-15T22:38:02.853 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-09-15T22:38:02.853 INFO:teuthology.orchestra.run.smithi016.stdout:smithi059 172.21.15.59 2024-09-15T22:38:02.853 INFO:teuthology.orchestra.run.smithi016.stdout:smithi130 172.21.15.130 2024-09-15T22:38:02.853 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-09-15T22:38:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62180a3300 msgr2=0x7f62180a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62180a3300 0x7f62180a57c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6228102c60 tx=0x7f621c005f90 comp rx=0 tx=0).stop 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6228102e00 msgr2=0x7f62281983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6228102e00 0x7f62281983e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f621400d950 tx=0x7f621400de10 comp rx=0 tx=0).stop 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 shutdown_connections 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f62180a3300 0x7f62180a57c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f62281037c0 0x7f622819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6228102e00 0x7f62281983e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 --2- 172.21.15.16:0/3254461258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6228101c00 0x7f6228197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 >> 172.21.15.16:0/3254461258 conn(0x7f62280fd390 msgr2=0x7f62281061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 shutdown_connections 2024-09-15T22:38:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:02.854+0000 7f622edcd640 1 -- 172.21.15.16:0/3254461258 wait complete. 2024-09-15T22:38:03.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-15T22:38:03.935 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:04.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:03 smithi016 ceph-mon[31492]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:38:04.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:03 smithi016 ceph-mon[31492]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:03 smithi059 ceph-mon[34872]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:38:04.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:03 smithi059 ceph-mon[34872]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:03 smithi130 ceph-mon[35742]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:38:04.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:03 smithi130 ceph-mon[35742]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 -- 172.21.15.16:0/3838338043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84481078c0 msgr2=0x7f8448109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3838338043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84481078c0 0x7f8448109d10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8440009900 tx=0x7f844002a410 comp rx=0 tx=0).stop 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 -- 172.21.15.16:0/3838338043 shutdown_connections 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3838338043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84481078c0 0x7f8448109d10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3838338043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8448104f90 0x7f8448107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3838338043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8448102660 0x7f8448104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 -- 172.21.15.16:0/3838338043 >> 172.21.15.16:0/3838338043 conn(0x7f84480fc280 msgr2=0x7f84480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 -- 172.21.15.16:0/3838338043 shutdown_connections 2024-09-15T22:38:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.434+0000 7f844cf1d640 1 -- 172.21.15.16:0/3838338043 wait complete. 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 Processor -- start 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 -- start start 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8448102660 0x7f844819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8448104f90 0x7f844819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84481078c0 0x7f84481a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f844810d880 con 0x7f8448102660 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f844810d700 con 0x7f8448104f90 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844cf1d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f844810da00 con 0x7f84481078c0 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8448102660 0x7f844819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8448102660 0x7f844819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49664/0 (socket says 172.21.15.16:49664) 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 -- 172.21.15.16:0/3977416854 learned_addr learned my addr 172.21.15.16:0/3977416854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f8445d9b640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8448104f90 0x7f844819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f8446d9d640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84481078c0 0x7f84481a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 -- 172.21.15.16:0/3977416854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84481078c0 msgr2=0x7f84481a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84481078c0 0x7f84481a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 -- 172.21.15.16:0/3977416854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8448104f90 msgr2=0x7f844819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8448104f90 0x7f844819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.435+0000 7f844659c640 1 -- 172.21.15.16:0/3977416854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84481a1d40 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.436+0000 7f844659c640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8448102660 0x7f844819a110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f843800eb90 tx=0x7f843800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.436+0000 7f843f7fe640 1 -- 172.21.15.16:0/3977416854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438004d20 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.436+0000 7f843f7fe640 1 -- 172.21.15.16:0/3977416854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843800ee90 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.436+0000 7f843f7fe640 1 -- 172.21.15.16:0/3977416854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438010830 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.436+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84481a1fd0 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.436+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84481a2430 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.437+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84481093b0 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.438+0000 7f843f7fe640 1 -- 172.21.15.16:0/3977416854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8438005020 con 0x7f8448102660 2024-09-15T22:38:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.439+0000 7f843f7fe640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f84340a3300 0x7f84340a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.439+0000 7f8445d9b640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f84340a3300 0x7f84340a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.439+0000 7f843f7fe640 1 -- 172.21.15.16:0/3977416854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f84380d0470 con 0x7f8448102660 2024-09-15T22:38:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.439+0000 7f8445d9b640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f84340a3300 0x7f84340a57c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f844819b630 tx=0x7f843000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.443+0000 7f843f7fe640 1 -- 172.21.15.16:0/3977416854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f843809cc70 con 0x7f8448102660 2024-09-15T22:38:04.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.624+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 --> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8448062630 con 0x7f84340a3300 2024-09-15T22:38:04.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.626+0000 7f843f7fe640 1 -- 172.21.15.16:0/3977416854 <== mgr.14150 v2:172.21.15.16:6800/81475521 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8448062630 con 0x7f84340a3300 2024-09-15T22:38:04.627 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-15T22:38:04.628 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:38:04.628 INFO:teuthology.orchestra.run.smithi016.stdout:smithi059 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009L400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:38:04.628 INFO:teuthology.orchestra.run.smithi016.stdout:smithi130 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500Q1400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-15T22:38:04.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.628+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f84340a3300 msgr2=0x7f84340a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.628+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f84340a3300 0x7f84340a57c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f844819b630 tx=0x7f843000b410 comp rx=0 tx=0).stop 2024-09-15T22:38:04.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8448102660 msgr2=0x7f844819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8448102660 0x7f844819a110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f843800eb90 tx=0x7f843800c6a0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 shutdown_connections 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f84340a3300 0x7f84340a57c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f84481078c0 0x7f84481a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8448104f90 0x7f844819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 --2- 172.21.15.16:0/3977416854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8448102660 0x7f844819a110 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 >> 172.21.15.16:0/3977416854 conn(0x7f84480fc280 msgr2=0x7f8448101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 shutdown_connections 2024-09-15T22:38:04.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:04.629+0000 7f844cf1d640 1 -- 172.21.15.16:0/3977416854 wait complete. 2024-09-15T22:38:05.454 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:38:05.464 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:38:05.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-15T22:38:05.723 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:06.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:05 smithi016 ceph-mon[31492]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:06.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:05 smithi016 ceph-mon[31492]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:06.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:05 smithi059 ceph-mon[34872]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:06.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:05 smithi059 ceph-mon[34872]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 -- 172.21.15.16:0/3744980908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8105030 msgr2=0x7f9ae8107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/3744980908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8105030 0x7f9ae8107420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009900 tx=0x7f9ad802a410 comp rx=0 tx=0).stop 2024-09-15T22:38:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 -- 172.21.15.16:0/3744980908 shutdown_connections 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/3744980908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ae8107960 0x7f9ae8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/3744980908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8105030 0x7f9ae8107420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/3744980908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae8102700 0x7f9ae8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 -- 172.21.15.16:0/3744980908 >> 172.21.15.16:0/3744980908 conn(0x7f9ae80fc280 msgr2=0x7f9ae80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 -- 172.21.15.16:0/3744980908 shutdown_connections 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 -- 172.21.15.16:0/3744980908 wait complete. 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 Processor -- start 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.177+0000 7f9aeeeff640 1 -- start start 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aeeeff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8102700 0x7f9ae8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aeeeff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae8105030 0x7f9ae81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aeeeff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ae8107960 0x7f9ae819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aeeeff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae810d830 con 0x7f9ae8102700 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aeeeff640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae810d6b0 con 0x7f9ae8105030 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aeeeff640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae810d9b0 con 0x7f9ae8107960 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8102700 0x7f9ae8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8102700 0x7f9ae8197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49676/0 (socket says 172.21.15.16:49676) 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 -- 172.21.15.16:0/1103598159 learned_addr learned my addr 172.21.15.16:0/1103598159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aed49c640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ae8107960 0x7f9ae819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9adffff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae8105030 0x7f9ae81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 -- 172.21.15.16:0/1103598159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ae8107960 msgr2=0x7f9ae819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ae8107960 0x7f9ae819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 -- 172.21.15.16:0/1103598159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae8105030 msgr2=0x7f9ae81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae8105030 0x7f9ae81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 -- 172.21.15.16:0/1103598159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae819fab0 con 0x7f9ae8102700 2024-09-15T22:38:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9adffff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae8105030 0x7f9ae81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.178+0000 7f9aecc9b640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8102700 0x7f9ae8197e80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400c920 tx=0x7f9ae400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.179+0000 7f9addffb640 1 -- 172.21.15.16:0/1103598159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4007c70 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.179+0000 7f9addffb640 1 -- 172.21.15.16:0/1103598159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae4007e10 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.179+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae819fda0 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.179+0000 7f9addffb640 1 -- 172.21.15.16:0/1103598159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4018890 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.179+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae81a01f0 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.180+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae80fdd70 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.182+0000 7f9addffb640 1 -- 172.21.15.16:0/1103598159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9ae4018a30 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.183+0000 7f9addffb640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9ad00a3300 0x7f9ad00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.183+0000 7f9addffb640 1 -- 172.21.15.16:0/1103598159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9ae40cffe0 con 0x7f9ae8102700 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.183+0000 7f9adffff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9ad00a3300 0x7f9ad00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.183+0000 7f9adffff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9ad00a3300 0x7f9ad00a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006010 tx=0x7f9ad8032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.187+0000 7f9addffb640 1 -- 172.21.15.16:0/1103598159 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ae4014030 con 0x7f9ae8102700 2024-09-15T22:38:06.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:05 smithi130 ceph-mon[35742]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:06.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:05 smithi130 ceph-mon[35742]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.405+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9ae802d7c0 con 0x7f9ae8102700 2024-09-15T22:38:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.406+0000 7f9addffb640 1 -- 172.21.15.16:0/1103598159 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3088 (secure 0 0 0) 0x7f9ae409c800 con 0x7f9ae8102700 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:e25 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:btime 2024-09-15T22:36:47:380763+0000 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 25 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-15T22:36:08.871593+0000 2024-09-15T22:38:06.408 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-15T22:36:47.103123+0000 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 3 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:in 0,1,2 2024-09-15T22:38:06.409 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24500,1=24479,2=24469} 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3,4] 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24500 members: 24469,24479,24500 2024-09-15T22:38:06.410 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24500} state up:active seq 4 addr [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{0:24457} state up:standby-replay seq 1 addr [v2:172.21.15.130:6836/1773539140,v1:172.21.15.130:6837/1773539140] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{1:24479} state up:active seq 16 addr [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{1:24452} state up:standby-replay seq 1 addr [v2:172.21.15.59:6834/2202482741,v1:172.21.15.59:6835/2202482741] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{2:24469} state up:active seq 15 addr [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{2:24445} state up:standby-replay seq 1 addr [v2:172.21.15.130:6834/2738921095,v1:172.21.15.130:6835/2738921095] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14532} state up:standby seq 1 addr [v2:172.21.15.16:6834/2727578831,v1:172.21.15.16:6835/2727578831] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14544} state up:standby seq 1 addr [v2:172.21.15.16:6836/4208747900,v1:172.21.15.16:6837/4208747900] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{-1:14592} state up:standby seq 1 addr [v2:172.21.15.16:6838/1483095493,v1:172.21.15.16:6839/1483095493] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{-1:14604} state up:standby seq 1 addr [v2:172.21.15.16:6840/433045305,v1:172.21.15.16:6841/433045305] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.412 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.130:6832/2632425547,v1:172.21.15.130:6833/2632425547] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.412 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{-1:24440} state up:standby seq 1 addr [v2:172.21.15.59:6832/1429864622,v1:172.21.15.59:6833/1429864622] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:38:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-15T22:38:06.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9ad00a3300 msgr2=0x7f9ad00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9ad00a3300 0x7f9ad00a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8006010 tx=0x7f9ad8032dd0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8102700 msgr2=0x7f9ae8197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8102700 0x7f9ae8197e80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400c920 tx=0x7f9ae400cde0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 shutdown_connections 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f9ad00a3300 0x7f9ad00a57c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9ae8107960 0x7f9ae819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae8105030 0x7f9ae81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 --2- 172.21.15.16:0/1103598159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ae8102700 0x7f9ae8197e80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 >> 172.21.15.16:0/1103598159 conn(0x7f9ae80fc280 msgr2=0x7f9ae81030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 shutdown_connections 2024-09-15T22:38:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:06.408+0000 7f9aeeeff640 1 -- 172.21.15.16:0/1103598159 wait complete. 2024-09-15T22:38:07.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:06 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1103598159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:38:07.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-15T22:38:07.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:06 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1103598159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:38:07.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:06 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1103598159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:38:07.464 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:07.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:07 smithi016 ceph-mon[31492]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:38:07.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:07 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:07.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:07 smithi059 ceph-mon[34872]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:38:07.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:07 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:08.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:07 smithi130 ceph-mon[35742]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:38:08.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:07 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 -- 172.21.15.16:0/2626523080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04101cd0 msgr2=0x7fcd04102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 --2- 172.21.15.16:0/2626523080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04101cd0 0x7fcd04102150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009900 tx=0x7fccf802a410 comp rx=0 tx=0).stop 2024-09-15T22:38:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 -- 172.21.15.16:0/2626523080 shutdown_connections 2024-09-15T22:38:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 --2- 172.21.15.16:0/2626523080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd04102690 0x7fcd04108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 --2- 172.21.15.16:0/2626523080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04101cd0 0x7fcd04102150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 --2- 172.21.15.16:0/2626523080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd04100ad0 0x7fcd04100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 -- 172.21.15.16:0/2626523080 >> 172.21.15.16:0/2626523080 conn(0x7fcd040fc280 msgr2=0x7fcd040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 -- 172.21.15.16:0/2626523080 shutdown_connections 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.583+0000 7fcd0890c640 1 -- 172.21.15.16:0/2626523080 wait complete. 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 Processor -- start 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 -- start start 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04100ad0 0x7fcd0419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd04101cd0 0x7fcd0419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd04102690 0x7fcd041a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0410d8e0 con 0x7fcd04100ad0 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0410d760 con 0x7fcd04101cd0 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0890c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0410da60 con 0x7fcd04102690 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04100ad0 0x7fcd0419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04100ad0 0x7fcd0419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49682/0 (socket says 172.21.15.16:49682) 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd0259c640 1 -- 172.21.15.16:0/3826839926 learned_addr learned my addr 172.21.15.16:0/3826839926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd01d9b640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd04101cd0 0x7fcd0419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.584+0000 7fcd02d9d640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd04102690 0x7fcd041a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0259c640 1 -- 172.21.15.16:0/3826839926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd04102690 msgr2=0x7fcd041a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0259c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd04102690 0x7fcd041a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0259c640 1 -- 172.21.15.16:0/3826839926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd04101cd0 msgr2=0x7fcd0419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0259c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd04101cd0 0x7fcd0419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0259c640 1 -- 172.21.15.16:0/3826839926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd041a1d40 con 0x7fcd04100ad0 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd02d9d640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd04102690 0x7fcd041a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd01d9b640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd04101cd0 0x7fcd0419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0259c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04100ad0 0x7fcd0419a110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9c0 tx=0x7fccf400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fccf37fe640 1 -- 172.21.15.16:0/3826839926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400cd80 con 0x7fcd04100ad0 2024-09-15T22:38:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fccf37fe640 1 -- 172.21.15.16:0/3826839926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccf40035f0 con 0x7fcd04100ad0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd041a1fd0 con 0x7fcd04100ad0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fccf37fe640 1 -- 172.21.15.16:0/3826839926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4010690 con 0x7fcd04100ad0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.585+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd041a2430 con 0x7fcd04100ad0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.586+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd041a26c0 con 0x7fcd04100ad0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.587+0000 7fccf37fe640 1 -- 172.21.15.16:0/3826839926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fccf40037a0 con 0x7fcd04100ad0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.588+0000 7fccf37fe640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fccec0a3300 0x7fccec0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.588+0000 7fcd01d9b640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fccec0a3300 0x7fccec0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.588+0000 7fccf37fe640 1 -- 172.21.15.16:0/3826839926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fccf40d0290 con 0x7fcd04100ad0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.589+0000 7fcd01d9b640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fccec0a3300 0x7fccec0a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fccf8031440 tx=0x7fccf80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.591+0000 7fccf37fe640 1 -- 172.21.15.16:0/3826839926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccf409caf0 con 0x7fcd04100ad0 2024-09-15T22:38:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.780+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fcd041a29b0 con 0x7fcd04100ad0 2024-09-15T22:38:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.781+0000 7fccf37fe640 1 -- 172.21.15.16:0/3826839926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5592 (secure 0 0 0) 0x7fccf409c3f0 con 0x7fcd04100ad0 2024-09-15T22:38:08.783 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 87 2024-09-15T22:38:08.783 INFO:teuthology.orchestra.run.smithi016.stdout:fsid af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:38:08.783 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-15T22:28:23.528863+0000 2024-09-15T22:38:08.783 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-15T22:37:12.511711+0000 2024-09-15T22:38:08.783 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T22:38:08.783 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 27 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-15T22:38:08.784 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/211991038,v1:172.21.15.16:6803/211991038] [v2:172.21.15.16:6804/211991038,v1:172.21.15.16:6805/211991038] exists,up 61d5e373-f70b-4535-bbb2-815a227c3404 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6800/1149325299,v1:172.21.15.59:6801/1149325299] [v2:172.21.15.59:6802/1149325299,v1:172.21.15.59:6803/1149325299] exists,up d9d5166a-3b14-4810-aa73-0fc1f5defad1 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6800/4062265693,v1:172.21.15.130:6801/4062265693] [v2:172.21.15.130:6802/4062265693,v1:172.21.15.130:6803/4062265693] exists,up ad8259e8-4042-4f91-8ca4-10bb8d40315a 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 24 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/1167232164,v1:172.21.15.16:6811/1167232164] [v2:172.21.15.16:6812/1167232164,v1:172.21.15.16:6813/1167232164] exists,up 157fd25f-1750-4f7e-8745-5131ea71a1a4 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 31 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6808/2983797571,v1:172.21.15.59:6809/2983797571] [v2:172.21.15.59:6810/2983797571,v1:172.21.15.59:6811/2983797571] exists,up 89b8e735-5627-4a56-b82c-8a9477c44fa9 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 35 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6808/4126131610,v1:172.21.15.130:6809/4126131610] [v2:172.21.15.130:6810/4126131610,v1:172.21.15.130:6811/4126131610] exists,up 5a628733-0612-40c3-97b1-cf54a3d990d4 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 42 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/2712238672,v1:172.21.15.16:6819/2712238672] [v2:172.21.15.16:6820/2712238672,v1:172.21.15.16:6821/2712238672] exists,up 8da934b1-44dc-4944-87e9-21e7b09dddfe 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 47 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6816/1929682102,v1:172.21.15.59:6817/1929682102] [v2:172.21.15.59:6818/1929682102,v1:172.21.15.59:6819/1929682102] exists,up d4d481db-97f6-4231-9da0-fc59f0dd4f27 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 54 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6816/3134922308,v1:172.21.15.130:6817/3134922308] [v2:172.21.15.130:6818/3134922308,v1:172.21.15.130:6819/3134922308] exists,up 07aa015e-5371-433a-b080-07fef3dd8c8a 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 61 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/2698575932,v1:172.21.15.16:6827/2698575932] [v2:172.21.15.16:6828/2698575932,v1:172.21.15.16:6829/2698575932] exists,up 995607d1-c7a6-4ad1-9538-192ed3d439fc 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 66 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6824/2888838982,v1:172.21.15.59:6825/2888838982] [v2:172.21.15.59:6826/2888838982,v1:172.21.15.59:6827/2888838982] exists,up 421e58d7-471b-43aa-908e-828f72805842 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 73 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.130:6824/790310360,v1:172.21.15.130:6825/790310360] [v2:172.21.15.130:6826/790310360,v1:172.21.15.130:6827/790310360] exists,up aa7b2bbd-e5b1-4b42-a4a1-934185f54929 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.b [8,11] 2024-09-15T22:38:08.785 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.10 [2,4] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.e [5,9] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.23 [4,8] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.2c [4,9] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.3c [4,9] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.a [5,10] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.e [9,11] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.16 [9,8] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 4.18 [9,8] 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1740394357 expires 2024-09-16T22:29:10.483605+0000 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3905659611 expires 2024-09-16T22:29:10.483605+0000 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3985544937 expires 2024-09-16T22:28:46.507397+0000 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3381483588 expires 2024-09-16T22:29:10.483605+0000 2024-09-15T22:38:08.786 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/2652317700 expires 2024-09-16T22:29:10.483605+0000 2024-09-15T22:38:08.787 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3860212428 expires 2024-09-16T22:28:46.507397+0000 2024-09-15T22:38:08.787 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/2652317700 expires 2024-09-16T22:29:10.483605+0000 2024-09-15T22:38:08.787 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1563124617 expires 2024-09-16T22:28:46.507397+0000 2024-09-15T22:38:08.787 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/3949807891 expires 2024-09-16T22:28:46.507397+0000 2024-09-15T22:38:08.787 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/3949807891 expires 2024-09-16T22:28:46.507397+0000 2024-09-15T22:38:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fccec0a3300 msgr2=0x7fccec0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fccec0a3300 0x7fccec0a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fccf8031440 tx=0x7fccf80315b0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04100ad0 msgr2=0x7fcd0419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04100ad0 0x7fcd0419a110 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fccf400e9c0 tx=0x7fccf400ee80 comp rx=0 tx=0).stop 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 shutdown_connections 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fccec0a3300 0x7fccec0a57c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcd04102690 0x7fcd041a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd04101cd0 0x7fcd0419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 --2- 172.21.15.16:0/3826839926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd04100ad0 0x7fcd0419a110 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 >> 172.21.15.16:0/3826839926 conn(0x7fcd040fc280 msgr2=0x7fcd041098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 shutdown_connections 2024-09-15T22:38:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:08.784+0000 7fcd0890c640 1 -- 172.21.15.16:0/3826839926 wait complete. 2024-09-15T22:38:09.590 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-15T22:38:09.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:09.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:09 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:09 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3826839926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:38:09.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:09 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:09 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3826839926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:38:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:09.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:09 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:09.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:09 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3826839926' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 -- 172.21.15.16:0/663047787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1082a0 msgr2=0x7f691c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 --2- 172.21.15.16:0/663047787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1082a0 0x7f691c10ec40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6914009cd0 tx=0x7f691402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 -- 172.21.15.16:0/663047787 shutdown_connections 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 --2- 172.21.15.16:0/663047787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1082a0 0x7f691c10ec40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 --2- 172.21.15.16:0/663047787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c1036e0 0x7f691c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 --2- 172.21.15.16:0/663047787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691c102d30 0x7f691c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 -- 172.21.15.16:0/663047787 >> 172.21.15.16:0/663047787 conn(0x7f691c0fc7d0 msgr2=0x7f691c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 -- 172.21.15.16:0/663047787 shutdown_connections 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.370+0000 7f69230ad640 1 -- 172.21.15.16:0/663047787 wait complete. 2024-09-15T22:38:10.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 Processor -- start 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 -- start start 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691c102d30 0x7f691c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1036e0 0x7f691c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c1082a0 0x7f691c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f691c1120d0 con 0x7f691c1036e0 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f691c111f50 con 0x7f691c102d30 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f69230ad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f691c112250 con 0x7f691c1082a0 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f691bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1036e0 0x7f691c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f691bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1036e0 0x7f691c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49696/0 (socket says 172.21.15.16:49696) 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f691bfff640 1 -- 172.21.15.16:0/2514242816 learned_addr learned my addr 172.21.15.16:0/2514242816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f6920e49640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691c102d30 0x7f691c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.371+0000 7f692164a640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c1082a0 0x7f691c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f691bfff640 1 -- 172.21.15.16:0/2514242816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c1082a0 msgr2=0x7f691c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f691bfff640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c1082a0 0x7f691c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f691bfff640 1 -- 172.21.15.16:0/2514242816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691c102d30 msgr2=0x7f691c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f691bfff640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691c102d30 0x7f691c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f691bfff640 1 -- 172.21.15.16:0/2514242816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691c19d4c0 con 0x7f691c1036e0 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f692164a640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c1082a0 0x7f691c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f691bfff640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1036e0 0x7f691c198a50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f690800c8c0 tx=0x7f690800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f6919ffb640 1 -- 172.21.15.16:0/2514242816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908007bb0 con 0x7f691c1036e0 2024-09-15T22:38:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f6919ffb640 1 -- 172.21.15.16:0/2514242816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6908007d50 con 0x7f691c1036e0 2024-09-15T22:38:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691c19d7b0 con 0x7f691c1036e0 2024-09-15T22:38:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f6919ffb640 1 -- 172.21.15.16:0/2514242816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908018830 con 0x7f691c1036e0 2024-09-15T22:38:10.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.372+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691c1a5090 con 0x7f691c1036e0 2024-09-15T22:38:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.374+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691c103110 con 0x7f691c1036e0 2024-09-15T22:38:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.375+0000 7f6919ffb640 1 -- 172.21.15.16:0/2514242816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6908018ad0 con 0x7f691c1036e0 2024-09-15T22:38:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.376+0000 7f6919ffb640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f690c0a3350 0x7f690c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.376+0000 7f6920e49640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f690c0a3350 0x7f690c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.376+0000 7f6919ffb640 1 -- 172.21.15.16:0/2514242816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f69080cfdd0 con 0x7f691c1036e0 2024-09-15T22:38:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.376+0000 7f6920e49640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f690c0a3350 0x7f690c0a5810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f691000bbe0 tx=0x7f691000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:10.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.380+0000 7f6919ffb640 1 -- 172.21.15.16:0/2514242816 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6908014030 con 0x7f691c1036e0 2024-09-15T22:38:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:10 smithi016 ceph-mon[31492]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:10.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:10 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.621+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f691c10a2b0 con 0x7f691c1036e0 2024-09-15T22:38:10.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.621+0000 7f6919ffb640 1 -- 172.21.15.16:0/2514242816 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f690809c630 con 0x7f691c1036e0 2024-09-15T22:38:10.623 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:10.623 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-15T22:36:47:380763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:47.103123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24469,24479,24500]},"id":1}]} 2024-09-15T22:38:10.624 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-15T22:38:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f690c0a3350 msgr2=0x7f690c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f690c0a3350 0x7f690c0a5810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f691000bbe0 tx=0x7f691000b390 comp rx=0 tx=0).stop 2024-09-15T22:38:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1036e0 msgr2=0x7f691c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1036e0 0x7f691c198a50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f690800c8c0 tx=0x7f690800cd80 comp rx=0 tx=0).stop 2024-09-15T22:38:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 shutdown_connections 2024-09-15T22:38:10.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f690c0a3350 0x7f690c0a5810 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f691c1082a0 0x7f691c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691c1036e0 0x7f691c198a50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 --2- 172.21.15.16:0/2514242816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691c102d30 0x7f691c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 >> 172.21.15.16:0/2514242816 conn(0x7f691c0fc7d0 msgr2=0x7f691c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 shutdown_connections 2024-09-15T22:38:10.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:10.624+0000 7f69230ad640 1 -- 172.21.15.16:0/2514242816 wait complete. 2024-09-15T22:38:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:10 smithi130 ceph-mon[35742]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:10 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:10 smithi059 ceph-mon[34872]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:38:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:10.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:10.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:10.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:10 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' 2024-09-15T22:38:11.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:11.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:11 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2514242816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:11.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:11 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2514242816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:11.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:11 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2514242816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 -- 172.21.15.16:0/1486205673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 msgr2=0x7f58a4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 --2- 172.21.15.16:0/1486205673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a4103120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f58a0009900 tx=0x7f58a002a480 comp rx=0 tx=0).stop 2024-09-15T22:38:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 -- 172.21.15.16:0/1486205673 shutdown_connections 2024-09-15T22:38:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 --2- 172.21.15.16:0/1486205673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a410c170 0x7f58a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 --2- 172.21.15.16:0/1486205673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f58a4103660 0x7f58a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 --2- 172.21.15.16:0/1486205673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a4103120 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 -- 172.21.15.16:0/1486205673 >> 172.21.15.16:0/1486205673 conn(0x7f58a40fc7a0 msgr2=0x7f58a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.004+0000 7f58abbcb640 1 -- 172.21.15.16:0/1486205673 shutdown_connections 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 -- 172.21.15.16:0/1486205673 wait complete. 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 Processor -- start 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 -- start start 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f58a4103660 0x7f58a4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a410c170 0x7f58a4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4111fb0 con 0x7f58a4102d40 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4111e30 con 0x7f58a410c170 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58abbcb640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f58a4112130 con 0x7f58a4103660 2024-09-15T22:38:12.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58a9967640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58a9967640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42966/0 (socket says 172.21.15.16:42966) 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.005+0000 7f58a9967640 1 -- 172.21.15.16:0/2423296213 learned_addr learned my addr 172.21.15.16:0/2423296213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58a9166640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f58a4103660 0x7f58a4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58aa168640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a410c170 0x7f58a4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58a9967640 1 -- 172.21.15.16:0/2423296213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f58a4103660 msgr2=0x7f58a4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58a9967640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f58a4103660 0x7f58a4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58a9967640 1 -- 172.21.15.16:0/2423296213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a410c170 msgr2=0x7f58a4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58a9967640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a410c170 0x7f58a4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58a9967640 1 -- 172.21.15.16:0/2423296213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a41a68b0 con 0x7f58a4102d40 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58a9967640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a41009a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005ab0 tx=0x7f58a00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f589affd640 1 -- 172.21.15.16:0/2423296213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a003f030 con 0x7f58a4102d40 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f589affd640 1 -- 172.21.15.16:0/2423296213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58a0003ce0 con 0x7f58a4102d40 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a41a6b40 con 0x7f58a4102d40 2024-09-15T22:38:12.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f589affd640 1 -- 172.21.15.16:0/2423296213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a0035070 con 0x7f58a4102d40 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.006+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a7070 con 0x7f58a4102d40 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.007+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a4103120 con 0x7f58a4102d40 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.008+0000 7f589affd640 1 -- 172.21.15.16:0/2423296213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f58a0042030 con 0x7f58a4102d40 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.009+0000 7f589affd640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f58900a32c0 0x7f58900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.009+0000 7f589affd640 1 -- 172.21.15.16:0/2423296213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f58a00f0450 con 0x7f58a4102d40 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.009+0000 7f58a9166640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f58900a32c0 0x7f58900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.010+0000 7f58a9166640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f58900a32c0 0x7f58900a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58a4102430 tx=0x7f589400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:12.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.013+0000 7f589affd640 1 -- 172.21.15.16:0/2423296213 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58a00bc7c0 con 0x7f58a4102d40 2024-09-15T22:38:12.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.226+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58a41071e0 con 0x7f58a4102d40 2024-09-15T22:38:12.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.227+0000 7f589affd640 1 -- 172.21.15.16:0/2423296213 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11912 (secure 0 0 0) 0x7f58a00c0ac0 con 0x7f58a4102d40 2024-09-15T22:38:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:12.228 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":25,"btime":"2024-09-15T22:36:47:380763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:36:47.103123+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24469,24479,24500]},"id":1}]} 2024-09-15T22:38:12.229 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 25 2024-09-15T22:38:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f58900a32c0 msgr2=0x7f58900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f58900a32c0 0x7f58900a5780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58a4102430 tx=0x7f589400a250 comp rx=0 tx=0).stop 2024-09-15T22:38:12.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 msgr2=0x7f58a41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a41009a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f58a0005ab0 tx=0x7f58a00042f0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 shutdown_connections 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f58900a32c0 0x7f58900a5780 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58a410c170 0x7f58a4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f58a4103660 0x7f58a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 --2- 172.21.15.16:0/2423296213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58a4102d40 0x7f58a41009a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 >> 172.21.15.16:0/2423296213 conn(0x7f58a40fc7a0 msgr2=0x7f58a40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 shutdown_connections 2024-09-15T22:38:12.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:12.229+0000 7f58abbcb640 1 -- 172.21.15.16:0/2423296213 wait complete. 2024-09-15T22:38:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:12 smithi130 ceph-mon[35742]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:38:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:12 smithi130 ceph-mon[35742]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:38:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:12 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-15T22:38:12.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:12 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2423296213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:12 smithi016 ceph-mon[31492]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:38:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:12 smithi016 ceph-mon[31492]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:38:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:12 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-15T22:38:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:12 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2423296213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:12 smithi059 ceph-mon[34872]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:38:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:12 smithi059 ceph-mon[34872]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:38:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:12 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-15T22:38:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:12 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2423296213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:12.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:38:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:13.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:13 smithi016 ceph-mon[31492]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-15T22:38:13.673 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:13 smithi016 ceph-mon[31492]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:38:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 -- 172.21.15.16:0/3571301116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab010c180 msgr2=0x7f2ab010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/3571301116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab010c180 0x7f2ab010e610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009ca0 tx=0x7f2aa802a950 comp rx=0 tx=0).stop 2024-09-15T22:38:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 -- 172.21.15.16:0/3571301116 shutdown_connections 2024-09-15T22:38:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/3571301116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab010c180 0x7f2ab010e610 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/3571301116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0103700 0x7f2ab010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/3571301116 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ab0102de0 0x7f2ab01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 -- 172.21.15.16:0/3571301116 >> 172.21.15.16:0/3571301116 conn(0x7f2ab00fc7a0 msgr2=0x7f2ab00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 -- 172.21.15.16:0/3571301116 shutdown_connections 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.671+0000 7f2ab80ad640 1 -- 172.21.15.16:0/3571301116 wait complete. 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 Processor -- start 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 -- start start 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab0102de0 0x7f2ab019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0103700 0x7f2ab019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ab010c180 0x7f2ab019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab0112000 con 0x7f2ab0102de0 2024-09-15T22:38:13.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab0111e80 con 0x7f2ab0103700 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab80ad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab0112180 con 0x7f2ab010c180 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab5e49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab0102de0 0x7f2ab019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab5e49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab0102de0 0x7f2ab019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42988/0 (socket says 172.21.15.16:42988) 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.672+0000 7f2ab5e49640 1 -- 172.21.15.16:0/2981223668 learned_addr learned my addr 172.21.15.16:0/2981223668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5648640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0103700 0x7f2ab019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab664a640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ab010c180 0x7f2ab019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5e49640 1 -- 172.21.15.16:0/2981223668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ab010c180 msgr2=0x7f2ab019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5e49640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ab010c180 0x7f2ab019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5e49640 1 -- 172.21.15.16:0/2981223668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0103700 msgr2=0x7f2ab019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5e49640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0103700 0x7f2ab019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5e49640 1 -- 172.21.15.16:0/2981223668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab019f690 con 0x7f2ab0102de0 2024-09-15T22:38:13.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab664a640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ab010c180 0x7f2ab019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5648640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0103700 0x7f2ab019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab5e49640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab0102de0 0x7f2ab019a6e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000b4a0 tx=0x7f2aa000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2aaeffd640 1 -- 172.21.15.16:0/2981223668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0014070 con 0x7f2ab0102de0 2024-09-15T22:38:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab019f980 con 0x7f2ab0102de0 2024-09-15T22:38:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2aaeffd640 1 -- 172.21.15.16:0/2981223668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2aa0010af0 con 0x7f2ab0102de0 2024-09-15T22:38:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2aaeffd640 1 -- 172.21.15.16:0/2981223668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0017d10 con 0x7f2ab0102de0 2024-09-15T22:38:13.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.673+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab01a7260 con 0x7f2ab0102de0 2024-09-15T22:38:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.675+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab01031c0 con 0x7f2ab0102de0 2024-09-15T22:38:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.676+0000 7f2aaeffd640 1 -- 172.21.15.16:0/2981223668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f2aa0017070 con 0x7f2ab0102de0 2024-09-15T22:38:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.677+0000 7f2aaeffd640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2a980a32c0 0x7f2a980a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.677+0000 7f2ab5648640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2a980a32c0 0x7f2a980a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:13.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.677+0000 7f2aaeffd640 1 -- 172.21.15.16:0/2981223668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f2aa00cf440 con 0x7f2ab0102de0 2024-09-15T22:38:13.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.678+0000 7f2ab5648640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2a980a32c0 0x7f2a980a5780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ab010eaf0 tx=0x7f2aa4005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:13.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.680+0000 7f2aaeffd640 1 -- 172.21.15.16:0/2981223668 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2aa009bde0 con 0x7f2ab0102de0 2024-09-15T22:38:13.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:13 smithi130 ceph-mon[35742]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-15T22:38:13.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:13 smithi130 ceph-mon[35742]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:38:13.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.857+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ab019ba40 con 0x7f2ab0102de0 2024-09-15T22:38:13.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.858+0000 7f2aaeffd640 1 -- 172.21.15.16:0/2981223668 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24764 (secure 0 0 0) 0x7f2ab019ba40 con 0x7f2ab0102de0 2024-09-15T22:38:13.860 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:13.861 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:38:13.491701+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:38:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2a980a32c0 msgr2=0x7f2a980a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2a980a32c0 0x7f2a980a5780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2ab010eaf0 tx=0x7f2aa4005ec0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab0102de0 msgr2=0x7f2ab019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab0102de0 0x7f2ab019a6e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000b4a0 tx=0x7f2aa000b960 comp rx=0 tx=0).stop 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 shutdown_connections 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2a980a32c0 0x7f2a980a5780 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2ab010c180 0x7f2ab019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ab0103700 0x7f2ab019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 --2- 172.21.15.16:0/2981223668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ab0102de0 0x7f2ab019a6e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 >> 172.21.15.16:0/2981223668 conn(0x7f2ab00fc7a0 msgr2=0x7f2ab0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 shutdown_connections 2024-09-15T22:38:13.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:13.861+0000 7f2ab80ad640 1 -- 172.21.15.16:0/2981223668 wait complete. 2024-09-15T22:38:13.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:13 smithi059 ceph-mon[34872]: from='mgr.14150 172.21.15.16:0/1604455277' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-15T22:38:13.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:13 smithi059 ceph-mon[34872]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:38:14.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:36:08.871593+0000', 'modified': '2024-09-15T22:36:47.103123+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24500, 'mds_1': 24479, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/2738921095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 2738921095}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 2738921095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6835/2202482741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2202482741}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2202482741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/1773539140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 1773539140}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 1773539140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.130:6839/1236682624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 1236682624}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 1236682624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.59:6837/2084675816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 2084675816}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2084675816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6839/384228279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 384228279}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 384228279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24469, 24479, 24500]} 2024-09-15T22:38:14.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:38:14.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:38:14.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:14 smithi130 ceph-mon[35742]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-15T22:38:14.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:14 smithi130 ceph-mon[35742]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:38:14.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:14 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2981223668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:14.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:14 smithi016 ceph-mon[31492]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-15T22:38:14.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:14 smithi016 ceph-mon[31492]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:38:14.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:14 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2981223668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:14.826 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:14 smithi059 ceph-mon[34872]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-15T22:38:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:14 smithi059 ceph-mon[34872]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:38:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:14 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2981223668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 -- 172.21.15.16:0/1759980258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 msgr2=0x7f336010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 --2- 172.21.15.16:0/1759980258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336010e610 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3358009900 tx=0x7f335802a480 comp rx=0 tx=0).stop 2024-09-15T22:38:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 -- 172.21.15.16:0/1759980258 shutdown_connections 2024-09-15T22:38:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 --2- 172.21.15.16:0/1759980258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336010e610 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 --2- 172.21.15.16:0/1759980258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3360103660 0x7f336010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 --2- 172.21.15.16:0/1759980258 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3360102d40 0x7f3360103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 -- 172.21.15.16:0/1759980258 >> 172.21.15.16:0/1759980258 conn(0x7f33600fc7a0 msgr2=0x7f33600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 -- 172.21.15.16:0/1759980258 shutdown_connections 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.289+0000 7f3367b24640 1 -- 172.21.15.16:0/1759980258 wait complete. 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 Processor -- start 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 -- start start 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3360102d40 0x7f3360198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3360103660 0x7f3360198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3360112030 con 0x7f336010c170 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3360111eb0 con 0x7f3360103660 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f3367b24640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f33601121b0 con 0x7f3360102d40 2024-09-15T22:38:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43002/0 (socket says 172.21.15.16:43002) 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 -- 172.21.15.16:0/1909289848 learned_addr learned my addr 172.21.15.16:0/1909289848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33658c0640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3360102d40 0x7f3360198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33650bf640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3360103660 0x7f3360198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 -- 172.21.15.16:0/1909289848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3360102d40 msgr2=0x7f3360198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3360102d40 0x7f3360198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 -- 172.21.15.16:0/1909289848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3360103660 msgr2=0x7f3360198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3360103660 0x7f3360198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 -- 172.21.15.16:0/1909289848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336019d520 con 0x7f336010c170 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33650bf640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3360103660 0x7f3360198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.290+0000 7f33660c1640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336019ce40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3358009230 tx=0x7f33580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:15.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.291+0000 7f3352ffd640 1 -- 172.21.15.16:0/1909289848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335803f030 con 0x7f336010c170 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.291+0000 7f3352ffd640 1 -- 172.21.15.16:0/1909289848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3358003ce0 con 0x7f336010c170 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.291+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336019d7b0 con 0x7f336010c170 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.291+0000 7f3352ffd640 1 -- 172.21.15.16:0/1909289848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358035070 con 0x7f336010c170 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.291+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33601a50f0 con 0x7f336010c170 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.293+0000 7f3369efd740 1 Processor -- start 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.294+0000 7f3352ffd640 1 -- 172.21.15.16:0/1909289848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3358042030 con 0x7f336010c170 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.294+0000 7f3369efd740 1 -- start start 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f3369efd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548ea9d0 0x55ea548eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f3369efd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548eb2f0 0x55ea548f5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f3369efd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f3369efd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea54851090 con 0x55ea548f5bb0 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f3369efd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea547f8a70 con 0x55ea548ea9d0 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f3369efd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ea5484d050 con 0x55ea548eb2f0 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548ea9d0 0x55ea548eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548ea9d0 0x55ea548eadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:39408/0 (socket says 172.21.15.16:39408) 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 -- 172.21.15.16:0/3942012357 learned_addr learned my addr 172.21.15.16:0/3942012357 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33658c0640 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548eb2f0 0x55ea548f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f3352ffd640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f334c0a32c0 0x7f334c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 -- 172.21.15.16:0/3942012357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548eb2f0 msgr2=0x55ea548f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548eb2f0 0x55ea548f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33660c1640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f334c0a32c0 0x7f334c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 -- 172.21.15.16:0/3942012357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 msgr2=0x55ea548f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33650bf640 1 -- 172.21.15.16:0/3942012357 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea54805130 con 0x55ea548ea9d0 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.295+0000 7f33658c0640 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548eb2f0 0x55ea548f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.296+0000 7f3352ffd640 1 -- 172.21.15.16:0/1909289848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f33580f05d0 con 0x7f336010c170 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.296+0000 7f33650bf640 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548ea9d0 0x55ea548eadb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3354006fd0 tx=0x7f335400c250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24703972f5d71ceb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.296+0000 7f334bfff640 1 -- 172.21.15.16:0/3942012357 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335401c070 con 0x55ea548ea9d0 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.296+0000 7f33660c1640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f334c0a32c0 0x7f334c0a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33580bc570 tx=0x7f3358087200 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.296+0000 7f334bfff640 1 -- 172.21.15.16:0/3942012357 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3354017af0 con 0x55ea548ea9d0 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.296+0000 7f334bfff640 1 -- 172.21.15.16:0/3942012357 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335400c030 con 0x55ea548ea9d0 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 -- 172.21.15.16:0/3942012357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548ea9d0 msgr2=0x55ea548eadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548ea9d0 0x55ea548eadb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3354006fd0 tx=0x7f335400c250 comp rx=0 tx=0).stop 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 -- 172.21.15.16:0/3942012357 shutdown_connections 2024-09-15T22:38:15.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548eb2f0 0x55ea548f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 --2- 172.21.15.16:0/3942012357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548ea9d0 0x55ea548eadb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 -- 172.21.15.16:0/3942012357 >> 172.21.15.16:0/3942012357 conn(0x55ea548e6f60 msgr2=0x55ea548e7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 -- 172.21.15.16:0/3942012357 shutdown_connections 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 -- 172.21.15.16:0/3942012357 wait complete. 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 -- start start 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.297+0000 7f3369efd740 1 Processor -- start 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f3369efd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea548f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f3369efd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea548f2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f3369efd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f3369efd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea548f8780 con 0x55ea548f5bb0 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f3369efd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea5480bf20 con 0x55ea548eb2f0 2024-09-15T22:38:15.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f3369efd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ea548f8900 con 0x55ea548ea9d0 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f3369efd740 10 client.0 _ll_register_callbacks cb 0x55ea547d5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43020/0 (socket says 172.21.15.16:43020) 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 -- 172.21.15.16:0/1281138717 learned_addr learned my addr 172.21.15.16:0/1281138717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33658c0640 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea548f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea548f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 -- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 msgr2=0x55ea548f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea548f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 -- 172.21.15.16:0/1281138717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 msgr2=0x55ea548f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea548f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 -- 172.21.15.16:0/1281138717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea545c22b0 con 0x55ea548f5bb0 2024-09-15T22:38:15.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33650bf640 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f2d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3354017010 tx=0x7f3354016630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994c2ab749e2a855 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.298+0000 7f33497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f33497fa640 1 -- 172.21.15.16:0/1281138717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335401c070 con 0x55ea548f5bb0 2024-09-15T22:38:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3354017af0 con 0x55ea548f5bb0 2024-09-15T22:38:15.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f33497fa640 1 -- 172.21.15.16:0/1281138717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f3354017ca0 con 0x55ea548f5bb0 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f33497fa640 1 -- 172.21.15.16:0/1281138717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3354032050 con 0x55ea548f5bb0 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f3369efd740 10 client.15171 fetch_fsmap learned FSMap version 25 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea549040f0 con 0x55ea548f5bb0 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f33497fa640 1 -- 172.21.15.16:0/1281138717 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3354017af0 con 0x55ea548f5bb0 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f33497fa640 1 -- 172.21.15.16:0/1281138717 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3354022070 con 0x55ea548f5bb0 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f3369efd740 10 client.15171 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.299+0000 7f3369efd740 10 client.15171 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:38:15.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.300+0000 7f3369efd740 20 client.15171 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.300+0000 7f3369efd740 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea549062f0 0x55ea54907760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.300+0000 7f3369efd740 4 client.15171 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.300+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea5485e600 con 0x55ea549062f0 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.300+0000 7f33658c0640 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea549062f0 0x55ea54907760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.300+0000 7f33497fa640 10 client.15171 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.300+0000 7f33658c0640 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea549062f0 0x55ea54907760 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=984dbd364e4dfd06 server_cookie=93b4ad6726f837d3 in_seq=0 out_seq=0 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.301+0000 7f33497fa640 1 -- 172.21.15.16:0/1281138717 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea5485e600 con 0x55ea549062f0 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.301+0000 7f33497fa640 10 client.15171 handle_command_reply: tid=0 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.301+0000 7f3369efd740 1 client.15171 shutdown 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 msgr2=0x55ea548f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f2d40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3354017010 tx=0x7f3354016630 comp rx=0 tx=0).stop 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 shutdown_connections 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea549062f0 0x55ea54907760 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548f5bb0 0x55ea548f2d40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea548f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 --2- 172.21.15.16:0/1281138717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea548f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 >> 172.21.15.16:0/1281138717 conn(0x55ea548e6f60 msgr2=0x55ea548ecc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f33497fa640 0 client.15171 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 shutdown_connections 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 1 -- 172.21.15.16:0/1281138717 wait complete. 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.302+0000 7f3369efd740 20 client.15171 trim_cache size 0 max 16384 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 Processor -- start 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 -- start start 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea54905c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548eb2f0 0x55ea54907090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548f5bb0 0x55ea548f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea54905aa0 con 0x55ea548eb2f0 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea54906170 con 0x55ea548f5bb0 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f3369efd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ea54906840 con 0x55ea548ea9d0 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548eb2f0 0x55ea54907090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548eb2f0 0x55ea54907090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43024/0 (socket says 172.21.15.16:43024) 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 -- 172.21.15.16:0/1602192963 learned_addr learned my addr 172.21.15.16:0/1602192963 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33650bf640 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea54905c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33650bf640 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548f5bb0 0x55ea548f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 -- 172.21.15.16:0/1602192963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 msgr2=0x55ea54905c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea54905c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 -- 172.21.15.16:0/1602192963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548f5bb0 msgr2=0x55ea548f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548f5bb0 0x55ea548f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 -- 172.21.15.16:0/1602192963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea548eb120 con 0x55ea548eb2f0 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33650bf640 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea54905c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.321+0000 7f33658c0640 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548eb2f0 0x55ea54907090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f335c000f80 tx=0x7f335c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989462a78962a2f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f334b7fe640 1 -- 172.21.15.16:0/1602192963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c01e070 con 0x55ea548eb2f0 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f334b7fe640 1 -- 172.21.15.16:0/1602192963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c01abd0 con 0x55ea548eb2f0 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f334b7fe640 1 -- 172.21.15.16:0/1602192963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c022e60 con 0x55ea548eb2f0 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f3369efd740 1 -- 172.21.15.16:0/1602192963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548eb2f0 msgr2=0x55ea54907090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f3369efd740 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548eb2f0 0x55ea54907090 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f335c000f80 tx=0x7f335c009830 comp rx=0 tx=0).stop 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f3369efd740 1 -- 172.21.15.16:0/1602192963 shutdown_connections 2024-09-15T22:38:15.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f3369efd740 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548f5bb0 0x55ea548f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f3369efd740 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548eb2f0 0x55ea54907090 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f3369efd740 1 --2- 172.21.15.16:0/1602192963 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548ea9d0 0x55ea54905c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.322+0000 7f3369efd740 1 -- 172.21.15.16:0/1602192963 >> 172.21.15.16:0/1602192963 conn(0x55ea548e6f60 msgr2=0x55ea548f10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 -- 172.21.15.16:0/1602192963 shutdown_connections 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 -- 172.21.15.16:0/1602192963 wait complete. 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 -- start start 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 Processor -- start 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548ea9d0 0x55ea54910300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea5490a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548f5bb0 0x55ea5490ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ea549075e0 con 0x55ea548ea9d0 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ea54906f10 con 0x55ea548eb2f0 2024-09-15T22:38:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.323+0000 7f3369efd740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ea54907cb0 con 0x55ea548f5bb0 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f3369efd740 10 client.0 _ll_register_callbacks cb 0x7f335401a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548ea9d0 0x55ea54910300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548ea9d0 0x55ea54910300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43026/0 (socket says 172.21.15.16:43026) 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 -- 172.21.15.16:0/1359772445 learned_addr learned my addr 172.21.15.16:0/1359772445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33658c0640 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea5490a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 -- 172.21.15.16:0/1359772445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548f5bb0 msgr2=0x55ea5490ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548f5bb0 0x55ea5490ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 -- 172.21.15.16:0/1359772445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 msgr2=0x55ea5490a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea5490a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 -- 172.21.15.16:0/1359772445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea5496e9f0 con 0x55ea548ea9d0 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33650bf640 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548ea9d0 0x55ea54910300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f335400c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6b7378df2811cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33497fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33497fa640 1 -- 172.21.15.16:0/1359772445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335401c070 con 0x55ea548ea9d0 2024-09-15T22:38:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f33497fa640 1 -- 172.21.15.16:0/1359772445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f335402a040 con 0x55ea548ea9d0 2024-09-15T22:38:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.324+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f335c01abd0 con 0x55ea548ea9d0 2024-09-15T22:38:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f33497fa640 1 -- 172.21.15.16:0/1359772445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f335400c040 con 0x55ea548ea9d0 2024-09-15T22:38:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 10 client.15177 fetch_fsmap learned FSMap version 25 2024-09-15T22:38:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea5496fb60 con 0x55ea548ea9d0 2024-09-15T22:38:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f33497fa640 1 -- 172.21.15.16:0/1359772445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f335c01abd0 con 0x55ea548ea9d0 2024-09-15T22:38:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f33497fa640 1 -- 172.21.15.16:0/1359772445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3354022040 con 0x55ea548ea9d0 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 10 client.15177 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 10 client.15177 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 20 client.15177 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea54974760 0x55ea54973470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 4 client.15177 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "status"}) -- 0x55ea54858b10 con 0x55ea54974760 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.325+0000 7f33658c0640 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea54974760 0x55ea54973470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f33658c0640 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea54974760 0x55ea54973470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f55d54e5af700a4e server_cookie=ef6783152c036ac6 in_seq=0 out_seq=0 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f33497fa640 10 client.15177 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f33497fa640 1 -- 172.21.15.16:0/1359772445 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ea54858b10 con 0x55ea54974760 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f33497fa640 10 client.15177 handle_command_reply: tid=0 2024-09-15T22:38:15.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 client.15177 shutdown 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548ea9d0 msgr2=0x55ea54910300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548ea9d0 0x55ea54910300 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f335400c6f0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 shutdown_connections 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55ea54974760 0x55ea54973470 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ea548f5bb0 0x55ea5490ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ea548eb2f0 0x55ea5490a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f33497fa640 0 client.15177 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 --2- 172.21.15.16:0/1359772445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ea548ea9d0 0x55ea54910300 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.326+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 >> 172.21.15.16:0/1359772445 conn(0x55ea548e6f60 msgr2=0x55ea548f14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 shutdown_connections 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3369efd740 1 -- 172.21.15.16:0/1359772445 wait complete. 2024-09-15T22:38:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3369efd740 20 client.15177 trim_cache size 0 max 16384 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24469, 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 90.950346844999999, 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 89, 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 85, 2024-09-15T22:38:15.347 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 146.134919822 2024-09-15T22:38:15.348 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f334c0a32c0 msgr2=0x7f334c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f334c0a32c0 0x7f334c0a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33580bc570 tx=0x7f3358087200 comp rx=0 tx=0).stop 2024-09-15T22:38:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 msgr2=0x7f336019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336019ce40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3358009230 tx=0x7f33580042f0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 shutdown_connections 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f334c0a32c0 0x7f334c0a5780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f336010c170 0x7f336019ce40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3360103660 0x7f3360198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 --2- 172.21.15.16:0/1909289848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3360102d40 0x7f3360198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.327+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 >> 172.21.15.16:0/1909289848 conn(0x7f33600fc7a0 msgr2=0x7f33600fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.328+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 shutdown_connections 2024-09-15T22:38:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:15.328+0000 7f3367b24640 1 -- 172.21.15.16:0/1909289848 wait complete. 2024-09-15T22:38:15.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:15 smithi016 ceph-mon[31492]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-15T22:38:15.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:15 smithi016 ceph-mon[31492]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-15T22:38:15.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:15 smithi059 ceph-mon[34872]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-15T22:38:15.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:15 smithi059 ceph-mon[34872]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-15T22:38:16.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:38:16.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:15 smithi130 ceph-mon[35742]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-15T22:38:16.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:15 smithi130 ceph-mon[35742]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-15T22:38:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 -- 172.21.15.16:0/4278418078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708102790 msgr2=0x7fe7081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 --2- 172.21.15.16:0/4278418078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708102790 0x7fe7081067e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009900 tx=0x7fe6f802a410 comp rx=0 tx=0).stop 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 -- 172.21.15.16:0/4278418078 shutdown_connections 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 --2- 172.21.15.16:0/4278418078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe708106f10 0x7fe70810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 --2- 172.21.15.16:0/4278418078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708102790 0x7fe7081067e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 --2- 172.21.15.16:0/4278418078 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe708101de0 0x7fe7081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 -- 172.21.15.16:0/4278418078 >> 172.21.15.16:0/4278418078 conn(0x7fe7080fd8c0 msgr2=0x7fe7080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 -- 172.21.15.16:0/4278418078 shutdown_connections 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.765+0000 7fe710158640 1 -- 172.21.15.16:0/4278418078 wait complete. 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 Processor -- start 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 -- start start 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708101de0 0x7fe708198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe708102790 0x7fe708198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe708106f10 0x7fe70819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe708111f90 con 0x7fe708101de0 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe708111e10 con 0x7fe708102790 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.766+0000 7fe710158640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe708112110 con 0x7fe708106f10 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708101de0 0x7fe708198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708101de0 0x7fe708198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43050/0 (socket says 172.21.15.16:43050) 2024-09-15T22:38:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 -- 172.21.15.16:0/1258983 learned_addr learned my addr 172.21.15.16:0/1258983 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe708102790 0x7fe708198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe708106f10 0x7fe70819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 -- 172.21.15.16:0/1258983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe708106f10 msgr2=0x7fe70819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe708106f10 0x7fe70819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 -- 172.21.15.16:0/1258983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe708102790 msgr2=0x7fe708198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe708102790 0x7fe708198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 -- 172.21.15.16:0/1258983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70819d500 con 0x7fe708101de0 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe708106f10 0x7fe70819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe70def4640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708101de0 0x7fe708198550 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe70400b4a0 tx=0x7fe70400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe6feffd640 1 -- 172.21.15.16:0/1258983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704014070 con 0x7fe708101de0 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe6feffd640 1 -- 172.21.15.16:0/1258983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe704010af0 con 0x7fe708101de0 2024-09-15T22:38:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70819d7f0 con 0x7fe708101de0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe6feffd640 1 -- 172.21.15.16:0/1258983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704017d10 con 0x7fe708101de0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.767+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7081a50d0 con 0x7fe708101de0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe6feffd640 1 -- 172.21.15.16:0/1258983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fe704017070 con 0x7fe708101de0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 Processor -- start 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 -- start start 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d39d0 0x5576d11d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576d113a090 con 0x5576d11d42f0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5576d10e1a70 con 0x5576d11debb0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe712531740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5576d1136050 con 0x5576d11d39d0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe70def4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe70def4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11de670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43056/0 (socket says 172.21.15.16:43056) 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.770+0000 7fe70def4640 1 -- 172.21.15.16:0/2329739984 learned_addr learned my addr 172.21.15.16:0/2329739984 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe6feffd640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6f40a32c0 0x7fe6f40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d39d0 0x5576d11d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe6feffd640 1 -- 172.21.15.16:0/1258983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe7040cef80 con 0x7fe708101de0 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6f40a32c0 0x7fe6f40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70def4640 1 -- 172.21.15.16:0/2329739984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d39d0 msgr2=0x5576d11d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70def4640 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d39d0 0x5576d11d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70def4640 1 -- 172.21.15.16:0/2329739984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 msgr2=0x5576d11e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70def4640 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70def4640 1 -- 172.21.15.16:0/2329739984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d10ee130 con 0x5576d11d42f0 2024-09-15T22:38:16.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70def4640 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11de670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe70409f110 tx=0x7fe7040047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d9b45e2c3c9410 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6f40a32c0 0x7fe6f40a5780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe70000afd0 tx=0x7fe700004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe6f3fff640 1 -- 172.21.15.16:0/2329739984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70401e030 con 0x5576d11d42f0 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe6f3fff640 1 -- 172.21.15.16:0/2329739984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70409b920 con 0x5576d11d42f0 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.771+0000 7fe6f3fff640 1 -- 172.21.15.16:0/2329739984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040b9e60 con 0x5576d11d42f0 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 -- 172.21.15.16:0/2329739984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 msgr2=0x5576d11de670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11de670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe70409f110 tx=0x7fe7040047d0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 -- 172.21.15.16:0/2329739984 shutdown_connections 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11de670 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 --2- 172.21.15.16:0/2329739984 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d39d0 0x5576d11d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 -- 172.21.15.16:0/2329739984 >> 172.21.15.16:0/2329739984 conn(0x5576d11cff60 msgr2=0x5576d11d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 -- 172.21.15.16:0/2329739984 shutdown_connections 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 -- 172.21.15.16:0/2329739984 wait complete. 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.772+0000 7fe712531740 1 -- start start 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 1 Processor -- start 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11de460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11db7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d11dbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576d11e1780 con 0x5576d11d42f0 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5576d10f4f20 con 0x5576d11d39d0 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5576d11e1900 con 0x5576d11debb0 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe712531740 10 client.0 _ll_register_callbacks cb 0x5576d10be030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe70d6f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11db7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe70d6f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11db7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43058/0 (socket says 172.21.15.16:43058) 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe70d6f3640 1 -- 172.21.15.16:0/3666483931 learned_addr learned my addr 172.21.15.16:0/3666483931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe70def4640 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11de460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.773+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d11dbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe70d6f3640 1 -- 172.21.15.16:0/3666483931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 msgr2=0x5576d11dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d11dbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe70d6f3640 1 -- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 msgr2=0x5576d11de460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11de460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe70d6f3640 1 -- 172.21.15.16:0/3666483931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d0eab2b0 con 0x5576d11d42f0 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11db7b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009900 tx=0x7fe6f8031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4294345d15d5c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe6f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:16.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe6f17fa640 1 -- 172.21.15.16:0/3666483931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8035040 con 0x5576d11d42f0 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe6f17fa640 1 -- 172.21.15.16:0/3666483931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe6f8004380 con 0x5576d11d42f0 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70409b920 con 0x5576d11d42f0 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe6f17fa640 1 -- 172.21.15.16:0/3666483931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6f804f050 con 0x5576d11d42f0 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe712531740 10 client.15189 fetch_fsmap learned FSMap version 25 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576d11ed0a0 con 0x5576d11d42f0 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.774+0000 7fe6f17fa640 1 -- 172.21.15.16:0/3666483931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70409b920 con 0x5576d11d42f0 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe6f17fa640 1 -- 172.21.15.16:0/3666483931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe6f80467e0 con 0x5576d11d42f0 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe712531740 10 client.15189 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe712531740 10 client.15189 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe712531740 20 client.15189 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe712531740 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d11ef720 0x5576d11f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe712531740 4 client.15189 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576d1147600 con 0x5576d11ef720 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe70def4640 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d11ef720 0x5576d11f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe6f17fa640 10 client.15189 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.775+0000 7fe70def4640 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d11ef720 0x5576d11f0b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47b7d906cf345f8a server_cookie=cc4450f2812edf9d in_seq=0 out_seq=0 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe6f17fa640 1 -- 172.21.15.16:0/3666483931 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe704079e50 con 0x5576d11ef720 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe6f17fa640 10 client.15189 handle_command_reply: tid=0 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 client.15189 shutdown 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 msgr2=0x5576d11db7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11db7b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009900 tx=0x7fe6f8031a60 comp rx=0 tx=0).stop 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 shutdown_connections 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d11ef720 0x5576d11f0b90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d11dbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11db7b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 --2- 172.21.15.16:0/3666483931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11de460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe6f17fa640 0 client.15189 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 >> 172.21.15.16:0/3666483931 conn(0x5576d11cff60 msgr2=0x5576d11d5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 shutdown_connections 2024-09-15T22:38:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.776+0000 7fe712531740 1 -- 172.21.15.16:0/3666483931 wait complete. 2024-09-15T22:38:16.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.777+0000 7fe712531740 20 client.15189 trim_cache size 0 max 16384 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.794+0000 7fe712531740 1 Processor -- start 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.794+0000 7fe712531740 1 -- start start 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe712531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d39d0 0x5576d11f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe712531740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d42f0 0x5576d11f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe712531740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe712531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576d11eeed0 con 0x5576d11d39d0 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe712531740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5576d11ef5a0 con 0x5576d11debb0 2024-09-15T22:38:16.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe712531740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5576d11efc70 con 0x5576d11d42f0 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d39d0 0x5576d11f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d39d0 0x5576d11f0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43072/0 (socket says 172.21.15.16:43072) 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 -- 172.21.15.16:0/4163762735 learned_addr learned my addr 172.21.15.16:0/4163762735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d42f0 0x5576d11f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 -- 172.21.15.16:0/4163762735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d42f0 msgr2=0x5576d11f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d42f0 0x5576d11f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 -- 172.21.15.16:0/4163762735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 msgr2=0x5576d11f93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11f93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 -- 172.21.15.16:0/4163762735 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d11d4e20 con 0x5576d11d39d0 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe70def4640 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d39d0 0x5576d11f0b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe7040176e0 tx=0x7fe70400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a59da5038b0b9dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe6f37fe640 1 -- 172.21.15.16:0/4163762735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70401e040 con 0x5576d11d39d0 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe6f37fe640 1 -- 172.21.15.16:0/4163762735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5576d1147600 con 0x5576d11d39d0 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.795+0000 7fe6f37fe640 1 -- 172.21.15.16:0/4163762735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040b9e60 con 0x5576d11d39d0 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 -- 172.21.15.16:0/4163762735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d39d0 msgr2=0x5576d11f0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d39d0 0x5576d11f0b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe7040176e0 tx=0x7fe70400bf90 comp rx=0 tx=0).stop 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 -- 172.21.15.16:0/4163762735 shutdown_connections 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11debb0 0x5576d11f93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11d42f0 0x5576d11f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 --2- 172.21.15.16:0/4163762735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d39d0 0x5576d11f0b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 -- 172.21.15.16:0/4163762735 >> 172.21.15.16:0/4163762735 conn(0x5576d11cff60 msgr2=0x5576d11da080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 -- 172.21.15.16:0/4163762735 shutdown_connections 2024-09-15T22:38:16.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 -- 172.21.15.16:0/4163762735 wait complete. 2024-09-15T22:38:16.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.796+0000 7fe712531740 1 -- start start 2024-09-15T22:38:16.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 1 Processor -- start 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d1257e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576d11f0a10 con 0x5576d11d42f0 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5576d11f0340 con 0x5576d11d39d0 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5576d11f10e0 con 0x5576d11debb0 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe712531740 10 client.0 _ll_register_callbacks cb 0x7fe6f8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe70d6f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe70d6f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11f44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43080/0 (socket says 172.21.15.16:43080) 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe70d6f3640 1 -- 172.21.15.16:0/1265443225 learned_addr learned my addr 172.21.15.16:0/1265443225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.797+0000 7fe70def4640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70d6f3640 1 -- 172.21.15.16:0/1265443225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 msgr2=0x5576d1257e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70e6f5640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d1257e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d1257e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70d6f3640 1 -- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 msgr2=0x5576d11f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70d6f3640 1 -- 172.21.15.16:0/1265443225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d1258360 con 0x5576d11d42f0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70def4640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe70d6f3640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11f44c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8046be0 tx=0x7fe6f8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83c8b1e5cb3ee8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe6f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe6f17fa640 1 -- 172.21.15.16:0/1265443225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8057020 con 0x5576d11d42f0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe6f17fa640 1 -- 172.21.15.16:0/1265443225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fe6f8004390 con 0x5576d11d42f0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576d1147600 con 0x5576d11d42f0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe6f17fa640 1 -- 172.21.15.16:0/1265443225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6f805f050 con 0x5576d11d42f0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe712531740 10 client.15195 fetch_fsmap learned FSMap version 25 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576d1259500 con 0x5576d11d42f0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.798+0000 7fe6f17fa640 1 -- 172.21.15.16:0/1265443225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5576d1147600 con 0x5576d11d42f0 2024-09-15T22:38:16.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe6f17fa640 1 -- 172.21.15.16:0/1265443225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe6f803dbe0 con 0x5576d11d42f0 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe712531740 10 client.15195 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe712531740 10 client.15195 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe712531740 20 client.15195 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe712531740 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d125e0c0 0x5576d125cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe712531740 4 client.15195 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "status"}) -- 0x5576d1141b10 con 0x5576d125e0c0 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe70def4640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d125e0c0 0x5576d125cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe70def4640 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d125e0c0 0x5576d125cdd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d47759e87da8830 server_cookie=1793c6076f0d6e81 in_seq=0 out_seq=0 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.799+0000 7fe6f17fa640 10 client.15195 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe6f17fa640 1 -- 172.21.15.16:0/1265443225 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5576d1141b10 con 0x5576d125e0c0 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe6f17fa640 10 client.15195 handle_command_reply: tid=0 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 client.15195 shutdown 2024-09-15T22:38:16.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 msgr2=0x5576d11f44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11f44c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8046be0 tx=0x7fe6f8005d90 comp rx=0 tx=0).stop 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 shutdown_connections 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5576d125e0c0 0x5576d125cdd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576d11debb0 0x5576d1257e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576d11d42f0 0x5576d11f44c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe6f17fa640 0 client.15195 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 --2- 172.21.15.16:0/1265443225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576d11d39d0 0x5576d11f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 >> 172.21.15.16:0/1265443225 conn(0x5576d11cff60 msgr2=0x5576d11dce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 shutdown_connections 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 1 -- 172.21.15.16:0/1265443225 wait complete. 2024-09-15T22:38:16.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.800+0000 7fe712531740 20 client.15195 trim_cache size 0 max 16384 2024-09-15T22:38:16.809 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:38:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-15T22:38:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24479, 2024-09-15T22:38:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:38:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:38:16.809 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:38:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 93.428732388, 2024-09-15T22:38:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-15T22:38:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 89, 2024-09-15T22:38:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 85, 2024-09-15T22:38:16.810 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 150.14705953000001 2024-09-15T22:38:16.810 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:16.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6f40a32c0 msgr2=0x7fe6f40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6f40a32c0 0x7fe6f40a5780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe70000afd0 tx=0x7fe700004ab0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708101de0 msgr2=0x7fe708198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708101de0 0x7fe708198550 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe70400b4a0 tx=0x7fe70400b960 comp rx=0 tx=0).stop 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 shutdown_connections 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fe6f40a32c0 0x7fe6f40a5780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe708106f10 0x7fe70819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe708102790 0x7fe708198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 --2- 172.21.15.16:0/1258983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe708101de0 0x7fe708198550 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 >> 172.21.15.16:0/1258983 conn(0x7fe7080fd8c0 msgr2=0x7fe7080fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 shutdown_connections 2024-09-15T22:38:16.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:16.801+0000 7fe710158640 1 -- 172.21.15.16:0/1258983 wait complete. 2024-09-15T22:38:17.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:38:17.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:17 smithi016 ceph-mon[31492]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-15T22:38:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:17.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:17 smithi059 ceph-mon[34872]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-15T22:38:18.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:17 smithi130 ceph-mon[35742]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 644 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 6 op/s 2024-09-15T22:38:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 -- 172.21.15.16:0/2279312321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0101bc0 msgr2=0x7f34e010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 --2- 172.21.15.16:0/2279312321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0101bc0 0x7f34e010d910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009ca0 tx=0x7f34d802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 -- 172.21.15.16:0/2279312321 shutdown_connections 2024-09-15T22:38:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 --2- 172.21.15.16:0/2279312321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0101bc0 0x7f34e010d910 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 --2- 172.21.15.16:0/2279312321 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e0101240 0x7f34e0101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 --2- 172.21.15.16:0/2279312321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34e0105a90 0x7f34e0105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 -- 172.21.15.16:0/2279312321 >> 172.21.15.16:0/2279312321 conn(0x7f34e0077210 msgr2=0x7f34e00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 -- 172.21.15.16:0/2279312321 shutdown_connections 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 -- 172.21.15.16:0/2279312321 wait complete. 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 Processor -- start 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.314+0000 7f34e602b640 1 -- start start 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34e602b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34e0101240 0x7f34e0197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34e602b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e0101bc0 0x7f34e01976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34e602b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0105a90 0x7f34e0197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34e602b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0110fd0 con 0x7f34e0105a90 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34e602b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0110e50 con 0x7f34e0101240 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34e602b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0111150 con 0x7f34e0101bc0 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0105a90 0x7f34e0197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0105a90 0x7f34e0197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43094/0 (socket says 172.21.15.16:43094) 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 -- 172.21.15.16:0/2147372285 learned_addr learned my addr 172.21.15.16:0/2147372285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34df7fe640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34e0101240 0x7f34e0197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34deffd640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e0101bc0 0x7f34e01976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 -- 172.21.15.16:0/2147372285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e0101bc0 msgr2=0x7f34e01976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e0101bc0 0x7f34e01976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 -- 172.21.15.16:0/2147372285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34e0101240 msgr2=0x7f34e0197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34e0101240 0x7f34e0197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 -- 172.21.15.16:0/2147372285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e01a69e0 con 0x7f34e0105a90 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34deffd640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e0101bc0 0x7f34e01976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.315+0000 7f34dffff640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0105a90 0x7f34e0197bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f34d8006010 tx=0x7f34d8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.316+0000 7f34dcff9640 1 -- 172.21.15.16:0/2147372285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8031650 con 0x7f34e0105a90 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.316+0000 7f34dcff9640 1 -- 172.21.15.16:0/2147372285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d80317f0 con 0x7f34e0105a90 2024-09-15T22:38:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.316+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e01a6c70 con 0x7f34e0105a90 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.316+0000 7f34dcff9640 1 -- 172.21.15.16:0/2147372285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80309c0 con 0x7f34e0105a90 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.316+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a71b0 con 0x7f34e0105a90 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.318+0000 7f34dcff9640 1 -- 172.21.15.16:0/2147372285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f34d8030b60 con 0x7f34e0105a90 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.318+0000 7f34e8404740 1 Processor -- start 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.318+0000 7f34e8404740 1 -- start start 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.318+0000 7f34e8404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c5ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.318+0000 7f34e8404740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5e2f0 0x558737c68670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34e8404740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c6afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34e8404740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558737bc4090 con 0x558737c68bb0 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34e8404740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558737b6ba70 con 0x558737c5e2f0 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34e8404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558737bc0050 con 0x558737c5d9d0 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c6afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c6afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43104/0 (socket says 172.21.15.16:43104) 2024-09-15T22:38:18.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 -- 172.21.15.16:0/1021369298 learned_addr learned my addr 172.21.15.16:0/1021369298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34dcff9640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f34c80a3240 0x7f34c80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c5ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34df7fe640 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5e2f0 0x558737c68670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34dffff640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f34c80a3240 0x7f34c80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 -- 172.21.15.16:0/1021369298 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 msgr2=0x558737c5ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c5ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 -- 172.21.15.16:0/1021369298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5e2f0 msgr2=0x558737c68670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5e2f0 0x558737c68670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 -- 172.21.15.16:0/1021369298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558737b78130 con 0x558737c68bb0 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34dcff9640 1 -- 172.21.15.16:0/2147372285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f34d80f1130 con 0x7f34e0105a90 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34deffd640 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c6afa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f34d4014a30 tx=0x7f34d400c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae76c1a53373cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34c5ffb640 1 -- 172.21.15.16:0/1021369298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40142d0 con 0x558737c68bb0 2024-09-15T22:38:18.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.319+0000 7f34dffff640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f34c80a3240 0x7f34c80a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34d80bd360 tx=0x7f34d8087e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34c5ffb640 1 -- 172.21.15.16:0/1021369298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4005500 con 0x558737c68bb0 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34c5ffb640 1 -- 172.21.15.16:0/1021369298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40047a0 con 0x558737c68bb0 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 -- 172.21.15.16:0/1021369298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 msgr2=0x558737c6afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c6afa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f34d4014a30 tx=0x7f34d400c570 comp rx=0 tx=0).stop 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 -- 172.21.15.16:0/1021369298 shutdown_connections 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c6afa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5e2f0 0x558737c68670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 --2- 172.21.15.16:0/1021369298 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c5ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 -- 172.21.15.16:0/1021369298 >> 172.21.15.16:0/1021369298 conn(0x558737c59f60 msgr2=0x558737c5a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 -- 172.21.15.16:0/1021369298 shutdown_connections 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.320+0000 7f34e8404740 1 -- 172.21.15.16:0/1021369298 wait complete. 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 -- start start 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 Processor -- start 2024-09-15T22:38:18.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c68410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c65760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c68bb0 0x558737c65ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558737c6b780 con 0x558737c5e2f0 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558737b7ef20 con 0x558737c68bb0 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558737c6b900 con 0x558737c5d9d0 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34e8404740 10 client.0 _ll_register_callbacks cb 0x558737b48030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c65760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c65760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43110/0 (socket says 172.21.15.16:43110) 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.321+0000 7f34df7fe640 1 -- 172.21.15.16:0/490891021 learned_addr learned my addr 172.21.15.16:0/490891021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34deffd640 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c68bb0 0x558737c65ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34deffd640 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c68410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34df7fe640 1 -- 172.21.15.16:0/490891021 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 msgr2=0x558737c68410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34df7fe640 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c68410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34df7fe640 1 -- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c68bb0 msgr2=0x558737c65ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34df7fe640 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c68bb0 0x558737c65ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34df7fe640 1 -- 172.21.15.16:0/490891021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587379352b0 con 0x558737c5e2f0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34df7fe640 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c65760 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f34d0011540 tx=0x7f34d0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dbc44910fbc6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34beffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34beffd640 1 -- 172.21.15.16:0/490891021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d000f6d0 con 0x558737c5e2f0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34beffd640 1 -- 172.21.15.16:0/490891021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f34d000f870 con 0x558737c5e2f0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d4005500 con 0x558737c5e2f0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34beffd640 1 -- 172.21.15.16:0/490891021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34d0026050 con 0x558737c5e2f0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34e8404740 10 client.15207 fetch_fsmap learned FSMap version 25 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.322+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558737c77130 con 0x558737c5e2f0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34beffd640 1 -- 172.21.15.16:0/490891021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4005500 con 0x558737c5e2f0 2024-09-15T22:38:18.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34beffd640 1 -- 172.21.15.16:0/490891021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f34d0014c80 con 0x558737c5e2f0 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34e8404740 10 client.15207 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34e8404740 10 client.15207 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34e8404740 20 client.15207 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34e8404740 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737c797a0 0x558737c7ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34e8404740 4 client.15207 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558737bd1600 con 0x558737c797a0 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34deffd640 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737c797a0 0x558737c7ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34deffd640 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737c797a0 0x558737c7ac10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce25ee5300251e7 server_cookie=7c40cc32133aa5cc in_seq=0 out_seq=0 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.323+0000 7f34beffd640 10 client.15207 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34beffd640 1 -- 172.21.15.16:0/490891021 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558737bd1600 con 0x558737c797a0 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34beffd640 10 client.15207 handle_command_reply: tid=0 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 client.15207 shutdown 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 msgr2=0x558737c65760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c65760 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f34d0011540 tx=0x7f34d0011a00 comp rx=0 tx=0).stop 2024-09-15T22:38:18.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 shutdown_connections 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737c797a0 0x558737c7ac10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c68bb0 0x558737c65ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c65760 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 --2- 172.21.15.16:0/490891021 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5d9d0 0x558737c68410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34beffd640 0 client.15207 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.324+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 >> 172.21.15.16:0/490891021 conn(0x558737c59f60 msgr2=0x558737c5fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.325+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 shutdown_connections 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.325+0000 7f34e8404740 1 -- 172.21.15.16:0/490891021 wait complete. 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.325+0000 7f34e8404740 20 client.15207 trim_cache size 0 max 16384 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.342+0000 7f34e8404740 1 Processor -- start 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.342+0000 7f34e8404740 1 -- start start 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34e8404740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c790d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34e8404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5e2f0 0x558737c629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34e8404740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c62f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34e8404740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558737c78f50 con 0x558737c68bb0 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34e8404740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558737c79620 con 0x558737c5d9d0 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34e8404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558737c79cf0 con 0x558737c5e2f0 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c62f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c62f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43124/0 (socket says 172.21.15.16:43124) 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 -- 172.21.15.16:0/1661497960 learned_addr learned my addr 172.21.15.16:0/1661497960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34df7fe640 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5e2f0 0x558737c629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 -- 172.21.15.16:0/1661497960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5e2f0 msgr2=0x558737c629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5e2f0 0x558737c629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 -- 172.21.15.16:0/1661497960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 msgr2=0x558737c790d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c790d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 -- 172.21.15.16:0/1661497960 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558737c5e120 con 0x558737c68bb0 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34df7fe640 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5e2f0 0x558737c629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34deffd640 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c62f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f34d4014a30 tx=0x7f34d4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58a31179114f4e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34c57fa640 1 -- 172.21.15.16:0/1661497960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d400f820 con 0x558737c68bb0 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34c57fa640 1 -- 172.21.15.16:0/1661497960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4032070 con 0x558737c68bb0 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.343+0000 7f34c57fa640 1 -- 172.21.15.16:0/1661497960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4012930 con 0x558737c68bb0 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 -- 172.21.15.16:0/1661497960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 msgr2=0x558737c62f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c62f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f34d4014a30 tx=0x7f34d4016040 comp rx=0 tx=0).stop 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 -- 172.21.15.16:0/1661497960 shutdown_connections 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c68bb0 0x558737c62f20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c5e2f0 0x558737c629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 --2- 172.21.15.16:0/1661497960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c790d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 -- 172.21.15.16:0/1661497960 >> 172.21.15.16:0/1661497960 conn(0x558737c59f60 msgr2=0x558737c68410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 -- 172.21.15.16:0/1661497960 shutdown_connections 2024-09-15T22:38:18.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 -- 172.21.15.16:0/1661497960 wait complete. 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.344+0000 7f34e8404740 1 -- start start 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 1 Processor -- start 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c83300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c62200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c68bb0 0x558737c62740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558737c7aa90 con 0x558737c5e2f0 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558737c7a3c0 con 0x558737c5d9d0 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558737c7b160 con 0x558737c68bb0 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34e8404740 10 client.0 _ll_register_callbacks cb 0x7f34d0009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c62200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c62200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43134/0 (socket says 172.21.15.16:43134) 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 -- 172.21.15.16:0/1109933649 learned_addr learned my addr 172.21.15.16:0/1109933649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:18.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34deffd640 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c83300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 -- 172.21.15.16:0/1109933649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c68bb0 msgr2=0x558737c62740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34deffd640 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c68bb0 0x558737c62740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c68bb0 0x558737c62740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 -- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 msgr2=0x558737c83300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c83300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.345+0000 7f34df7fe640 1 -- 172.21.15.16:0/1109933649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558737c62c80 con 0x558737c5e2f0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34df7fe640 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c62200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34d0004810 tx=0x7f34d000f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbecaca4eef727 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34beffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34beffd640 1 -- 172.21.15.16:0/1109933649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d002e020 con 0x558737c5e2f0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34beffd640 1 -- 172.21.15.16:0/1109933649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f34d001d8d0 con 0x558737c5e2f0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d4032070 con 0x558737c5e2f0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34beffd640 1 -- 172.21.15.16:0/1109933649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34d001dad0 con 0x558737c5e2f0 2024-09-15T22:38:18.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 10 client.15213 fetch_fsmap learned FSMap version 25 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558737c81b00 con 0x558737c5e2f0 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34beffd640 1 -- 172.21.15.16:0/1109933649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4032070 con 0x558737c5e2f0 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34beffd640 1 -- 172.21.15.16:0/1109933649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f34d0014440 con 0x558737c5e2f0 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 10 client.15213 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 20 client.15213 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737ce8680 0x558737ce7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 4 client.15213 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.346+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "status"}) -- 0x558737bcbb10 con 0x558737ce8680 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34deffd640 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737ce8680 0x558737ce7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34deffd640 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737ce8680 0x558737ce7390 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b99630698397da server_cookie=173945570509e99f in_seq=0 out_seq=0 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34beffd640 10 client.15213 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34beffd640 1 -- 172.21.15.16:0/1109933649 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x558737bcbb10 con 0x558737ce8680 2024-09-15T22:38:18.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34beffd640 10 client.15213 handle_command_reply: tid=0 2024-09-15T22:38:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 client.15213 shutdown 2024-09-15T22:38:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 msgr2=0x558737c62200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c62200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f34d0004810 tx=0x7f34d000f2d0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.374 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:18.374 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:38:18.374 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 129.096787573, 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 25, 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 89, 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 85, 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 139.947941738 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:18.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 shutdown_connections 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558737ce8680 0x558737ce7390 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558737c68bb0 0x558737c62740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558737c5e2f0 0x558737c62200 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34beffd640 0 client.15213 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 --2- 172.21.15.16:0/1109933649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558737c5d9d0 0x558737c83300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 >> 172.21.15.16:0/1109933649 conn(0x558737c59f60 msgr2=0x558737c605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.347+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 shutdown_connections 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e8404740 1 -- 172.21.15.16:0/1109933649 wait complete. 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e8404740 20 client.15213 trim_cache size 0 max 16384 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f34c80a3240 msgr2=0x7f34c80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f34c80a3240 0x7f34c80a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34d80bd360 tx=0x7f34d8087e80 comp rx=0 tx=0).stop 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0105a90 msgr2=0x7f34e0197bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0105a90 0x7f34e0197bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f34d8006010 tx=0x7f34d8003f10 comp rx=0 tx=0).stop 2024-09-15T22:38:18.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 shutdown_connections 2024-09-15T22:38:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f34c80a3240 0x7f34c80a5700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34e0105a90 0x7f34e0197bf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f34e0101bc0 0x7f34e01976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 --2- 172.21.15.16:0/2147372285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34e0101240 0x7f34e0197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 >> 172.21.15.16:0/2147372285 conn(0x7f34e0077210 msgr2=0x7f34e0103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 shutdown_connections 2024-09-15T22:38:18.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:18.348+0000 7f34e602b640 1 -- 172.21.15.16:0/2147372285 wait complete. 2024-09-15T22:38:19.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:18 smithi016 ceph-mon[31492]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-15T22:38:19.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:18 smithi016 ceph-mon[31492]: Cluster is now healthy 2024-09-15T22:38:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:18 smithi130 ceph-mon[35742]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-15T22:38:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:18 smithi130 ceph-mon[35742]: Cluster is now healthy 2024-09-15T22:38:19.043 INFO:teuthology.run_tasks:Running task sequential... 2024-09-15T22:38:19.053 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-15T22:38:19.053 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-15T22:38:19.054 INFO:teuthology.task.print:Enabling mgr modules 2024-09-15T22:38:19.054 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-15T22:38:19.054 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:38:19.055 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:38:19.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 -- 172.21.15.16:0/1105727938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e78101c30 msgr2=0x7f7e781020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 --2- 172.21.15.16:0/1105727938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e78101c30 0x7f7e781020b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009900 tx=0x7f7e6802a410 comp rx=0 tx=0).stop 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 -- 172.21.15.16:0/1105727938 shutdown_connections 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 --2- 172.21.15.16:0/1105727938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e781025f0 0x7f7e78108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 --2- 172.21.15.16:0/1105727938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e78101c30 0x7f7e781020b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 --2- 172.21.15.16:0/1105727938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e78100a40 0x7f7e78100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 -- 172.21.15.16:0/1105727938 >> 172.21.15.16:0/1105727938 conn(0x7f7e780fc1e0 msgr2=0x7f7e780fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 -- 172.21.15.16:0/1105727938 shutdown_connections 2024-09-15T22:38:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.161+0000 7f7e806ba640 1 -- 172.21.15.16:0/1105727938 wait complete. 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 Processor -- start 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 -- start start 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e78100a40 0x7f7e78197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e78101c30 0x7f7e78198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781025f0 0x7f7e7819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7810c6c0 con 0x7f7e781025f0 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7810c540 con 0x7f7e78100a40 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e806ba640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7810c840 con 0x7f7e78101c30 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e7ec57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781025f0 0x7f7e7819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e7ec57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781025f0 0x7f7e7819f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43162/0 (socket says 172.21.15.16:43162) 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e7ec57640 1 -- 172.21.15.16:0/776215491 learned_addr learned my addr 172.21.15.16:0/776215491 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e7e456640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e78100a40 0x7f7e78197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.162+0000 7f7e7dc55640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e78101c30 0x7f7e78198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e7ec57640 1 -- 172.21.15.16:0/776215491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e78101c30 msgr2=0x7f7e78198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e7ec57640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e78101c30 0x7f7e78198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e7ec57640 1 -- 172.21.15.16:0/776215491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e78100a40 msgr2=0x7f7e78197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e7ec57640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e78100a40 0x7f7e78197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e7ec57640 1 -- 172.21.15.16:0/776215491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7819f960 con 0x7f7e781025f0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e7dc55640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e78101c30 0x7f7e78198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e7ec57640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781025f0 0x7f7e7819f2c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000eaa0 tx=0x7f7e7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e6f7fe640 1 -- 172.21.15.16:0/776215491 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000cdb0 con 0x7f7e781025f0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e6f7fe640 1 -- 172.21.15.16:0/776215491 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e70005020 con 0x7f7e781025f0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7819fc50 con 0x7f7e781025f0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e6f7fe640 1 -- 172.21.15.16:0/776215491 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e700116b0 con 0x7f7e781025f0 2024-09-15T22:38:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.163+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e781a01a0 con 0x7f7e781025f0 2024-09-15T22:38:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.164+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e781a0370 con 0x7f7e781025f0 2024-09-15T22:38:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.165+0000 7f7e6f7fe640 1 -- 172.21.15.16:0/776215491 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7e70005560 con 0x7f7e781025f0 2024-09-15T22:38:19.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.166+0000 7f7e6f7fe640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7e640a32f0 0x7f7e640a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.166+0000 7f7e7e456640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7e640a32f0 0x7f7e640a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.166+0000 7f7e6f7fe640 1 -- 172.21.15.16:0/776215491 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f7e700cffd0 con 0x7f7e781025f0 2024-09-15T22:38:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.166+0000 7f7e7e456640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7e640a32f0 0x7f7e640a57b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7e78199220 tx=0x7f7e74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.170+0000 7f7e6f7fe640 1 -- 172.21.15.16:0/776215491 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e70016030 con 0x7f7e781025f0 2024-09-15T22:38:19.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:18 smithi059 ceph-mon[34872]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-15T22:38:19.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:18 smithi059 ceph-mon[34872]: Cluster is now healthy 2024-09-15T22:38:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.395+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f7e781a05e0 con 0x7f7e781025f0 2024-09-15T22:38:19.727 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.726+0000 7f7e6f7fe640 1 -- 172.21.15.16:0/776215491 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f7e7009c910 con 0x7f7e781025f0 2024-09-15T22:38:19.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7e640a32f0 msgr2=0x7f7e640a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7e640a32f0 0x7f7e640a57b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7e78199220 tx=0x7f7e74005f90 comp rx=0 tx=0).stop 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781025f0 msgr2=0x7f7e7819f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781025f0 0x7f7e7819f2c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000eaa0 tx=0x7f7e7000ef60 comp rx=0 tx=0).stop 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 shutdown_connections 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f7e640a32f0 0x7f7e640a57b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e781025f0 0x7f7e7819f2c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7e78101c30 0x7f7e78198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 --2- 172.21.15.16:0/776215491 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e78100a40 0x7f7e78197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 >> 172.21.15.16:0/776215491 conn(0x7f7e780fc1e0 msgr2=0x7f7e780fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 shutdown_connections 2024-09-15T22:38:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:19.728+0000 7f7e806ba640 1 -- 172.21.15.16:0/776215491 wait complete. 2024-09-15T22:38:19.746 INFO:teuthology.run_tasks:Running task kclient... 2024-09-15T22:38:19.759 INFO:tasks.kclient:Mounting kernel clients... 2024-09-15T22:38:19.759 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-15T22:38:19.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:19.759 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-15T22:38:19.776 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-15T22:38:19.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:19.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:38:19.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:19.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-15T22:38:19.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:19 smithi016 ceph-mon[31492]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-15T22:38:19.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:19 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/776215491' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:38:19.909 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:19 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:19.909 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:19 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:19.909 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:19 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:19.884+0000 7f4bb148c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:38:19.966 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-15T22:38:19.966 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T22:38:19.966 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:19.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-15T22:38:20.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:19 smithi130 ceph-mon[35742]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-15T22:38:20.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:19 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/776215491' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:38:20.029 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:20.029 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:20.029 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:19 smithi130 ceph-mgr[37554]: -- 172.21.15.130:0/2478424275 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559203dc50e0 con 0x559203da2800 2024-09-15T22:38:20.029 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:19.880+0000 7f80cd6ac040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:38:20.029 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:19.937+0000 7f80cd6ac040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:38:20.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:19 smithi059 ceph-mon[34872]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-15T22:38:20.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:19 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/776215491' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:38:20.095 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: ignoring --setuser ceph since I am not root 2024-09-15T22:38:20.095 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: ignoring --setgroup ceph since I am not root 2024-09-15T22:38:20.095 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:19 smithi059 ceph-mgr[36247]: -- 172.21.15.59:0/1639255967 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6f17810e0 con 0x55a6f175f400 2024-09-15T22:38:20.095 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:19.882+0000 7fac2f3bb040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:38:20.095 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:19.940+0000 7fac2f3bb040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:38:20.096 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:20.092+0000 7fac2f3bb040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:38:20.180 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:19 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:19.946+0000 7f4bb148c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:38:20.180 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:20.107+0000 7f4bb148c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:38:20.180 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:20.178+0000 7f4bb148c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:38:20.272 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:20.438 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:20.085+0000 7f80cd6ac040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:38:20.438 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:20.150+0000 7f80cd6ac040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:38:20.438 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:20.294+0000 7f80cd6ac040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:38:20.438 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:20.435+0000 7f80cd6ac040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:38:20.443 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:20.162+0000 7fac2f3bb040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:38:20.443 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:20.314+0000 7fac2f3bb040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:38:20.500 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:20.346+0000 7f4bb148c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:38:20.728 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:20.460+0000 7fac2f3bb040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:38:20.795 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:20.498+0000 7f4bb148c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:38:20.795 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/776215491' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:38:20.795 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-mon[31492]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.792+0000 7f4132cba640 1 -- 172.21.15.16:0/2977787265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102570 msgr2=0x7f412c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.792+0000 7f4132cba640 1 --2- 172.21.15.16:0/2977787265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102570 0x7f412c1029b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f411c009ca0 tx=0x7f411c02a950 comp rx=0 tx=0).stop 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 -- 172.21.15.16:0/2977787265 shutdown_connections 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 --2- 172.21.15.16:0/2977787265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c102ef0 0x7f412c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 --2- 172.21.15.16:0/2977787265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102570 0x7f412c1029b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 --2- 172.21.15.16:0/2977787265 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f412c100e20 0x7f412c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 -- 172.21.15.16:0/2977787265 >> 172.21.15.16:0/2977787265 conn(0x7f412c0fc7d0 msgr2=0x7f412c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 -- 172.21.15.16:0/2977787265 shutdown_connections 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 -- 172.21.15.16:0/2977787265 wait complete. 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 Processor -- start 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.793+0000 7f4132cba640 1 -- start start 2024-09-15T22:38:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4132cba640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c100e20 0x7f412c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4132cba640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f412c102570 0x7f412c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4132cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102ef0 0x7f412c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4132cba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f412c110f80 con 0x7f412c102ef0 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4132cba640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f412c110e00 con 0x7f412c100e20 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4132cba640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f412c111100 con 0x7f412c102570 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c100e20 0x7f412c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c100e20 0x7f412c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:39586/0 (socket says 172.21.15.16:39586) 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f412bfff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f412c102570 0x7f412c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 -- 172.21.15.16:0/3145015490 learned_addr learned my addr 172.21.15.16:0/3145015490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4131257640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102ef0 0x7f412c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 -- 172.21.15.16:0/3145015490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f412c102570 msgr2=0x7f412c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f412c102570 0x7f412c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 -- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102ef0 msgr2=0x7f412c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102ef0 0x7f412c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4130a56640 1 -- 172.21.15.16:0/3145015490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c1a6a70 con 0x7f412c100e20 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f412bfff640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f412c102570 0x7f412c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.794+0000 7f4131257640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102ef0 0x7f412c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.795+0000 7f4130a56640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c100e20 0x7f412c0fcc80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f411400e9f0 tx=0x7f411400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.795+0000 7f4129ffb640 1 -- 172.21.15.16:0/3145015490 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411400cd80 con 0x7f412c100e20 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.795+0000 7f4129ffb640 1 -- 172.21.15.16:0/3145015490 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4114005500 con 0x7f412c100e20 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.795+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c1a6d60 con 0x7f412c100e20 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.795+0000 7f4129ffb640 1 -- 172.21.15.16:0/3145015490 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114010690 con 0x7f412c100e20 2024-09-15T22:38:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.795+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c1a71f0 con 0x7f412c100e20 2024-09-15T22:38:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.796+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c101200 con 0x7f412c100e20 2024-09-15T22:38:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.797+0000 7f4129ffb640 1 -- 172.21.15.16:0/3145015490 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f41140046c0 con 0x7f412c100e20 2024-09-15T22:38:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.798+0000 7f4129ffb640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 0x7f41180a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.798+0000 7f412bfff640 1 -- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 msgr2=0x7f41180a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.799+0000 7f4129ffb640 1 -- 172.21.15.16:0/3145015490 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f41140d0170 con 0x7f412c100e20 2024-09-15T22:38:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.799+0000 7f412bfff640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 0x7f41180a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:38:20.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:20.806+0000 7f4129ffb640 1 -- 172.21.15.16:0/3145015490 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f411409cab0 con 0x7f412c100e20 2024-09-15T22:38:20.995 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/776215491' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:38:20.996 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-mon[35742]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:38:20.996 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:20.746+0000 7f80cd6ac040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:38:20.996 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:20.807+0000 7f80cd6ac040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:38:21.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.000+0000 7f412bfff640 1 -- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 msgr2=0x7f41180a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:21.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.000+0000 7f412bfff640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 0x7f41180a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:38:21.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.020+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f412c10a0a0 con 0x7f412c100e20 2024-09-15T22:38:21.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.021+0000 7f4129ffb640 1 -- 172.21.15.16:0/3145015490 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f411409c3b0 con 0x7f412c100e20 2024-09-15T22:38:21.022 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T22:38:21.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 msgr2=0x7f41180a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:38:21.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 0x7f41180a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c100e20 msgr2=0x7f412c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c100e20 0x7f412c0fcc80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f411400e9f0 tx=0x7f411400edf0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 shutdown_connections 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f41180a3290 0x7f41180a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c102ef0 0x7f412c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f412c102570 0x7f412c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 --2- 172.21.15.16:0/3145015490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f412c100e20 0x7f412c0fcc80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 >> 172.21.15.16:0/3145015490 conn(0x7f412c0fc7d0 msgr2=0x7f412c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 shutdown_connections 2024-09-15T22:38:21.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21.023+0000 7f4132cba640 1 -- 172.21.15.16:0/3145015490 wait complete. 2024-09-15T22:38:21.045 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/776215491' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:38:21.046 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-mon[34872]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:38:21.046 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:20.784+0000 7fac2f3bb040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:38:21.046 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:20.851+0000 7fac2f3bb040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:38:21.046 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:21.043+0000 7fac2f3bb040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:38:21.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:20.840+0000 7f4bb148c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:38:21.057 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:20 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:20.906+0000 7f4bb148c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:38:21.278 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:20.993+0000 7f80cd6ac040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:38:21.512 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:21 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:21.110+0000 7f4bb148c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:38:21.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:21 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3145015490' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:38:21.843 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T22:38:21.843 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T22:38:21.843 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-15T22:38:21.843 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-15T22:38:21.843 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:21.843 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T22:38:21.844 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T22:38:21.844 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T22:38:21.844 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-15T22:38:21.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:21.844 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:38:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42188sec preferred_lft 42188sec 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:38:21.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-15T22:38:21.863 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T22:38:21.863 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:38:21.863 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:38:21.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-15T22:38:21.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-15T22:38:21.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-15T22:38:21.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T22:38:21.863 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:38:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:38:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:22.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:22.010 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T22:38:22.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:21 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3145015490' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:38:22.029 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:21.871+0000 7f80cd6ac040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:38:22.029 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:21.939+0000 7f80cd6ac040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:38:22.029 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:21 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3145015490' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:38:22.030 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:21.957+0000 7fac2f3bb040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:38:22.036 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-15T22:38:22.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:22.038 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-15T22:38:22.097 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-15T22:38:22.097 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T22:38:22.097 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T22:38:22.097 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T22:38:22.097 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T22:38:22.098 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:38:22.098 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:38:22.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T22:38:22.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T22:38:22.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T22:38:22.098 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:38:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:38:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:22.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:22.230 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-15T22:38:22.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:22.246 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-15T22:38:22.302 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:38:22.302 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:38:22.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T22:38:22.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-15T22:38:22.303 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:38:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:38:22.401 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:22 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:22.065+0000 7f4bb148c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:38:22.401 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:22 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:22.138+0000 7f4bb148c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:38:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:22.423 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-15T22:38:22.423 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:38:22.424 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:38:22.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-15T22:38:22.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T22:38:22.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-15T22:38:22.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-15T22:38:22.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-15T22:38:22.424 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:38:22.443 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:22.027+0000 7fac2f3bb040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:38:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:38:22.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:22.546 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:38:22.546 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-15T22:38:22.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-15T22:38:22.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T22:38:22.546 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-15T22:38:22.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:38:22.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:22.627 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:38:22.627 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:22.643 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-15T22:38:22.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:38:22.644 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:22.699 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-15T22:38:22.699 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-15T22:38:22.699 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/8f003b02-afc3-47c1-8722-deab403be128 2024-09-15T22:38:22.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:22.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/8f003b02-afc3-47c1-8722-deab403be128 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-15T22:38:22.785 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-15T22:38:22.786 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-15T22:38:22.909 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:22.647+0000 7f80cd6ac040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:38:22.909 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:22.761+0000 7f80cd6ac040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-09-15T22:38:22.931 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.59:3300,172.21.15.130:3300" 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@af43fc90-73b1-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/8f003b02-afc3-47c1-8722-deab403be128 2024-09-15T22:38:22.932 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.16:3300/172.21.15.59:3300/172.21.15.130:3300" will pass to kernel 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T22:38:22.933 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T22:38:22.935 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T22:38:22.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:38:22.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:22.967 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:38:23.036 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:22.764+0000 7fac2f3bb040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:38:23.036 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:22.885+0000 7fac2f3bb040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:38:23.056 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:22 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:22.931+0000 7f4bb148c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:38:23.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:38:23.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:23.067 INFO:teuthology.orchestra.run.smithi016.stdout:client15246 2024-09-15T22:38:23.067 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:38:23.067 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:38:23.085 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:38:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:38:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:23.183 INFO:teuthology.orchestra.run.smithi016.stdout:client15246 2024-09-15T22:38:23.184 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:38:23.187 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:22.906+0000 7f80cd6ac040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:38:23.187 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:22.968+0000 7f80cd6ac040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:38:23.187 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.117+0000 7f80cd6ac040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:38:23.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:38:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:23.242 INFO:teuthology.orchestra.run.smithi016.stdout:client15246 2024-09-15T22:38:23.243 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/af43fc90-73b1-11ef-bceb-c7b262605968.client15246/status') 2024-09-15T22:38:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:38:23.305 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.15246 (3)192.168.0.1:0/3852512286 2024-09-15T22:38:23.305 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-15T22:38:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-15T22:38:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-15T22:38:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 2.446e-05 s, 2.9 MB/s 2024-09-15T22:38:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:38:23.314 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:38:23.322 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:38:23.322 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:38:23.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-15T22:38:23.327 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.034+0000 7fac2f3bb040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:38:23.327 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.097+0000 7fac2f3bb040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:38:23.327 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.252+0000 7fac2f3bb040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:38:23.327 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.325+0000 7fac2f3bb040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:38:23.345 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.060+0000 7f4bb148c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:38:23.345 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.214+0000 7f4bb148c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:38:23.345 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.281+0000 7f4bb148c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:38:23.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/3265473195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0101b60 msgr2=0x7f1fc0101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/3265473195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0101b60 0x7f1fc0101f60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0009900 tx=0x7f1fb002a410 comp rx=0 tx=0).stop 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/3265473195 shutdown_connections 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/3265473195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0103710 0x7f1fc0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/3265473195 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc0102d50 0x7f1fc01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/3265473195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0101b60 0x7f1fc0101f60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/3265473195 >> 172.21.15.16:0/3265473195 conn(0x7f1fc00fd2c0 msgr2=0x7f1fc00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/3265473195 shutdown_connections 2024-09-15T22:38:23.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.428+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/3265473195 wait complete. 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 Processor -- start 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 -- start start 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0101b60 0x7f1fc0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc0102d50 0x7f1fc01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0103710 0x7f1fc019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc010d7e0 con 0x7f1fc0103710 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc010d660 con 0x7f1fc0101b60 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fc5b4b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc010d960 con 0x7f1fc0102d50 2024-09-15T22:38:23.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0103710 0x7f1fc019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fbffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0103710 0x7f1fc019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46596/0 (socket says 172.21.15.16:46596) 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fbffff640 1 -- 172.21.15.16:0/1182045674 learned_addr learned my addr 172.21.15.16:0/1182045674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fbf7fe640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0101b60 0x7f1fc0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fb6ffd640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc0102d50 0x7f1fc01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fbffff640 1 -- 172.21.15.16:0/1182045674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc0102d50 msgr2=0x7f1fc01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fbffff640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc0102d50 0x7f1fc01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.429+0000 7f1fbffff640 1 -- 172.21.15.16:0/1182045674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0101b60 msgr2=0x7f1fc0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fbffff640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0101b60 0x7f1fc0197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fbffff640 1 -- 172.21.15.16:0/1182045674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc019fa90 con 0x7f1fc0103710 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fbffff640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0103710 0x7f1fc019f420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800efd0 tx=0x7f1fb800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fbd7fa640 1 -- 172.21.15.16:0/1182045674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8003b70 con 0x7f1fc0103710 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc019fd80 con 0x7f1fc0103710 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fbd7fa640 1 -- 172.21.15.16:0/1182045674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb8003d10 con 0x7f1fc0103710 2024-09-15T22:38:23.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fbd7fa640 1 -- 172.21.15.16:0/1182045674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80109f0 con 0x7f1fc0103710 2024-09-15T22:38:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.430+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01a0270 con 0x7f1fc0103710 2024-09-15T22:38:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.431+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc0108f40 con 0x7f1fc0103710 2024-09-15T22:38:23.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.433+0000 7f1fbd7fa640 1 -- 172.21.15.16:0/1182045674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f1fb8004020 con 0x7f1fc0103710 2024-09-15T22:38:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.434+0000 7f1fbd7fa640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 0x7f1fac0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.434+0000 7f1fbf7fe640 1 -- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 msgr2=0x7f1fac0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.434+0000 7f1fbd7fa640 1 -- 172.21.15.16:0/1182045674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f1fb80d0170 con 0x7f1fc0103710 2024-09-15T22:38:23.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.434+0000 7f1fbf7fe640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 0x7f1fac0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:38:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.438+0000 7f1fbd7fa640 1 -- 172.21.15.16:0/1182045674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fb809cab0 con 0x7f1fc0103710 2024-09-15T22:38:23.473 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.185+0000 7f80cd6ac040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:38:23.474 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.245+0000 7f80cd6ac040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:38:23.474 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.405+0000 7f80cd6ac040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:38:23.623 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.387+0000 7fac2f3bb040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:38:23.623 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.553+0000 7fac2f3bb040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:38:23.623 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.621+0000 7fac2f3bb040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:38:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.635+0000 7f1fbf7fe640 1 -- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 msgr2=0x7f1fac0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.635+0000 7f1fbf7fe640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 0x7f1fac0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:38:23.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.656+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f1fc0062df0 con 0x7f1fc0103710 2024-09-15T22:38:23.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.708+0000 7f1fbd7fa640 1 -- 172.21.15.16:0/1182045674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7f1fb809c3b0 con 0x7f1fc0103710 2024-09-15T22:38:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 msgr2=0x7f1fac0a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:38:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 0x7f1fac0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0103710 msgr2=0x7f1fc019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0103710 0x7f1fc019f420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800efd0 tx=0x7f1fb800c6a0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 shutdown_connections 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f1fac0a3340 0x7f1fac0a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1fc0103710 0x7f1fc019f420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1fc0102d50 0x7f1fc01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 --2- 172.21.15.16:0/1182045674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fc0101b60 0x7f1fc0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 >> 172.21.15.16:0/1182045674 conn(0x7f1fc00fd2c0 msgr2=0x7f1fc0106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 shutdown_connections 2024-09-15T22:38:23.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.710+0000 7f1fc5b4b640 1 -- 172.21.15.16:0/1182045674 wait complete. 2024-09-15T22:38:23.713 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.449+0000 7f4bb148c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:38:23.713 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.523+0000 7f4bb148c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:38:23.713 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.588+0000 7f4bb148c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:38:23.723 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-15T22:38:23.733 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-15T22:38:23.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T22:38:23.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1182045674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-15T22:38:23.780 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.471+0000 7f80cd6ac040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:38:23.780 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.539+0000 7f80cd6ac040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:38:23.780 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.621+0000 7f80cd6ac040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:38:23.780 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.681+0000 7f80cd6ac040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:38:23.780 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:23.741+0000 7f80cd6ac040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 -- 172.21.15.16:0/2896757071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60107890 msgr2=0x7fcc60109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/2896757071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60107890 0x7fcc60109c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009ca0 tx=0x7fcc5802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 -- 172.21.15.16:0/2896757071 shutdown_connections 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/2896757071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60107890 0x7fcc60109c80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/2896757071 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc60104f60 0x7fcc60107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/2896757071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60102630 0x7fcc60104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 -- 172.21.15.16:0/2896757071 >> 172.21.15.16:0/2896757071 conn(0x7fcc600fc170 msgr2=0x7fcc600fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:23.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 -- 172.21.15.16:0/2896757071 shutdown_connections 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.884+0000 7fcc67cd7640 1 -- 172.21.15.16:0/2896757071 wait complete. 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 Processor -- start 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 -- start start 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc60102630 0x7fcc60197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60104f60 0x7fcc60198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60107890 0x7fcc6019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6010d7b0 con 0x7fcc60104f60 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6010d630 con 0x7fcc60107890 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc67cd7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6010d930 con 0x7fcc60102630 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc65272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60104f60 0x7fcc60198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc65272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60104f60 0x7fcc60198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46610/0 (socket says 172.21.15.16:46610) 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc65272640 1 -- 172.21.15.16:0/3478098694 learned_addr learned my addr 172.21.15.16:0/3478098694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:23.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc66274640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60107890 0x7fcc6019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.885+0000 7fcc65a73640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc60102630 0x7fcc60197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc65272640 1 -- 172.21.15.16:0/3478098694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc60102630 msgr2=0x7fcc60197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc65272640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc60102630 0x7fcc60197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc65272640 1 -- 172.21.15.16:0/3478098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60107890 msgr2=0x7fcc6019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc65272640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60107890 0x7fcc6019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc65272640 1 -- 172.21.15.16:0/3478098694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6019fa50 con 0x7fcc60104f60 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc66274640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60107890 0x7fcc6019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc65272640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60104f60 0x7fcc60198370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005fd0 tx=0x7fcc5000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc56ffd640 1 -- 172.21.15.16:0/3478098694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50017070 con 0x7fcc60104f60 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc56ffd640 1 -- 172.21.15.16:0/3478098694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc50009c30 con 0x7fcc60104f60 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6019fd40 con 0x7fcc60104f60 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc56ffd640 1 -- 172.21.15.16:0/3478098694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50010460 con 0x7fcc60104f60 2024-09-15T22:38:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.886+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc601a0230 con 0x7fcc60104f60 2024-09-15T22:38:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.887+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc600fdc60 con 0x7fcc60104f60 2024-09-15T22:38:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.888+0000 7fcc56ffd640 1 -- 172.21.15.16:0/3478098694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fcc50003e70 con 0x7fcc60104f60 2024-09-15T22:38:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.889+0000 7fcc56ffd640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcc4c0a32a0 0x7fcc4c0a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.889+0000 7fcc65a73640 1 -- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcc4c0a32a0 msgr2=0x7fcc4c0a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.889+0000 7fcc56ffd640 1 -- 172.21.15.16:0/3478098694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7fcc500d0060 con 0x7fcc60104f60 2024-09-15T22:38:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.889+0000 7fcc65a73640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcc4c0a32a0 0x7fcc4c0a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:38:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:23.896+0000 7fcc56ffd640 1 -- 172.21.15.16:0/3478098694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc50020030 con 0x7fcc60104f60 2024-09-15T22:38:23.900 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1182045674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-15T22:38:23.901 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.689+0000 7fac2f3bb040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:38:23.901 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.776+0000 7fac2f3bb040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:38:23.901 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.837+0000 7fac2f3bb040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:38:23.901 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:23.898+0000 7fac2f3bb040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:38:23.997 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1182045674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-15T22:38:23.997 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.759+0000 7f4bb148c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:38:23.998 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.830+0000 7f4bb148c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:38:23.998 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.907+0000 7f4bb148c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:38:23.998 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:23.996+0000 7f4bb148c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:38:24.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.076+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc6002da10 con 0x7fcc60104f60 2024-09-15T22:38:24.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.079+0000 7fcc56ffd640 1 -- 172.21.15.16:0/3478098694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24764 (secure 0 0 0) 0x7fcc5009ca00 con 0x7fcc60104f60 2024-09-15T22:38:24.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:24.081 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:38:13.491701+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:38:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcc4c0a32a0 msgr2=0x7fcc4c0a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:38:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcc4c0a32a0 0x7fcc4c0a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60104f60 msgr2=0x7fcc60198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60104f60 0x7fcc60198370 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005fd0 tx=0x7fcc5000dde0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 shutdown_connections 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7fcc4c0a32a0 0x7fcc4c0a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc60107890 0x7fcc6019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60104f60 0x7fcc60198370 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 --2- 172.21.15.16:0/3478098694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc60102630 0x7fcc60197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 >> 172.21.15.16:0/3478098694 conn(0x7fcc600fc170 msgr2=0x7fcc60102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 shutdown_connections 2024-09-15T22:38:24.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.081+0000 7fcc67cd7640 1 -- 172.21.15.16:0/3478098694 wait complete. 2024-09-15T22:38:24.094 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:31:31.005703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T22:36:04.983944+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T22:36:07.225026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-15T22:36:13.700134+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-15T22:38:24.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T22:38:24.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4010695407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4100a00 msgr2=0x7f2df4100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4010695407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4100a00 0x7f2df4100e00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009900 tx=0x7f2de402a410 comp rx=0 tx=0).stop 2024-09-15T22:38:24.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4010695407 shutdown_connections 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4010695407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2df41025b0 0x7f2df4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4010695407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df4101bf0 0x7f2df4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4010695407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4100a00 0x7f2df4100e00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4010695407 >> 172.21.15.16:0/4010695407 conn(0x7f2df40fc140 msgr2=0x7f2df40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4010695407 shutdown_connections 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.202+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4010695407 wait complete. 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 Processor -- start 2024-09-15T22:38:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 -- start start 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2df4100a00 0x7f2df4104990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4101bf0 0x7f2df4102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df41025b0 0x7f2df4103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2df410c700 con 0x7f2df4101bf0 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2df410c580 con 0x7f2df41025b0 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df8cd0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2df410c880 con 0x7f2df4100a00 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4101bf0 0x7f2df4102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4101bf0 0x7f2df4102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46632/0 (socket says 172.21.15.16:46632) 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 -- 172.21.15.16:0/4076557669 learned_addr learned my addr 172.21.15.16:0/4076557669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df2d9d640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df41025b0 0x7f2df4103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df259c640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2df4100a00 0x7f2df4104990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 -- 172.21.15.16:0/4076557669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2df4100a00 msgr2=0x7f2df4104990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2df4100a00 0x7f2df4104990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 -- 172.21.15.16:0/4076557669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df41025b0 msgr2=0x7f2df4103510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df41025b0 0x7f2df4103510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.203+0000 7f2df1d9b640 1 -- 172.21.15.16:0/4076557669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df4103da0 con 0x7f2df4101bf0 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.204+0000 7f2df1d9b640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4101bf0 0x7f2df4102fd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9f0 tx=0x7f2de800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.204+0000 7f2de37fe640 1 -- 172.21.15.16:0/4076557669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de800cd80 con 0x7f2df4101bf0 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.204+0000 7f2de37fe640 1 -- 172.21.15.16:0/4076557669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8010910 con 0x7f2df4101bf0 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.204+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df41a2090 con 0x7f2df4101bf0 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.204+0000 7f2de37fe640 1 -- 172.21.15.16:0/4076557669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8018990 con 0x7f2df4101bf0 2024-09-15T22:38:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.204+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df41a2660 con 0x7f2df4101bf0 2024-09-15T22:38:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.205+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df41a2920 con 0x7f2df4101bf0 2024-09-15T22:38:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.206+0000 7f2de37fe640 1 -- 172.21.15.16:0/4076557669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f2de8010ac0 con 0x7f2df4101bf0 2024-09-15T22:38:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.207+0000 7f2de37fe640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2ddc0a32a0 0x7f2ddc0a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.207+0000 7f2df259c640 1 -- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2ddc0a32a0 msgr2=0x7f2ddc0a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:24.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.207+0000 7f2de37fe640 1 -- 172.21.15.16:0/4076557669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f2de80cfff0 con 0x7f2df4101bf0 2024-09-15T22:38:24.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.207+0000 7f2df259c640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2ddc0a32a0 0x7f2ddc0a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:38:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.211+0000 7f2de37fe640 1 -- 172.21.15.16:0/4076557669 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2de809c340 con 0x7f2df4101bf0 2024-09-15T22:38:24.306 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:24.059+0000 7f4bb148c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:38:24.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:24.127+0000 7f4bb148c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:38:24.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.392+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2df41a2af0 con 0x7f2df4101bf0 2024-09-15T22:38:24.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.392+0000 7f2de37fe640 1 -- 172.21.15.16:0/4076557669 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f2de80a0640 con 0x7f2df4101bf0 2024-09-15T22:38:24.394 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-15T22:38:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2ddc0a32a0 msgr2=0x7f2ddc0a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:38:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2ddc0a32a0 0x7f2ddc0a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4101bf0 msgr2=0x7f2df4102fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4101bf0 0x7f2df4102fd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9f0 tx=0x7f2de800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 shutdown_connections 2024-09-15T22:38:24.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f2ddc0a32a0 0x7f2ddc0a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2df41025b0 0x7f2df4103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df4101bf0 0x7f2df4102fd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 --2- 172.21.15.16:0/4076557669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2df4100a00 0x7f2df4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.394+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 >> 172.21.15.16:0/4076557669 conn(0x7f2df40fc140 msgr2=0x7f2df40693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.395+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 shutdown_connections 2024-09-15T22:38:24.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.395+0000 7f2df8cd0640 1 -- 172.21.15.16:0/4076557669 wait complete. 2024-09-15T22:38:24.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T22:38:24.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 -- 172.21.15.16:0/2191487158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a481077e0 msgr2=0x7f8a48109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 --2- 172.21.15.16:0/2191487158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a481077e0 0x7f8a48109c50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009ca0 tx=0x7f8a4002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 -- 172.21.15.16:0/2191487158 shutdown_connections 2024-09-15T22:38:24.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 --2- 172.21.15.16:0/2191487158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a481077e0 0x7f8a48109c50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 --2- 172.21.15.16:0/2191487158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a48104eb0 0x7f8a481072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 --2- 172.21.15.16:0/2191487158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48102580 0x7f8a48104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 -- 172.21.15.16:0/2191487158 >> 172.21.15.16:0/2191487158 conn(0x7f8a480fc140 msgr2=0x7f8a480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 -- 172.21.15.16:0/2191487158 shutdown_connections 2024-09-15T22:38:24.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.513+0000 7f8a4e949640 1 -- 172.21.15.16:0/2191487158 wait complete. 2024-09-15T22:38:24.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 Processor -- start 2024-09-15T22:38:24.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 -- start start 2024-09-15T22:38:24.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48102580 0x7f8a4819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48104eb0 0x7f8a4819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a481077e0 0x7f8a481a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810d7c0 con 0x7f8a48102580 2024-09-15T22:38:24.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810d640 con 0x7f8a48104eb0 2024-09-15T22:38:24.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a4e949640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810d940 con 0x7f8a481077e0 2024-09-15T22:38:24.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.514+0000 7f8a47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48102580 0x7f8a4819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48104eb0 0x7f8a4819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48104eb0 0x7f8a4819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:40388/0 (socket says 172.21.15.16:40388) 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 -- 172.21.15.16:0/3855382938 learned_addr learned my addr 172.21.15.16:0/3855382938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a47fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48102580 0x7f8a4819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46642/0 (socket says 172.21.15.16:46642) 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a4cee6640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a481077e0 0x7f8a481a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 -- 172.21.15.16:0/3855382938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a481077e0 msgr2=0x7f8a481a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a481077e0 0x7f8a481a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 -- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48102580 msgr2=0x7f8a4819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48102580 0x7f8a4819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 -- 172.21.15.16:0/3855382938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a481a1ce0 con 0x7f8a48104eb0 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a47fff640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48102580 0x7f8a4819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a477fe640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48104eb0 0x7f8a4819a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9f0 tx=0x7f8a3c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a457fa640 1 -- 172.21.15.16:0/3855382938 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00ccc0 con 0x7f8a48104eb0 2024-09-15T22:38:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a457fa640 1 -- 172.21.15.16:0/3855382938 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a3c005500 con 0x7f8a48104eb0 2024-09-15T22:38:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a481a1fd0 con 0x7f8a48104eb0 2024-09-15T22:38:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a457fa640 1 -- 172.21.15.16:0/3855382938 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c010690 con 0x7f8a48104eb0 2024-09-15T22:38:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.515+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a481a2440 con 0x7f8a48104eb0 2024-09-15T22:38:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.516+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a481092d0 con 0x7f8a48104eb0 2024-09-15T22:38:24.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.517+0000 7f8a457fa640 1 -- 172.21.15.16:0/3855382938 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f8a3c0046c0 con 0x7f8a48104eb0 2024-09-15T22:38:24.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.518+0000 7f8a457fa640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a300a32a0 0x7f8a300a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.518+0000 7f8a47fff640 1 -- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a300a32a0 msgr2=0x7f8a300a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:24.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.518+0000 7f8a457fa640 1 -- 172.21.15.16:0/3855382938 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f8a3c0d0060 con 0x7f8a48104eb0 2024-09-15T22:38:24.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.518+0000 7f8a47fff640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a300a32a0 0x7f8a300a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:38:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.523+0000 7f8a457fa640 1 -- 172.21.15.16:0/3855382938 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a3c09c9e0 con 0x7f8a48104eb0 2024-09-15T22:38:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.701+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8a48062e90 con 0x7f8a48104eb0 2024-09-15T22:38:24.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.701+0000 7f8a457fa640 1 -- 172.21.15.16:0/3855382938 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f8a3c09c2e0 con 0x7f8a48104eb0 2024-09-15T22:38:24.703 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a300a32a0 msgr2=0x7f8a300a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a300a32a0 0x7f8a300a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48104eb0 msgr2=0x7f8a4819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48104eb0 0x7f8a4819a600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00e9f0 tx=0x7f8a3c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 shutdown_connections 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f8a300a32a0 0x7f8a300a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a481077e0 0x7f8a481a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a48104eb0 0x7f8a4819a600 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 --2- 172.21.15.16:0/3855382938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a48102580 0x7f8a4819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 >> 172.21.15.16:0/3855382938 conn(0x7f8a480fc140 msgr2=0x7f8a48101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 shutdown_connections 2024-09-15T22:38:24.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.703+0000 7f8a4e949640 1 -- 172.21.15.16:0/3855382938 wait complete. 2024-09-15T22:38:24.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T22:38:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 -- 172.21.15.16:0/382896201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c741025e0 msgr2=0x7f3c74108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/382896201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c741025e0 0x7f3c74108e70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009ca0 tx=0x7f3c6c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 -- 172.21.15.16:0/382896201 shutdown_connections 2024-09-15T22:38:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/382896201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c741025e0 0x7f3c74108e70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/382896201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c74101c20 0x7f3c741020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/382896201 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c74100a30 0x7f3c74100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 -- 172.21.15.16:0/382896201 >> 172.21.15.16:0/382896201 conn(0x7f3c740fc1b0 msgr2=0x7f3c740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 -- 172.21.15.16:0/382896201 shutdown_connections 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 -- 172.21.15.16:0/382896201 wait complete. 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.828+0000 7f3c7aa42640 1 Processor -- start 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c7aa42640 1 -- start start 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c7aa42640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c74100a30 0x7f3c74197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c7aa42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c74101c20 0x7f3c74198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c7aa42640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c741025e0 0x7f3c7419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c7aa42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410d7c0 con 0x7f3c74101c20 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c7aa42640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410d640 con 0x7f3c741025e0 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c7aa42640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3c7410d940 con 0x7f3c74100a30 2024-09-15T22:38:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c74101c20 0x7f3c74198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c74101c20 0x7f3c74198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46656/0 (socket says 172.21.15.16:46656) 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 -- 172.21.15.16:0/1613428091 learned_addr learned my addr 172.21.15.16:0/1613428091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c78fdf640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c741025e0 0x7f3c7419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c73fff640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c74100a30 0x7f3c74197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 -- 172.21.15.16:0/1613428091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c74100a30 msgr2=0x7f3c74197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c74100a30 0x7f3c74197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 -- 172.21.15.16:0/1613428091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c741025e0 msgr2=0x7f3c7419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c741025e0 0x7f3c7419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 -- 172.21.15.16:0/1613428091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7419fa40 con 0x7f3c74101c20 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c73fff640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c74100a30 0x7f3c74197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c6bfff640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c74101c20 0x7f3c74198360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00e9f0 tx=0x7f3c5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.829+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00cd80 con 0x7f3c74101c20 2024-09-15T22:38:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.830+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7419fd30 con 0x7f3c74101c20 2024-09-15T22:38:24.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.830+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5c010910 con 0x7f3c74101c20 2024-09-15T22:38:24.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.830+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c018990 con 0x7f3c74101c20 2024-09-15T22:38:24.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.830+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c741a0220 con 0x7f3c74101c20 2024-09-15T22:38:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.831+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c74107e40 con 0x7f3c74101c20 2024-09-15T22:38:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.832+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 138372+0+0 (secure 0 0 0) 0x7f3c5c010ac0 con 0x7f3c74101c20 2024-09-15T22:38:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.832+0000 7f3c71ffb640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3c600a32a0 0x7f3c600a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.833+0000 7f3c73fff640 1 -- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3c600a32a0 msgr2=0x7f3c600a5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.16:6800/81475521 2024-09-15T22:38:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.833+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9121+0+0 (secure 0 0 0) 0x7f3c5c0d0530 con 0x7f3c74101c20 2024-09-15T22:38:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.833+0000 7f3c73fff640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3c600a32a0 0x7f3c600a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:38:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:24.836+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c5c09ced0 con 0x7f3c74101c20 2024-09-15T22:38:24.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1182045674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-15T22:38:24.999 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3478098694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4076557669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: Standby manager daemon z restarted 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: Standby manager daemon z started 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:25.000 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3855382938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.012+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f3c5c098e40 con 0x7f3c74101c20 2024-09-15T22:38:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.013+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3c600a32a0 msgr2=0x7f3c600a5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:38:25.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.013+0000 7f3c71ffb640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3c600a32a0 0x7f3c600a5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.022+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3c7402da10 con 0x7f3c74101c20 2024-09-15T22:38:25.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.022+0000 7f3c71ffb640 1 -- 172.21.15.16:0/1613428091 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f3c5c056490 con 0x7f3c74101c20 2024-09-15T22:38:25.024 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-15T22:38:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c74101c20 msgr2=0x7f3c74198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c74101c20 0x7f3c74198360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00e9f0 tx=0x7f3c5c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 shutdown_connections 2024-09-15T22:38:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:6800/81475521,v1:172.21.15.16:6801/81475521] conn(0x7f3c600a32a0 0x7f3c600a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c741025e0 0x7f3c7419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c74101c20 0x7f3c74198360 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 --2- 172.21.15.16:0/1613428091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c74100a30 0x7f3c74197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 >> 172.21.15.16:0/1613428091 conn(0x7f3c740fc1b0 msgr2=0x7f3c74105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 shutdown_connections 2024-09-15T22:38:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.024+0000 7f3c7aa42640 1 -- 172.21.15.16:0/1613428091 wait complete. 2024-09-15T22:38:25.028 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z[37550]: 2024-09-15T22:38:24.540+0000 7f80cd6ac040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:38:25.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1182045674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3478098694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4076557669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: Standby manager daemon z restarted 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: Standby manager daemon z started 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:25.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:24 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3855382938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y[36243]: 2024-09-15T22:38:24.723+0000 7fac2f3bb040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1182045674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3478098694' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4076557669' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: Standby manager daemon z restarted 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: Standby manager daemon z started 2024-09-15T22:38:25.095 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:38:25.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:25.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:38:25.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.130:0/2719935727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:25.096 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:24 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3855382938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.175+0000 7f6465b49640 1 -- 172.21.15.16:0/3591801537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6460102d10 msgr2=0x7f6460103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.175+0000 7f6465b49640 1 --2- 172.21.15.16:0/3591801537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6460102d10 0x7f6460103190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6448009860 tx=0x7f644802a340 comp rx=0 tx=0).stop 2024-09-15T22:38:25.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.177+0000 7f6465b49640 1 -- 172.21.15.16:0/3591801537 shutdown_connections 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.177+0000 7f6465b49640 1 --2- 172.21.15.16:0/3591801537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64601036d0 0x7f6460109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.177+0000 7f6465b49640 1 --2- 172.21.15.16:0/3591801537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6460102d10 0x7f6460103190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.177+0000 7f6465b49640 1 --2- 172.21.15.16:0/3591801537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6460101b20 0x7f6460101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.177+0000 7f6465b49640 1 -- 172.21.15.16:0/3591801537 >> 172.21.15.16:0/3591801537 conn(0x7f64600fd2e0 msgr2=0x7f64600ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:25.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.177+0000 7f6465b49640 1 -- 172.21.15.16:0/3591801537 shutdown_connections 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.177+0000 7f6465b49640 1 -- 172.21.15.16:0/3591801537 wait complete. 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.178+0000 7f6465b49640 1 Processor -- start 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.178+0000 7f6465b49640 1 -- start start 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.178+0000 7f6465b49640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6460101b20 0x7f6460110f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.178+0000 7f6465b49640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6460102d10 0x7f64601114b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.178+0000 7f6465b49640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64601036d0 0x7f64601ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.178+0000 7f6465b49640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6460076100 con 0x7f6460102d10 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f6465b49640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6460075f80 con 0x7f64601036d0 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f6465b49640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6460076280 con 0x7f6460101b20 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f6456ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6460102d10 0x7f64601114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f645ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64601036d0 0x7f64601ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f6456ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6460102d10 0x7f64601114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46754/0 (socket says 172.21.15.16:46754) 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f6456ffd640 1 -- 172.21.15.16:0/3440207813 learned_addr learned my addr 172.21.15.16:0/3440207813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:25.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f645f7fe640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6460101b20 0x7f6460110f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.179+0000 7f645ffff640 1 -- 172.21.15.16:0/3440207813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6460101b20 msgr2=0x7f6460110f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645ffff640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6460101b20 0x7f6460110f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645ffff640 1 -- 172.21.15.16:0/3440207813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6460102d10 msgr2=0x7f64601114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645ffff640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6460102d10 0x7f64601114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645ffff640 1 -- 172.21.15.16:0/3440207813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646006bc10 con 0x7f64601036d0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645ffff640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64601036d0 0x7f64601ae3e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f645800ec50 tx=0x7f645800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645d7fa640 1 -- 172.21.15.16:0/3440207813 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458003b70 con 0x7f64601036d0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645d7fa640 1 -- 172.21.15.16:0/3440207813 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6458003d10 con 0x7f64601036d0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.180+0000 7f645d7fa640 1 -- 172.21.15.16:0/3440207813 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458010970 con 0x7f64601036d0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.181+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646006bea0 con 0x7f64601036d0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.181+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646006c370 con 0x7f64601036d0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.183+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6460101f20 con 0x7f64601036d0 2024-09-15T22:38:25.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.184+0000 7f645d7fa640 1 -- 172.21.15.16:0/3440207813 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f6458003ec0 con 0x7f64601036d0 2024-09-15T22:38:25.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.186+0000 7f645d7fa640 1 -- 172.21.15.16:0/3440207813 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f64580cfcd0 con 0x7f64601036d0 2024-09-15T22:38:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.189+0000 7f645d7fa640 1 -- 172.21.15.16:0/3440207813 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64580d5020 con 0x7f64601036d0 2024-09-15T22:38:25.301 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:24 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:24.997+0000 7f4bb148c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:38:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.503+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6460062df0 con 0x7f64601036d0 2024-09-15T22:38:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.504+0000 7f645d7fa640 1 -- 172.21.15.16:0/3440207813 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f645809c910 con 0x7f64601036d0 2024-09-15T22:38:25.506 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64601036d0 msgr2=0x7f64601ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64601036d0 0x7f64601ae3e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f645800ec50 tx=0x7f645800c6a0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 shutdown_connections 2024-09-15T22:38:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64601036d0 0x7f64601ae3e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6460102d10 0x7f64601114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 --2- 172.21.15.16:0/3440207813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6460101b20 0x7f6460110f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 >> 172.21.15.16:0/3440207813 conn(0x7f64600fd2e0 msgr2=0x7f6460105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 shutdown_connections 2024-09-15T22:38:25.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:25.507+0000 7f6465b49640 1 -- 172.21.15.16:0/3440207813 wait complete. 2024-09-15T22:38:25.519 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-15T22:38:25.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b18a5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b18a5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b18a5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b18a5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b18a5e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b1ba64640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b1ba64640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.557 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b1ba64640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.558 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b1ba64640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.558 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:25.299+0000 7f4b1ba64640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:38:25.802 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:25.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-15T22:38:25.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: Standby manager daemon y restarted 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: Standby manager daemon y started 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: Active manager daemon x restarted 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: Activating manager daemon x 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: mgrmap e21: x(active, starting, since 0.0131259s), standbys: z, y 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:38:25.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:25.946 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:38:25.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1613428091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: Manager daemon x is now available 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:38:25.948 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:25 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3440207813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: Standby manager daemon y restarted 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: Standby manager daemon y started 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:26.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: Active manager daemon x restarted 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: Activating manager daemon x 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: mgrmap e21: x(active, starting, since 0.0131259s), standbys: z, y 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:38:26.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:38:26.031 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1613428091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: Manager daemon x is now available 2024-09-15T22:38:26.032 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:26.033 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:38:26.033 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:38:26.033 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:25 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3440207813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: Standby manager daemon y restarted 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: Standby manager daemon y started 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.? 172.21.15.59:0/606227263' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: Active manager daemon x restarted 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: Activating manager daemon x 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:38:26.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: mgrmap e21: x(active, starting, since 0.0131259s), standbys: z, y 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:38:26.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:26.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:38:26.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:38:26.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1613428091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:38:26.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: Manager daemon x is now available 2024-09-15T22:38:26.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:26.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:38:26.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:38:26.061 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:25 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3440207813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 -- 172.21.15.16:0/2253648445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400101df0 msgr2=0x7f64001021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 --2- 172.21.15.16:0/2253648445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400101df0 0x7f64001021d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009900 tx=0x7f63f002a410 comp rx=0 tx=0).stop 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 -- 172.21.15.16:0/2253648445 shutdown_connections 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 --2- 172.21.15.16:0/2253648445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400106f20 0x7f640010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 --2- 172.21.15.16:0/2253648445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 0x7f64001067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 --2- 172.21.15.16:0/2253648445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400101df0 0x7f64001021d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 -- 172.21.15.16:0/2253648445 >> 172.21.15.16:0/2253648445 conn(0x7f64000fd8f0 msgr2=0x7f64000ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 -- 172.21.15.16:0/2253648445 shutdown_connections 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.278+0000 7f640497a640 1 -- 172.21.15.16:0/2253648445 wait complete. 2024-09-15T22:38:26.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 Processor -- start 2024-09-15T22:38:26.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 -- start start 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400101df0 0x7f6400198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 0x7f6400198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400106f20 0x7f640019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111f80 con 0x7f6400106f20 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6400111e00 con 0x7f6400101df0 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f640497a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6400112100 con 0x7f64001027a0 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 0x7f6400198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fe59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400101df0 0x7f6400198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 0x7f6400198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:60664/0 (socket says 172.21.15.16:60664) 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 -- 172.21.15.16:0/282919501 learned_addr learned my addr 172.21.15.16:0/282919501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fed9d640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400106f20 0x7f640019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 -- 172.21.15.16:0/282919501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400101df0 msgr2=0x7f6400198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400101df0 0x7f6400198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 -- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400106f20 msgr2=0x7f640019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400106f20 0x7f640019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.279+0000 7f63fdd9b640 1 -- 172.21.15.16:0/282919501 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640019d3f0 con 0x7f64001027a0 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.280+0000 7f63fdd9b640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 0x7f6400198980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63f400e9f0 tx=0x7f63f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.280+0000 7f63ef7fe640 1 -- 172.21.15.16:0/282919501 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f400cd80 con 0x7f64001027a0 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.280+0000 7f63ef7fe640 1 -- 172.21.15.16:0/282919501 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f4010910 con 0x7f64001027a0 2024-09-15T22:38:26.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.280+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640019d6e0 con 0x7f64001027a0 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.281+0000 7f63ef7fe640 1 -- 172.21.15.16:0/282919501 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4018990 con 0x7f64001027a0 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.281+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001a4fc0 con 0x7f64001027a0 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.282+0000 7f63ef7fe640 1 -- 172.21.15.16:0/282919501 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f63f4018b30 con 0x7f64001027a0 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.282+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64001032d0 con 0x7f64001027a0 2024-09-15T22:38:26.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.283+0000 7f63ef7fe640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63e80a2fa0 0x7f63e80a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.284+0000 7f63fe59c640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63e80a2fa0 0x7f63e80a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.284+0000 7f63ef7fe640 1 -- 172.21.15.16:0/282919501 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f63f40cfbe0 con 0x7f64001027a0 2024-09-15T22:38:26.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.284+0000 7f63fe59c640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63e80a2fa0 0x7f63e80a5460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009230 tx=0x7f63f00058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:26.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.287+0000 7f63ef7fe640 1 -- 172.21.15.16:0/282919501 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63f409c730 con 0x7f64001027a0 2024-09-15T22:38:26.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.564+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64001997a0 con 0x7f64001027a0 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.565+0000 7f63ef7fe640 1 -- 172.21.15.16:0/282919501 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f63f409c030 con 0x7f64001027a0 2024-09-15T22:38:26.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:38:26.567 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63e80a2fa0 msgr2=0x7f63e80a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63e80a2fa0 0x7f63e80a5460 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009230 tx=0x7f63f00058d0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 msgr2=0x7f6400198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 0x7f6400198980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63f400e9f0 tx=0x7f63f400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 shutdown_connections 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6400106f20 0x7f640019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63e80a2fa0 0x7f63e80a5460 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64001027a0 0x7f6400198980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 --2- 172.21.15.16:0/282919501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6400101df0 0x7f6400198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 >> 172.21.15.16:0/282919501 conn(0x7f64000fd8f0 msgr2=0x7f64000fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 shutdown_connections 2024-09-15T22:38:26.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:26.567+0000 7f640497a640 1 -- 172.21.15.16:0/282919501 wait complete. 2024-09-15T22:38:27.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:27 smithi130 ceph-mon[35742]: mgrmap e22: x(active, since 1.01921s), standbys: z, y 2024-09-15T22:38:27.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:27 smithi130 ceph-mon[35742]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:27.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:27 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/282919501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:27.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-mon[31492]: mgrmap e22: x(active, since 1.01921s), standbys: z, y 2024-09-15T22:38:27.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-mon[31492]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:27.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/282919501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:27.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:27.032+0000 7f4b04cb7640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:38:27] ENGINE Error in HTTPServer.serve 2024-09-15T22:38:27.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: Traceback (most recent call last): 2024-09-15T22:38:27.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:38:27.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._connections.run(self.expiration_interval) 2024-09-15T22:38:27.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._run(expiration_interval) 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: s = self.context.wrap_socket( 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: return self.sslsocket_class._create( 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self.do_handshake() 2024-09-15T22:38:27.308 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:38:27.309 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: self._sslobj.do_handshake() 2024-09-15T22:38:27.309 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:38:27.309 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:38:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:38:27.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:27 smithi059 ceph-mon[34872]: mgrmap e22: x(active, since 1.01921s), standbys: z, y 2024-09-15T22:38:27.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:27 smithi059 ceph-mon[34872]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:27.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:27 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/282919501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:27.446 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-15T22:38:27.446 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-15T22:38:27.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:38:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1646580004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac941077f0 msgr2=0x7fac94109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1646580004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac941077f0 0x7fac94109c60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fac8c009ca0 tx=0x7fac8c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1646580004 shutdown_connections 2024-09-15T22:38:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1646580004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac941077f0 0x7fac94109c60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1646580004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 0x7fac941072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1646580004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac94102590 0x7fac94104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1646580004 >> 172.21.15.16:0/1646580004 conn(0x7fac940fc170 msgr2=0x7fac940fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1646580004 shutdown_connections 2024-09-15T22:38:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.592+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1646580004 wait complete. 2024-09-15T22:38:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 Processor -- start 2024-09-15T22:38:27.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 -- start start 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac94102590 0x7fac9419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 0x7fac9419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac941077f0 0x7fac941a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac9410d7b0 con 0x7fac94102590 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac9410d630 con 0x7fac94104ec0 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.593+0000 7fac9bc7b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac9410d930 con 0x7fac941077f0 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 0x7fac9419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 0x7fac9419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:40628/0 (socket says 172.21.15.16:40628) 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 -- 172.21.15.16:0/1999540192 learned_addr learned my addr 172.21.15.16:0/1999540192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99a17640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac94102590 0x7fac9419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac9a218640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac941077f0 0x7fac941a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 -- 172.21.15.16:0/1999540192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac941077f0 msgr2=0x7fac941a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac941077f0 0x7fac941a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 -- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac94102590 msgr2=0x7fac9419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac94102590 0x7fac9419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 -- 172.21.15.16:0/1999540192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac941a1c90 con 0x7fac94104ec0 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac99216640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 0x7fac9419a5b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fac8400e9c0 tx=0x7fac8400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.594+0000 7fac8affd640 1 -- 172.21.15.16:0/1999540192 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8400cd80 con 0x7fac94104ec0 2024-09-15T22:38:27.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.595+0000 7fac8affd640 1 -- 172.21.15.16:0/1999540192 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac84005500 con 0x7fac94104ec0 2024-09-15T22:38:27.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.595+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac941a1f80 con 0x7fac94104ec0 2024-09-15T22:38:27.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.595+0000 7fac8affd640 1 -- 172.21.15.16:0/1999540192 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84010690 con 0x7fac94104ec0 2024-09-15T22:38:27.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.595+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac941a2490 con 0x7fac94104ec0 2024-09-15T22:38:27.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.596+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac941092e0 con 0x7fac94104ec0 2024-09-15T22:38:27.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.597+0000 7fac8affd640 1 -- 172.21.15.16:0/1999540192 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 23) ==== 138475+0+0 (secure 0 0 0) 0x7fac840046c0 con 0x7fac94104ec0 2024-09-15T22:38:27.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.598+0000 7fac8affd640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac800a3440 0x7fac800a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.598+0000 7fac99a17640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac800a3440 0x7fac800a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.598+0000 7fac8affd640 1 -- 172.21.15.16:0/1999540192 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fac840d0700 con 0x7fac94104ec0 2024-09-15T22:38:27.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.598+0000 7fac99a17640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac800a3440 0x7fac800a5900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac90005fd0 tx=0x7fac90005850 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:27.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.601+0000 7fac8affd640 1 -- 172.21.15.16:0/1999540192 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac8409cd20 con 0x7fac94104ec0 2024-09-15T22:38:27.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.778+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac94062630 con 0x7fac800a3440 2024-09-15T22:38:27.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.787+0000 7fac8affd640 1 -- 172.21.15.16:0/1999540192 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339380 (secure 0 0 0) 0x7fac94062630 con 0x7fac800a3440 2024-09-15T22:38:27.793 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:27.795 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:38:27.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac800a3440 msgr2=0x7fac800a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac800a3440 0x7fac800a5900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac90005fd0 tx=0x7fac90005850 comp rx=0 tx=0).stop 2024-09-15T22:38:27.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 msgr2=0x7fac9419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 0x7fac9419a5b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fac8400e9c0 tx=0x7fac8400ee80 comp rx=0 tx=0).stop 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 shutdown_connections 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac800a3440 0x7fac800a5900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac941077f0 0x7fac941a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac94104ec0 0x7fac9419a5b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 --2- 172.21.15.16:0/1999540192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac94102590 0x7fac9419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 >> 172.21.15.16:0/1999540192 conn(0x7fac940fc170 msgr2=0x7fac94101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 shutdown_connections 2024-09-15T22:38:27.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.791+0000 7fac9bc7b640 1 -- 172.21.15.16:0/1999540192 wait complete. 2024-09-15T22:38:27.803 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-15T22:38:27.019577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190377,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":866,"num_read_kb":595,"num_write":481,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414860,"kb_used_data":15656,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124526964,"statfs":{"total":1151940427776,"available":1151515611136,"internally_reserved":0,"allocated":16031744,"data_stored":9588665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992200"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108773+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:38:25.108773+0000","last_peered":"2024-09-15T22:38:25.108773+0000","last_clean":"2024-09-15T22:38:25.108773+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:38:25.108773+0000","last_undegraded":"2024-09-15T22:38:25.108773+0000","last_fullsized":"2024-09-15T22:38:25.108773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114924+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:38:25.114924+0000","last_peered":"2024-09-15T22:38:25.114924+0000","last_clean":"2024-09-15T22:38:25.114924+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:38:25.114924+0000","last_undegraded":"2024-09-15T22:38:25.114924+0000","last_fullsized":"2024-09-15T22:38:25.114924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241365+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:38:25.241365+0000","last_peered":"2024-09-15T22:38:25.241365+0000","last_clean":"2024-09-15T22:38:25.241365+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:38:25.241365+0000","last_undegraded":"2024-09-15T22:38:25.241365+0000","last_fullsized":"2024-09-15T22:38:25.241365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115300+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:38:25.115300+0000","last_peered":"2024-09-15T22:38:25.115300+0000","last_clean":"2024-09-15T22:38:25.115300+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:38:25.115300+0000","last_undegraded":"2024-09-15T22:38:25.115300+0000","last_fullsized":"2024-09-15T22:38:25.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250490+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:38:25.250490+0000","last_peered":"2024-09-15T22:38:25.250490+0000","last_clean":"2024-09-15T22:38:25.250490+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:38:25.250490+0000","last_undegraded":"2024-09-15T22:38:25.250490+0000","last_fullsized":"2024-09-15T22:38:25.250490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240846+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:38:25.240846+0000","last_peered":"2024-09-15T22:38:25.240846+0000","last_clean":"2024-09-15T22:38:25.240846+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:38:25.240846+0000","last_undegraded":"2024-09-15T22:38:25.240846+0000","last_fullsized":"2024-09-15T22:38:25.240846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112496+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:38:25.112496+0000","last_peered":"2024-09-15T22:38:25.112496+0000","last_clean":"2024-09-15T22:38:25.112496+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:38:25.112496+0000","last_undegraded":"2024-09-15T22:38:25.112496+0000","last_fullsized":"2024-09-15T22:38:25.112496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241710+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:38:25.241710+0000","last_peered":"2024-09-15T22:38:25.241710+0000","last_clean":"2024-09-15T22:38:25.241710+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:38:25.241710+0000","last_undegraded":"2024-09-15T22:38:25.241710+0000","last_fullsized":"2024-09-15T22:38:25.241710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241001+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:38:25.241001+0000","last_peered":"2024-09-15T22:38:25.241001+0000","last_clean":"2024-09-15T22:38:25.241001+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:38:25.241001+0000","last_undegraded":"2024-09-15T22:38:25.241001+0000","last_fullsized":"2024-09-15T22:38:25.241001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112329+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:38:25.112329+0000","last_peered":"2024-09-15T22:38:25.112329+0000","last_clean":"2024-09-15T22:38:25.112329+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:38:25.112329+0000","last_undegraded":"2024-09-15T22:38:25.112329+0000","last_fullsized":"2024-09-15T22:38:25.112329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009573+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:38:25.009573+0000","last_peered":"2024-09-15T22:38:25.009573+0000","last_clean":"2024-09-15T22:38:25.009573+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:38:25.009573+0000","last_undegraded":"2024-09-15T22:38:25.009573+0000","last_fullsized":"2024-09-15T22:38:25.009573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240840+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:38:25.240840+0000","last_peered":"2024-09-15T22:38:25.240840+0000","last_clean":"2024-09-15T22:38:25.240840+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:38:25.240840+0000","last_undegraded":"2024-09-15T22:38:25.240840+0000","last_fullsized":"2024-09-15T22:38:25.240840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010739+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:38:25.010739+0000","last_peered":"2024-09-15T22:38:25.010739+0000","last_clean":"2024-09-15T22:38:25.010739+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:38:25.010739+0000","last_undegraded":"2024-09-15T22:38:25.010739+0000","last_fullsized":"2024-09-15T22:38:25.010739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241387+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:38:25.241387+0000","last_peered":"2024-09-15T22:38:25.241387+0000","last_clean":"2024-09-15T22:38:25.241387+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:38:25.241387+0000","last_undegraded":"2024-09-15T22:38:25.241387+0000","last_fullsized":"2024-09-15T22:38:25.241387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241192+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:38:25.241192+0000","last_peered":"2024-09-15T22:38:25.241192+0000","last_clean":"2024-09-15T22:38:25.241192+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:38:25.241192+0000","last_undegraded":"2024-09-15T22:38:25.241192+0000","last_fullsized":"2024-09-15T22:38:25.241192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113745+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:38:25.113745+0000","last_peered":"2024-09-15T22:38:25.113745+0000","last_clean":"2024-09-15T22:38:25.113745+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:38:25.113745+0000","last_undegraded":"2024-09-15T22:38:25.113745+0000","last_fullsized":"2024-09-15T22:38:25.113745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241575+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:38:25.241575+0000","last_peered":"2024-09-15T22:38:25.241575+0000","last_clean":"2024-09-15T22:38:25.241575+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:38:25.241575+0000","last_undegraded":"2024-09-15T22:38:25.241575+0000","last_fullsized":"2024-09-15T22:38:25.241575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241190+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:38:25.241190+0000","last_peered":"2024-09-15T22:38:25.241190+0000","last_clean":"2024-09-15T22:38:25.241190+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:38:25.241190+0000","last_undegraded":"2024-09-15T22:38:25.241190+0000","last_fullsized":"2024-09-15T22:38:25.241190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241234+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:38:25.241234+0000","last_peered":"2024-09-15T22:38:25.241234+0000","last_clean":"2024-09-15T22:38:25.241234+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:38:25.241234+0000","last_undegraded":"2024-09-15T22:38:25.241234+0000","last_fullsized":"2024-09-15T22:38:25.241234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114246+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:38:25.114246+0000","last_peered":"2024-09-15T22:38:25.114246+0000","last_clean":"2024-09-15T22:38:25.114246+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:38:25.114246+0000","last_undegraded":"2024-09-15T22:38:25.114246+0000","last_fullsized":"2024-09-15T22:38:25.114246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108613+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:38:25.108613+0000","last_peered":"2024-09-15T22:38:25.108613+0000","last_clean":"2024-09-15T22:38:25.108613+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:38:25.108613+0000","last_undegraded":"2024-09-15T22:38:25.108613+0000","last_fullsized":"2024-09-15T22:38:25.108613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112056+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:38:25.112056+0000","last_peered":"2024-09-15T22:38:25.112056+0000","last_clean":"2024-09-15T22:38:25.112056+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:38:25.112056+0000","last_undegraded":"2024-09-15T22:38:25.112056+0000","last_fullsized":"2024-09-15T22:38:25.112056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241262+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:38:25.241262+0000","last_peered":"2024-09-15T22:38:25.241262+0000","last_clean":"2024-09-15T22:38:25.241262+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:38:25.241262+0000","last_undegraded":"2024-09-15T22:38:25.241262+0000","last_fullsized":"2024-09-15T22:38:25.241262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241248+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:38:25.241248+0000","last_peered":"2024-09-15T22:38:25.241248+0000","last_clean":"2024-09-15T22:38:25.241248+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:38:25.241248+0000","last_undegraded":"2024-09-15T22:38:25.241248+0000","last_fullsized":"2024-09-15T22:38:25.241248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249966+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:38:25.249966+0000","last_peered":"2024-09-15T22:38:25.249966+0000","last_clean":"2024-09-15T22:38:25.249966+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:38:25.249966+0000","last_undegraded":"2024-09-15T22:38:25.249966+0000","last_fullsized":"2024-09-15T22:38:25.249966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241375+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:38:25.241375+0000","last_peered":"2024-09-15T22:38:25.241375+0000","last_clean":"2024-09-15T22:38:25.241375+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:38:25.241375+0000","last_undegraded":"2024-09-15T22:38:25.241375+0000","last_fullsized":"2024-09-15T22:38:25.241375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241418+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:38:25.241418+0000","last_peered":"2024-09-15T22:38:25.241418+0000","last_clean":"2024-09-15T22:38:25.241418+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:38:25.241418+0000","last_undegraded":"2024-09-15T22:38:25.241418+0000","last_fullsized":"2024-09-15T22:38:25.241418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115826+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:38:25.115826+0000","last_peered":"2024-09-15T22:38:25.115826+0000","last_clean":"2024-09-15T22:38:25.115826+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:38:25.115826+0000","last_undegraded":"2024-09-15T22:38:25.115826+0000","last_fullsized":"2024-09-15T22:38:25.115826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'48","reported_seq":112,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.239768+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:38:25.239768+0000","last_peered":"2024-09-15T22:38:25.239768+0000","last_clean":"2024-09-15T22:38:25.239768+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:38:25.239768+0000","last_undegraded":"2024-09-15T22:38:25.239768+0000","last_fullsized":"2024-09-15T22:38:25.239768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.111983+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:38:25.111983+0000","last_peered":"2024-09-15T22:38:25.111983+0000","last_clean":"2024-09-15T22:38:25.111983+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:38:25.111983+0000","last_undegraded":"2024-09-15T22:38:25.111983+0000","last_fullsized":"2024-09-15T22:38:25.111983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006514+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:38:25.006514+0000","last_peered":"2024-09-15T22:38:25.006514+0000","last_clean":"2024-09-15T22:38:25.006514+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:38:25.006514+0000","last_undegraded":"2024-09-15T22:38:25.006514+0000","last_fullsized":"2024-09-15T22:38:25.006514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112492+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:38:25.112492+0000","last_peered":"2024-09-15T22:38:25.112492+0000","last_clean":"2024-09-15T22:38:25.112492+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:38:25.112492+0000","last_undegraded":"2024-09-15T22:38:25.112492+0000","last_fullsized":"2024-09-15T22:38:25.112492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241541+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:38:25.241541+0000","last_peered":"2024-09-15T22:38:25.241541+0000","last_clean":"2024-09-15T22:38:25.241541+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:38:25.241541+0000","last_undegraded":"2024-09-15T22:38:25.241541+0000","last_fullsized":"2024-09-15T22:38:25.241541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124170+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:38:25.124170+0000","last_peered":"2024-09-15T22:38:25.124170+0000","last_clean":"2024-09-15T22:38:25.124170+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:38:25.124170+0000","last_undegraded":"2024-09-15T22:38:25.124170+0000","last_fullsized":"2024-09-15T22:38:25.124170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.110438+0000","last_change":"2024-09-15T22:38:14.242090+0000","last_active":"2024-09-15T22:38:25.110438+0000","last_peered":"2024-09-15T22:38:25.110438+0000","last_clean":"2024-09-15T22:38:25.110438+0000","last_became_active":"2024-09-15T22:38:14.241634+0000","last_became_peered":"2024-09-15T22:38:14.241634+0000","last_unstale":"2024-09-15T22:38:25.110438+0000","last_undegraded":"2024-09-15T22:38:25.110438+0000","last_fullsized":"2024-09-15T22:38:25.110438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:13.485506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112062+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:38:25.112062+0000","last_peered":"2024-09-15T22:38:25.112062+0000","last_clean":"2024-09-15T22:38:25.112062+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:38:25.112062+0000","last_undegraded":"2024-09-15T22:38:25.112062+0000","last_fullsized":"2024-09-15T22:38:25.112062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124297+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:38:25.124297+0000","last_peered":"2024-09-15T22:38:25.124297+0000","last_clean":"2024-09-15T22:38:25.124297+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:38:25.124297+0000","last_undegraded":"2024-09-15T22:38:25.124297+0000","last_fullsized":"2024-09-15T22:38:25.124297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010822+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:38:25.010822+0000","last_peered":"2024-09-15T22:38:25.010822+0000","last_clean":"2024-09-15T22:38:25.010822+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:38:25.010822+0000","last_undegraded":"2024-09-15T22:38:25.010822+0000","last_fullsized":"2024-09-15T22:38:25.010822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006453+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:38:25.006453+0000","last_peered":"2024-09-15T22:38:25.006453+0000","last_clean":"2024-09-15T22:38:25.006453+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:38:25.006453+0000","last_undegraded":"2024-09-15T22:38:25.006453+0000","last_fullsized":"2024-09-15T22:38:25.006453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009249+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:38:25.009249+0000","last_peered":"2024-09-15T22:38:25.009249+0000","last_clean":"2024-09-15T22:38:25.009249+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:38:25.009249+0000","last_undegraded":"2024-09-15T22:38:25.009249+0000","last_fullsized":"2024-09-15T22:38:25.009249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006157+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:38:25.006157+0000","last_peered":"2024-09-15T22:38:25.006157+0000","last_clean":"2024-09-15T22:38:25.006157+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:38:25.006157+0000","last_undegraded":"2024-09-15T22:38:25.006157+0000","last_fullsized":"2024-09-15T22:38:25.006157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006955+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:38:25.006955+0000","last_peered":"2024-09-15T22:38:25.006955+0000","last_clean":"2024-09-15T22:38:25.006955+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:38:25.006955+0000","last_undegraded":"2024-09-15T22:38:25.006955+0000","last_fullsized":"2024-09-15T22:38:25.006955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005813+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:38:25.005813+0000","last_peered":"2024-09-15T22:38:25.005813+0000","last_clean":"2024-09-15T22:38:25.005813+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:38:25.005813+0000","last_undegraded":"2024-09-15T22:38:25.005813+0000","last_fullsized":"2024-09-15T22:38:25.005813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010164+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:38:25.010164+0000","last_peered":"2024-09-15T22:38:25.010164+0000","last_clean":"2024-09-15T22:38:25.010164+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:38:25.010164+0000","last_undegraded":"2024-09-15T22:38:25.010164+0000","last_fullsized":"2024-09-15T22:38:25.010164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009598+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:38:25.009598+0000","last_peered":"2024-09-15T22:38:25.009598+0000","last_clean":"2024-09-15T22:38:25.009598+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:38:25.009598+0000","last_undegraded":"2024-09-15T22:38:25.009598+0000","last_fullsized":"2024-09-15T22:38:25.009598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113256+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:38:25.113256+0000","last_peered":"2024-09-15T22:38:25.113256+0000","last_clean":"2024-09-15T22:38:25.113256+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:38:25.113256+0000","last_undegraded":"2024-09-15T22:38:25.113256+0000","last_fullsized":"2024-09-15T22:38:25.113256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241238+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:38:25.241238+0000","last_peered":"2024-09-15T22:38:25.241238+0000","last_clean":"2024-09-15T22:38:25.241238+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:38:25.241238+0000","last_undegraded":"2024-09-15T22:38:25.241238+0000","last_fullsized":"2024-09-15T22:38:25.241238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010362+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:38:25.010362+0000","last_peered":"2024-09-15T22:38:25.010362+0000","last_clean":"2024-09-15T22:38:25.010362+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:38:25.010362+0000","last_undegraded":"2024-09-15T22:38:25.010362+0000","last_fullsized":"2024-09-15T22:38:25.010362+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010316+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:38:25.010316+0000","last_peered":"2024-09-15T22:38:25.010316+0000","last_clean":"2024-09-15T22:38:25.010316+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:38:25.010316+0000","last_undegraded":"2024-09-15T22:38:25.010316+0000","last_fullsized":"2024-09-15T22:38:25.010316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010530+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:38:25.010530+0000","last_peered":"2024-09-15T22:38:25.010530+0000","last_clean":"2024-09-15T22:38:25.010530+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:38:25.010530+0000","last_undegraded":"2024-09-15T22:38:25.010530+0000","last_fullsized":"2024-09-15T22:38:25.010530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241899+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:38:25.241899+0000","last_peered":"2024-09-15T22:38:25.241899+0000","last_clean":"2024-09-15T22:38:25.241899+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:38:25.241899+0000","last_undegraded":"2024-09-15T22:38:25.241899+0000","last_fullsized":"2024-09-15T22:38:25.241899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241629+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:38:25.241629+0000","last_peered":"2024-09-15T22:38:25.241629+0000","last_clean":"2024-09-15T22:38:25.241629+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:38:25.241629+0000","last_undegraded":"2024-09-15T22:38:25.241629+0000","last_fullsized":"2024-09-15T22:38:25.241629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010356+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:38:25.010356+0000","last_peered":"2024-09-15T22:38:25.010356+0000","last_clean":"2024-09-15T22:38:25.010356+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:38:25.010356+0000","last_undegraded":"2024-09-15T22:38:25.010356+0000","last_fullsized":"2024-09-15T22:38:25.010356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242363+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:38:25.242363+0000","last_peered":"2024-09-15T22:38:25.242363+0000","last_clean":"2024-09-15T22:38:25.242363+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:38:25.242363+0000","last_undegraded":"2024-09-15T22:38:25.242363+0000","last_fullsized":"2024-09-15T22:38:25.242363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248973+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:38:25.248973+0000","last_peered":"2024-09-15T22:38:25.248973+0000","last_clean":"2024-09-15T22:38:25.248973+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:38:25.248973+0000","last_undegraded":"2024-09-15T22:38:25.248973+0000","last_fullsized":"2024-09-15T22:38:25.248973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010508+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:38:25.010508+0000","last_peered":"2024-09-15T22:38:25.010508+0000","last_clean":"2024-09-15T22:38:25.010508+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:38:25.010508+0000","last_undegraded":"2024-09-15T22:38:25.010508+0000","last_fullsized":"2024-09-15T22:38:25.010508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006902+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:38:25.006902+0000","last_peered":"2024-09-15T22:38:25.006902+0000","last_clean":"2024-09-15T22:38:25.006902+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:38:25.006902+0000","last_undegraded":"2024-09-15T22:38:25.006902+0000","last_fullsized":"2024-09-15T22:38:25.006902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009045+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:38:25.009045+0000","last_peered":"2024-09-15T22:38:25.009045+0000","last_clean":"2024-09-15T22:38:25.009045+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:38:25.009045+0000","last_undegraded":"2024-09-15T22:38:25.009045+0000","last_fullsized":"2024-09-15T22:38:25.009045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242337+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:38:25.242337+0000","last_peered":"2024-09-15T22:38:25.242337+0000","last_clean":"2024-09-15T22:38:25.242337+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:38:25.242337+0000","last_undegraded":"2024-09-15T22:38:25.242337+0000","last_fullsized":"2024-09-15T22:38:25.242337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.395057+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:38:25.395057+0000","last_peered":"2024-09-15T22:38:25.395057+0000","last_clean":"2024-09-15T22:38:25.395057+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:38:25.395057+0000","last_undegraded":"2024-09-15T22:38:25.395057+0000","last_fullsized":"2024-09-15T22:38:25.395057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009192+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:38:25.009192+0000","last_peered":"2024-09-15T22:38:25.009192+0000","last_clean":"2024-09-15T22:38:25.009192+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:38:25.009192+0000","last_undegraded":"2024-09-15T22:38:25.009192+0000","last_fullsized":"2024-09-15T22:38:25.009192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009375+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:38:25.009375+0000","last_peered":"2024-09-15T22:38:25.009375+0000","last_clean":"2024-09-15T22:38:25.009375+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:38:25.009375+0000","last_undegraded":"2024-09-15T22:38:25.009375+0000","last_fullsized":"2024-09-15T22:38:25.009375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112094+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:38:25.112094+0000","last_peered":"2024-09-15T22:38:25.112094+0000","last_clean":"2024-09-15T22:38:25.112094+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:38:25.112094+0000","last_undegraded":"2024-09-15T22:38:25.112094+0000","last_fullsized":"2024-09-15T22:38:25.112094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006770+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:38:25.006770+0000","last_peered":"2024-09-15T22:38:25.006770+0000","last_clean":"2024-09-15T22:38:25.006770+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:38:25.006770+0000","last_undegraded":"2024-09-15T22:38:25.006770+0000","last_fullsized":"2024-09-15T22:38:25.006770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010086+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:38:25.010086+0000","last_peered":"2024-09-15T22:38:25.010086+0000","last_clean":"2024-09-15T22:38:25.010086+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:38:25.010086+0000","last_undegraded":"2024-09-15T22:38:25.010086+0000","last_fullsized":"2024-09-15T22:38:25.010086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.539508+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:38:25.539508+0000","last_peered":"2024-09-15T22:38:25.539508+0000","last_clean":"2024-09-15T22:38:25.539508+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:38:25.539508+0000","last_undegraded":"2024-09-15T22:38:25.539508+0000","last_fullsized":"2024-09-15T22:38:25.539508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108636+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:38:25.108636+0000","last_peered":"2024-09-15T22:38:25.108636+0000","last_clean":"2024-09-15T22:38:25.108636+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:38:25.108636+0000","last_undegraded":"2024-09-15T22:38:25.108636+0000","last_fullsized":"2024-09-15T22:38:25.108636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010264+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:38:25.010264+0000","last_peered":"2024-09-15T22:38:25.010264+0000","last_clean":"2024-09-15T22:38:25.010264+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:38:25.010264+0000","last_undegraded":"2024-09-15T22:38:25.010264+0000","last_fullsized":"2024-09-15T22:38:25.010264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009541+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:38:25.009541+0000","last_peered":"2024-09-15T22:38:25.009541+0000","last_clean":"2024-09-15T22:38:25.009541+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:38:25.009541+0000","last_undegraded":"2024-09-15T22:38:25.009541+0000","last_fullsized":"2024-09-15T22:38:25.009541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241984+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:38:25.241984+0000","last_peered":"2024-09-15T22:38:25.241984+0000","last_clean":"2024-09-15T22:38:25.241984+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:38:25.241984+0000","last_undegraded":"2024-09-15T22:38:25.241984+0000","last_fullsized":"2024-09-15T22:38:25.241984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005995+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:38:25.005995+0000","last_peered":"2024-09-15T22:38:25.005995+0000","last_clean":"2024-09-15T22:38:25.005995+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:38:25.005995+0000","last_undegraded":"2024-09-15T22:38:25.005995+0000","last_fullsized":"2024-09-15T22:38:25.005995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241792+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:38:25.241792+0000","last_peered":"2024-09-15T22:38:25.241792+0000","last_clean":"2024-09-15T22:38:25.241792+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:38:25.241792+0000","last_undegraded":"2024-09-15T22:38:25.241792+0000","last_fullsized":"2024-09-15T22:38:25.241792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009484+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:38:25.009484+0000","last_peered":"2024-09-15T22:38:25.009484+0000","last_clean":"2024-09-15T22:38:25.009484+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:38:25.009484+0000","last_undegraded":"2024-09-15T22:38:25.009484+0000","last_fullsized":"2024-09-15T22:38:25.009484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112389+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:38:25.112389+0000","last_peered":"2024-09-15T22:38:25.112389+0000","last_clean":"2024-09-15T22:38:25.112389+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:38:25.112389+0000","last_undegraded":"2024-09-15T22:38:25.112389+0000","last_fullsized":"2024-09-15T22:38:25.112389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.537439+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:38:25.537439+0000","last_peered":"2024-09-15T22:38:25.537439+0000","last_clean":"2024-09-15T22:38:25.537439+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:38:25.537439+0000","last_undegraded":"2024-09-15T22:38:25.537439+0000","last_fullsized":"2024-09-15T22:38:25.537439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69721,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":214,"num_read_kb":175,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107742+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:38:25.107742+0000","last_peered":"2024-09-15T22:38:25.107742+0000","last_clean":"2024-09-15T22:38:25.107742+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:38:25.107742+0000","last_undegraded":"2024-09-15T22:38:25.107742+0000","last_fullsized":"2024-09-15T22:38:25.107742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241717+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:38:25.241717+0000","last_peered":"2024-09-15T22:38:25.241717+0000","last_clean":"2024-09-15T22:38:25.241717+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:38:25.241717+0000","last_undegraded":"2024-09-15T22:38:25.241717+0000","last_fullsized":"2024-09-15T22:38:25.241717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009805+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:38:25.009805+0000","last_peered":"2024-09-15T22:38:25.009805+0000","last_clean":"2024-09-15T22:38:25.009805+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:38:25.009805+0000","last_undegraded":"2024-09-15T22:38:25.009805+0000","last_fullsized":"2024-09-15T22:38:25.009805+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241127+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:38:25.241127+0000","last_peered":"2024-09-15T22:38:25.241127+0000","last_clean":"2024-09-15T22:38:25.241127+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:38:25.241127+0000","last_undegraded":"2024-09-15T22:38:25.241127+0000","last_fullsized":"2024-09-15T22:38:25.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250624+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:38:25.250624+0000","last_peered":"2024-09-15T22:38:25.250624+0000","last_clean":"2024-09-15T22:38:25.250624+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:38:25.250624+0000","last_undegraded":"2024-09-15T22:38:25.250624+0000","last_fullsized":"2024-09-15T22:38:25.250624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010393+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:38:25.010393+0000","last_peered":"2024-09-15T22:38:25.010393+0000","last_clean":"2024-09-15T22:38:25.010393+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:38:25.010393+0000","last_undegraded":"2024-09-15T22:38:25.010393+0000","last_fullsized":"2024-09-15T22:38:25.010393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107495+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:38:25.107495+0000","last_peered":"2024-09-15T22:38:25.107495+0000","last_clean":"2024-09-15T22:38:25.107495+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:38:25.107495+0000","last_undegraded":"2024-09-15T22:38:25.107495+0000","last_fullsized":"2024-09-15T22:38:25.107495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124524+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:38:25.124524+0000","last_peered":"2024-09-15T22:38:25.124524+0000","last_clean":"2024-09-15T22:38:25.124524+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:38:25.124524+0000","last_undegraded":"2024-09-15T22:38:25.124524+0000","last_fullsized":"2024-09-15T22:38:25.124524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112430+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:38:25.112430+0000","last_peered":"2024-09-15T22:38:25.112430+0000","last_clean":"2024-09-15T22:38:25.112430+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:38:25.112430+0000","last_undegraded":"2024-09-15T22:38:25.112430+0000","last_fullsized":"2024-09-15T22:38:25.112430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249732+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:38:25.249732+0000","last_peered":"2024-09-15T22:38:25.249732+0000","last_clean":"2024-09-15T22:38:25.249732+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:38:25.249732+0000","last_undegraded":"2024-09-15T22:38:25.249732+0000","last_fullsized":"2024-09-15T22:38:25.249732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107744+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:38:25.107744+0000","last_peered":"2024-09-15T22:38:25.107744+0000","last_clean":"2024-09-15T22:38:25.107744+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:38:25.107744+0000","last_undegraded":"2024-09-15T22:38:25.107744+0000","last_fullsized":"2024-09-15T22:38:25.107744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009479+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:38:25.009479+0000","last_peered":"2024-09-15T22:38:25.009479+0000","last_clean":"2024-09-15T22:38:25.009479+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:38:25.009479+0000","last_undegraded":"2024-09-15T22:38:25.009479+0000","last_fullsized":"2024-09-15T22:38:25.009479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242387+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:38:25.242387+0000","last_peered":"2024-09-15T22:38:25.242387+0000","last_clean":"2024-09-15T22:38:25.242387+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:38:25.242387+0000","last_undegraded":"2024-09-15T22:38:25.242387+0000","last_fullsized":"2024-09-15T22:38:25.242387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112126+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:38:25.112126+0000","last_peered":"2024-09-15T22:38:25.112126+0000","last_clean":"2024-09-15T22:38:25.112126+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:38:25.112126+0000","last_undegraded":"2024-09-15T22:38:25.112126+0000","last_fullsized":"2024-09-15T22:38:25.112126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005971+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:38:25.005971+0000","last_peered":"2024-09-15T22:38:25.005971+0000","last_clean":"2024-09-15T22:38:25.005971+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:38:25.005971+0000","last_undegraded":"2024-09-15T22:38:25.005971+0000","last_fullsized":"2024-09-15T22:38:25.005971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124202+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:38:25.124202+0000","last_peered":"2024-09-15T22:38:25.124202+0000","last_clean":"2024-09-15T22:38:25.124202+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:38:25.124202+0000","last_undegraded":"2024-09-15T22:38:25.124202+0000","last_fullsized":"2024-09-15T22:38:25.124202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009565+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:38:25.009565+0000","last_peered":"2024-09-15T22:38:25.009565+0000","last_clean":"2024-09-15T22:38:25.009565+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:38:25.009565+0000","last_undegraded":"2024-09-15T22:38:25.009565+0000","last_fullsized":"2024-09-15T22:38:25.009565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241674+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:38:25.241674+0000","last_peered":"2024-09-15T22:38:25.241674+0000","last_clean":"2024-09-15T22:38:25.241674+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:38:25.241674+0000","last_undegraded":"2024-09-15T22:38:25.241674+0000","last_fullsized":"2024-09-15T22:38:25.241674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009806+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:38:25.009806+0000","last_peered":"2024-09-15T22:38:25.009806+0000","last_clean":"2024-09-15T22:38:25.009806+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:38:25.009806+0000","last_undegraded":"2024-09-15T22:38:25.009806+0000","last_fullsized":"2024-09-15T22:38:25.009806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112093+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:38:25.112093+0000","last_peered":"2024-09-15T22:38:25.112093+0000","last_clean":"2024-09-15T22:38:25.112093+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:38:25.112093+0000","last_undegraded":"2024-09-15T22:38:25.112093+0000","last_fullsized":"2024-09-15T22:38:25.112093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241736+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:38:25.241736+0000","last_peered":"2024-09-15T22:38:25.241736+0000","last_clean":"2024-09-15T22:38:25.241736+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:38:25.241736+0000","last_undegraded":"2024-09-15T22:38:25.241736+0000","last_fullsized":"2024-09-15T22:38:25.241736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241163+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:38:25.241163+0000","last_peered":"2024-09-15T22:38:25.241163+0000","last_clean":"2024-09-15T22:38:25.241163+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:38:25.241163+0000","last_undegraded":"2024-09-15T22:38:25.241163+0000","last_fullsized":"2024-09-15T22:38:25.241163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241675+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:38:25.241675+0000","last_peered":"2024-09-15T22:38:25.241675+0000","last_clean":"2024-09-15T22:38:25.241675+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:38:25.241675+0000","last_undegraded":"2024-09-15T22:38:25.241675+0000","last_fullsized":"2024-09-15T22:38:25.241675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241517+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:38:25.241517+0000","last_peered":"2024-09-15T22:38:25.241517+0000","last_clean":"2024-09-15T22:38:25.241517+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:38:25.241517+0000","last_undegraded":"2024-09-15T22:38:25.241517+0000","last_fullsized":"2024-09-15T22:38:25.241517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249023+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:38:25.249023+0000","last_peered":"2024-09-15T22:38:25.249023+0000","last_clean":"2024-09-15T22:38:25.249023+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:38:25.249023+0000","last_undegraded":"2024-09-15T22:38:25.249023+0000","last_fullsized":"2024-09-15T22:38:25.249023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006558+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:38:25.006558+0000","last_peered":"2024-09-15T22:38:25.006558+0000","last_clean":"2024-09-15T22:38:25.006558+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:38:25.006558+0000","last_undegraded":"2024-09-15T22:38:25.006558+0000","last_fullsized":"2024-09-15T22:38:25.006558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242143+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:38:25.242143+0000","last_peered":"2024-09-15T22:38:25.242143+0000","last_clean":"2024-09-15T22:38:25.242143+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:38:25.242143+0000","last_undegraded":"2024-09-15T22:38:25.242143+0000","last_fullsized":"2024-09-15T22:38:25.242143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241544+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:38:25.241544+0000","last_peered":"2024-09-15T22:38:25.241544+0000","last_clean":"2024-09-15T22:38:25.241544+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:38:25.241544+0000","last_undegraded":"2024-09-15T22:38:25.241544+0000","last_fullsized":"2024-09-15T22:38:25.241544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115695+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:38:25.115695+0000","last_peered":"2024-09-15T22:38:25.115695+0000","last_clean":"2024-09-15T22:38:25.115695+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:38:25.115695+0000","last_undegraded":"2024-09-15T22:38:25.115695+0000","last_fullsized":"2024-09-15T22:38:25.115695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009267+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:38:25.009267+0000","last_peered":"2024-09-15T22:38:25.009267+0000","last_clean":"2024-09-15T22:38:25.009267+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:38:25.009267+0000","last_undegraded":"2024-09-15T22:38:25.009267+0000","last_fullsized":"2024-09-15T22:38:25.009267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112660+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:38:25.112660+0000","last_peered":"2024-09-15T22:38:25.112660+0000","last_clean":"2024-09-15T22:38:25.112660+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:38:25.112660+0000","last_undegraded":"2024-09-15T22:38:25.112660+0000","last_fullsized":"2024-09-15T22:38:25.112660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241756+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:38:25.241756+0000","last_peered":"2024-09-15T22:38:25.241756+0000","last_clean":"2024-09-15T22:38:25.241756+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:38:25.241756+0000","last_undegraded":"2024-09-15T22:38:25.241756+0000","last_fullsized":"2024-09-15T22:38:25.241756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006219+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:38:25.006219+0000","last_peered":"2024-09-15T22:38:25.006219+0000","last_clean":"2024-09-15T22:38:25.006219+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:38:25.006219+0000","last_undegraded":"2024-09-15T22:38:25.006219+0000","last_fullsized":"2024-09-15T22:38:25.006219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242055+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:38:25.242055+0000","last_peered":"2024-09-15T22:38:25.242055+0000","last_clean":"2024-09-15T22:38:25.242055+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:38:25.242055+0000","last_undegraded":"2024-09-15T22:38:25.242055+0000","last_fullsized":"2024-09-15T22:38:25.242055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241921+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:38:25.241921+0000","last_peered":"2024-09-15T22:38:25.241921+0000","last_clean":"2024-09-15T22:38:25.241921+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:38:25.241921+0000","last_undegraded":"2024-09-15T22:38:25.241921+0000","last_fullsized":"2024-09-15T22:38:25.241921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.109165+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:38:25.109165+0000","last_peered":"2024-09-15T22:38:25.109165+0000","last_clean":"2024-09-15T22:38:25.109165+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:38:25.109165+0000","last_undegraded":"2024-09-15T22:38:25.109165+0000","last_fullsized":"2024-09-15T22:38:25.109165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242550+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:38:25.242550+0000","last_peered":"2024-09-15T22:38:25.242550+0000","last_clean":"2024-09-15T22:38:25.242550+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:38:25.242550+0000","last_undegraded":"2024-09-15T22:38:25.242550+0000","last_fullsized":"2024-09-15T22:38:25.242550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241031+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:38:25.241031+0000","last_peered":"2024-09-15T22:38:25.241031+0000","last_clean":"2024-09-15T22:38:25.241031+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:38:25.241031+0000","last_undegraded":"2024-09-15T22:38:25.241031+0000","last_fullsized":"2024-09-15T22:38:25.241031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248274+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:38:25.248274+0000","last_peered":"2024-09-15T22:38:25.248274+0000","last_clean":"2024-09-15T22:38:25.248274+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:38:25.248274+0000","last_undegraded":"2024-09-15T22:38:25.248274+0000","last_fullsized":"2024-09-15T22:38:25.248274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248580+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:38:25.248580+0000","last_peered":"2024-09-15T22:38:25.248580+0000","last_clean":"2024-09-15T22:38:25.248580+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:38:25.248580+0000","last_undegraded":"2024-09-15T22:38:25.248580+0000","last_fullsized":"2024-09-15T22:38:25.248580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006155+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:38:25.006155+0000","last_peered":"2024-09-15T22:38:25.006155+0000","last_clean":"2024-09-15T22:38:25.006155+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:38:25.006155+0000","last_undegraded":"2024-09-15T22:38:25.006155+0000","last_fullsized":"2024-09-15T22:38:25.006155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241453+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:38:25.241453+0000","last_peered":"2024-09-15T22:38:25.241453+0000","last_clean":"2024-09-15T22:38:25.241453+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:38:25.241453+0000","last_undegraded":"2024-09-15T22:38:25.241453+0000","last_fullsized":"2024-09-15T22:38:25.241453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010699+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:38:25.010699+0000","last_peered":"2024-09-15T22:38:25.010699+0000","last_clean":"2024-09-15T22:38:25.010699+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:38:25.010699+0000","last_undegraded":"2024-09-15T22:38:25.010699+0000","last_fullsized":"2024-09-15T22:38:25.010699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124494+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:38:25.124494+0000","last_peered":"2024-09-15T22:38:25.124494+0000","last_clean":"2024-09-15T22:38:25.124494+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:38:25.124494+0000","last_undegraded":"2024-09-15T22:38:25.124494+0000","last_fullsized":"2024-09-15T22:38:25.124494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112097+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:38:25.112097+0000","last_peered":"2024-09-15T22:38:25.112097+0000","last_clean":"2024-09-15T22:38:25.112097+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:38:25.112097+0000","last_undegraded":"2024-09-15T22:38:25.112097+0000","last_fullsized":"2024-09-15T22:38:25.112097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010114+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:38:25.010114+0000","last_peered":"2024-09-15T22:38:25.010114+0000","last_clean":"2024-09-15T22:38:25.010114+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:38:25.010114+0000","last_undegraded":"2024-09-15T22:38:25.010114+0000","last_fullsized":"2024-09-15T22:38:25.010114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124146+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:38:25.124146+0000","last_peered":"2024-09-15T22:38:25.124146+0000","last_clean":"2024-09-15T22:38:25.124146+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:38:25.124146+0000","last_undegraded":"2024-09-15T22:38:25.124146+0000","last_fullsized":"2024-09-15T22:38:25.124146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112085+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:38:25.112085+0000","last_peered":"2024-09-15T22:38:25.112085+0000","last_clean":"2024-09-15T22:38:25.112085+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:38:25.112085+0000","last_undegraded":"2024-09-15T22:38:25.112085+0000","last_fullsized":"2024-09-15T22:38:25.112085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108245+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:38:25.108245+0000","last_peered":"2024-09-15T22:38:25.108245+0000","last_clean":"2024-09-15T22:38:25.108245+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:38:25.108245+0000","last_undegraded":"2024-09-15T22:38:25.108245+0000","last_fullsized":"2024-09-15T22:38:25.108245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249437+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:38:25.249437+0000","last_peered":"2024-09-15T22:38:25.249437+0000","last_clean":"2024-09-15T22:38:25.249437+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:38:25.249437+0000","last_undegraded":"2024-09-15T22:38:25.249437+0000","last_fullsized":"2024-09-15T22:38:25.249437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249830+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:38:25.249830+0000","last_peered":"2024-09-15T22:38:25.249830+0000","last_clean":"2024-09-15T22:38:25.249830+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:38:25.249830+0000","last_undegraded":"2024-09-15T22:38:25.249830+0000","last_fullsized":"2024-09-15T22:38:25.249830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242437+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:38:25.242437+0000","last_peered":"2024-09-15T22:38:25.242437+0000","last_clean":"2024-09-15T22:38:25.242437+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:38:25.242437+0000","last_undegraded":"2024-09-15T22:38:25.242437+0000","last_fullsized":"2024-09-15T22:38:25.242437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599775,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":730,"num_read_kb":476,"num_write":298,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1799325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":519982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841581,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28852,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716300,"statfs":{"total":95995035648,"available":95965491200,"internally_reserved":0,"allocated":1871872,"data_stored":1309665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005107,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36768,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708384,"statfs":{"total":95995035648,"available":95957385216,"internally_reserved":0,"allocated":1454080,"data_stored":917530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36572,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708580,"statfs":{"total":95995035648,"available":95957585920,"internally_reserved":0,"allocated":1253376,"data_stored":716746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1650688,"data_stored":1116498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":515394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855431,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1794048,"data_stored":1243378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":521250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1040404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":586797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":584239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":516782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:38:27.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:38:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 -- 172.21.15.16:0/943705227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 msgr2=0x7fa9f4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/943705227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f4100e30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009900 tx=0x7fa9e402a410 comp rx=0 tx=0).stop 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 -- 172.21.15.16:0/943705227 shutdown_connections 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/943705227 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9f41025e0 0x7fa9f4108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/943705227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f4101c20 0x7fa9f41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/943705227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f4100e30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 -- 172.21.15.16:0/943705227 >> 172.21.15.16:0/943705227 conn(0x7fa9f40fc190 msgr2=0x7fa9f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 -- 172.21.15.16:0/943705227 shutdown_connections 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 -- 172.21.15.16:0/943705227 wait complete. 2024-09-15T22:38:27.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.955+0000 7fa9f9af1640 1 Processor -- start 2024-09-15T22:38:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f9af1640 1 -- start start 2024-09-15T22:38:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f9af1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f406af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f9af1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9f4101c20 0x7fa9f419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f9af1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f41025e0 0x7fa9f419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f9af1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410c690 con 0x7fa9f4100a30 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f9af1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410c510 con 0x7fa9f41025e0 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f9af1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410c810 con 0x7fa9f4101c20 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f406af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f406af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46850/0 (socket says 172.21.15.16:46850) 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 -- 172.21.15.16:0/1274019006 learned_addr learned my addr 172.21.15.16:0/1274019006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f2ffd640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9f4101c20 0x7fa9f419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f3fff640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f41025e0 0x7fa9f419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 -- 172.21.15.16:0/1274019006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9f4101c20 msgr2=0x7fa9f419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9f4101c20 0x7fa9f419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 -- 172.21.15.16:0/1274019006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f41025e0 msgr2=0x7fa9f419f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f41025e0 0x7fa9f419f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 -- 172.21.15.16:0/1274019006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f419faf0 con 0x7fa9f4100a30 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f3fff640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f41025e0 0x7fa9f419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.956+0000 7fa9f37fe640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f406af10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006010 tx=0x7fa9e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.957+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/1274019006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e403f030 con 0x7fa9f4100a30 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.957+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/1274019006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e4003ce0 con 0x7fa9f4100a30 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.957+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f419fd80 con 0x7fa9f4100a30 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.957+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/1274019006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4035070 con 0x7fa9f4100a30 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.957+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a01c0 con 0x7fa9f4100a30 2024-09-15T22:38:27.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.958+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f4107ec0 con 0x7fa9f4100a30 2024-09-15T22:38:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.959+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/1274019006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138475+0+0 (secure 0 0 0) 0x7fa9e4042030 con 0x7fa9f4100a30 2024-09-15T22:38:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.960+0000 7fa9f0ff9640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9dc0a33c0 0x7fa9dc0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.960+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/1274019006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa9e40f02f0 con 0x7fa9f4100a30 2024-09-15T22:38:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.960+0000 7fa9f2ffd640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9dc0a33c0 0x7fa9dc0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.960+0000 7fa9f2ffd640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9dc0a33c0 0x7fa9dc0a5880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9f406ac80 tx=0x7fa9e8005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:27.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:27.964+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/1274019006 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9e40bc910 con 0x7fa9f4100a30 2024-09-15T22:38:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: [15/Sep/2024:22:38:26] ENGINE Bus STARTING 2024-09-15T22:38:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-15T22:38:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: [15/Sep/2024:22:38:27] ENGINE Serving on https://172.21.15.16:7150 2024-09-15T22:38:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: [15/Sep/2024:22:38:27] ENGINE Error in HTTPServer.serve 2024-09-15T22:38:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-15T22:38:28.057 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:38:28.058 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: [15/Sep/2024:22:38:27] ENGINE Serving on http://172.21.15.16:8765 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: [15/Sep/2024:22:38:27] ENGINE Bus STARTED 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi130 to 176.0M 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: Unable to set osd_memory_target on smithi130 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.060 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:28 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.159+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9f4062630 con 0x7fa9dc0a33c0 2024-09-15T22:38:28.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.172+0000 7fa9f0ff9640 1 -- 172.21.15.16:0/1274019006 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339380 (secure 0 0 0) 0x7fa9f4062630 con 0x7fa9dc0a33c0 2024-09-15T22:38:28.178 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:28.179 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9dc0a33c0 msgr2=0x7fa9dc0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9dc0a33c0 0x7fa9dc0a5880 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa9f406ac80 tx=0x7fa9e8005f90 comp rx=0 tx=0).stop 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 msgr2=0x7fa9f406af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f406af10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006010 tx=0x7fa9e40042f0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 shutdown_connections 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9f41025e0 0x7fa9f419f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9dc0a33c0 0x7fa9dc0a5880 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9f4101c20 0x7fa9f419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 --2- 172.21.15.16:0/1274019006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f4100a30 0x7fa9f406af10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 >> 172.21.15.16:0/1274019006 conn(0x7fa9f40fc190 msgr2=0x7fa9f4105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 shutdown_connections 2024-09-15T22:38:28.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.176+0000 7fa9f9af1640 1 -- 172.21.15.16:0/1274019006 wait complete. 2024-09-15T22:38:28.189 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-15T22:38:27.019577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190377,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":866,"num_read_kb":595,"num_write":481,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414860,"kb_used_data":15656,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124526964,"statfs":{"total":1151940427776,"available":1151515611136,"internally_reserved":0,"allocated":16031744,"data_stored":9588665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992200"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108773+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:38:25.108773+0000","last_peered":"2024-09-15T22:38:25.108773+0000","last_clean":"2024-09-15T22:38:25.108773+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:38:25.108773+0000","last_undegraded":"2024-09-15T22:38:25.108773+0000","last_fullsized":"2024-09-15T22:38:25.108773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114924+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:38:25.114924+0000","last_peered":"2024-09-15T22:38:25.114924+0000","last_clean":"2024-09-15T22:38:25.114924+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:38:25.114924+0000","last_undegraded":"2024-09-15T22:38:25.114924+0000","last_fullsized":"2024-09-15T22:38:25.114924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241365+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:38:25.241365+0000","last_peered":"2024-09-15T22:38:25.241365+0000","last_clean":"2024-09-15T22:38:25.241365+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:38:25.241365+0000","last_undegraded":"2024-09-15T22:38:25.241365+0000","last_fullsized":"2024-09-15T22:38:25.241365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115300+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:38:25.115300+0000","last_peered":"2024-09-15T22:38:25.115300+0000","last_clean":"2024-09-15T22:38:25.115300+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:38:25.115300+0000","last_undegraded":"2024-09-15T22:38:25.115300+0000","last_fullsized":"2024-09-15T22:38:25.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250490+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:38:25.250490+0000","last_peered":"2024-09-15T22:38:25.250490+0000","last_clean":"2024-09-15T22:38:25.250490+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:38:25.250490+0000","last_undegraded":"2024-09-15T22:38:25.250490+0000","last_fullsized":"2024-09-15T22:38:25.250490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240846+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:38:25.240846+0000","last_peered":"2024-09-15T22:38:25.240846+0000","last_clean":"2024-09-15T22:38:25.240846+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:38:25.240846+0000","last_undegraded":"2024-09-15T22:38:25.240846+0000","last_fullsized":"2024-09-15T22:38:25.240846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112496+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:38:25.112496+0000","last_peered":"2024-09-15T22:38:25.112496+0000","last_clean":"2024-09-15T22:38:25.112496+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:38:25.112496+0000","last_undegraded":"2024-09-15T22:38:25.112496+0000","last_fullsized":"2024-09-15T22:38:25.112496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241710+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:38:25.241710+0000","last_peered":"2024-09-15T22:38:25.241710+0000","last_clean":"2024-09-15T22:38:25.241710+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:38:25.241710+0000","last_undegraded":"2024-09-15T22:38:25.241710+0000","last_fullsized":"2024-09-15T22:38:25.241710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241001+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:38:25.241001+0000","last_peered":"2024-09-15T22:38:25.241001+0000","last_clean":"2024-09-15T22:38:25.241001+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:38:25.241001+0000","last_undegraded":"2024-09-15T22:38:25.241001+0000","last_fullsized":"2024-09-15T22:38:25.241001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112329+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:38:25.112329+0000","last_peered":"2024-09-15T22:38:25.112329+0000","last_clean":"2024-09-15T22:38:25.112329+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:38:25.112329+0000","last_undegraded":"2024-09-15T22:38:25.112329+0000","last_fullsized":"2024-09-15T22:38:25.112329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009573+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:38:25.009573+0000","last_peered":"2024-09-15T22:38:25.009573+0000","last_clean":"2024-09-15T22:38:25.009573+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:38:25.009573+0000","last_undegraded":"2024-09-15T22:38:25.009573+0000","last_fullsized":"2024-09-15T22:38:25.009573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240840+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:38:25.240840+0000","last_peered":"2024-09-15T22:38:25.240840+0000","last_clean":"2024-09-15T22:38:25.240840+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:38:25.240840+0000","last_undegraded":"2024-09-15T22:38:25.240840+0000","last_fullsized":"2024-09-15T22:38:25.240840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010739+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:38:25.010739+0000","last_peered":"2024-09-15T22:38:25.010739+0000","last_clean":"2024-09-15T22:38:25.010739+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:38:25.010739+0000","last_undegraded":"2024-09-15T22:38:25.010739+0000","last_fullsized":"2024-09-15T22:38:25.010739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241387+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:38:25.241387+0000","last_peered":"2024-09-15T22:38:25.241387+0000","last_clean":"2024-09-15T22:38:25.241387+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:38:25.241387+0000","last_undegraded":"2024-09-15T22:38:25.241387+0000","last_fullsized":"2024-09-15T22:38:25.241387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241192+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:38:25.241192+0000","last_peered":"2024-09-15T22:38:25.241192+0000","last_clean":"2024-09-15T22:38:25.241192+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:38:25.241192+0000","last_undegraded":"2024-09-15T22:38:25.241192+0000","last_fullsized":"2024-09-15T22:38:25.241192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113745+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:38:25.113745+0000","last_peered":"2024-09-15T22:38:25.113745+0000","last_clean":"2024-09-15T22:38:25.113745+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:38:25.113745+0000","last_undegraded":"2024-09-15T22:38:25.113745+0000","last_fullsized":"2024-09-15T22:38:25.113745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241575+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:38:25.241575+0000","last_peered":"2024-09-15T22:38:25.241575+0000","last_clean":"2024-09-15T22:38:25.241575+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:38:25.241575+0000","last_undegraded":"2024-09-15T22:38:25.241575+0000","last_fullsized":"2024-09-15T22:38:25.241575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241190+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:38:25.241190+0000","last_peered":"2024-09-15T22:38:25.241190+0000","last_clean":"2024-09-15T22:38:25.241190+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:38:25.241190+0000","last_undegraded":"2024-09-15T22:38:25.241190+0000","last_fullsized":"2024-09-15T22:38:25.241190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241234+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:38:25.241234+0000","last_peered":"2024-09-15T22:38:25.241234+0000","last_clean":"2024-09-15T22:38:25.241234+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:38:25.241234+0000","last_undegraded":"2024-09-15T22:38:25.241234+0000","last_fullsized":"2024-09-15T22:38:25.241234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114246+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:38:25.114246+0000","last_peered":"2024-09-15T22:38:25.114246+0000","last_clean":"2024-09-15T22:38:25.114246+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:38:25.114246+0000","last_undegraded":"2024-09-15T22:38:25.114246+0000","last_fullsized":"2024-09-15T22:38:25.114246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108613+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:38:25.108613+0000","last_peered":"2024-09-15T22:38:25.108613+0000","last_clean":"2024-09-15T22:38:25.108613+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:38:25.108613+0000","last_undegraded":"2024-09-15T22:38:25.108613+0000","last_fullsized":"2024-09-15T22:38:25.108613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112056+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:38:25.112056+0000","last_peered":"2024-09-15T22:38:25.112056+0000","last_clean":"2024-09-15T22:38:25.112056+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:38:25.112056+0000","last_undegraded":"2024-09-15T22:38:25.112056+0000","last_fullsized":"2024-09-15T22:38:25.112056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241262+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:38:25.241262+0000","last_peered":"2024-09-15T22:38:25.241262+0000","last_clean":"2024-09-15T22:38:25.241262+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:38:25.241262+0000","last_undegraded":"2024-09-15T22:38:25.241262+0000","last_fullsized":"2024-09-15T22:38:25.241262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241248+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:38:25.241248+0000","last_peered":"2024-09-15T22:38:25.241248+0000","last_clean":"2024-09-15T22:38:25.241248+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:38:25.241248+0000","last_undegraded":"2024-09-15T22:38:25.241248+0000","last_fullsized":"2024-09-15T22:38:25.241248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249966+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:38:25.249966+0000","last_peered":"2024-09-15T22:38:25.249966+0000","last_clean":"2024-09-15T22:38:25.249966+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:38:25.249966+0000","last_undegraded":"2024-09-15T22:38:25.249966+0000","last_fullsized":"2024-09-15T22:38:25.249966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241375+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:38:25.241375+0000","last_peered":"2024-09-15T22:38:25.241375+0000","last_clean":"2024-09-15T22:38:25.241375+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:38:25.241375+0000","last_undegraded":"2024-09-15T22:38:25.241375+0000","last_fullsized":"2024-09-15T22:38:25.241375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241418+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:38:25.241418+0000","last_peered":"2024-09-15T22:38:25.241418+0000","last_clean":"2024-09-15T22:38:25.241418+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:38:25.241418+0000","last_undegraded":"2024-09-15T22:38:25.241418+0000","last_fullsized":"2024-09-15T22:38:25.241418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115826+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:38:25.115826+0000","last_peered":"2024-09-15T22:38:25.115826+0000","last_clean":"2024-09-15T22:38:25.115826+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:38:25.115826+0000","last_undegraded":"2024-09-15T22:38:25.115826+0000","last_fullsized":"2024-09-15T22:38:25.115826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'48","reported_seq":112,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.239768+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:38:25.239768+0000","last_peered":"2024-09-15T22:38:25.239768+0000","last_clean":"2024-09-15T22:38:25.239768+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:38:25.239768+0000","last_undegraded":"2024-09-15T22:38:25.239768+0000","last_fullsized":"2024-09-15T22:38:25.239768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.111983+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:38:25.111983+0000","last_peered":"2024-09-15T22:38:25.111983+0000","last_clean":"2024-09-15T22:38:25.111983+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:38:25.111983+0000","last_undegraded":"2024-09-15T22:38:25.111983+0000","last_fullsized":"2024-09-15T22:38:25.111983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006514+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:38:25.006514+0000","last_peered":"2024-09-15T22:38:25.006514+0000","last_clean":"2024-09-15T22:38:25.006514+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:38:25.006514+0000","last_undegraded":"2024-09-15T22:38:25.006514+0000","last_fullsized":"2024-09-15T22:38:25.006514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112492+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:38:25.112492+0000","last_peered":"2024-09-15T22:38:25.112492+0000","last_clean":"2024-09-15T22:38:25.112492+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:38:25.112492+0000","last_undegraded":"2024-09-15T22:38:25.112492+0000","last_fullsized":"2024-09-15T22:38:25.112492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241541+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:38:25.241541+0000","last_peered":"2024-09-15T22:38:25.241541+0000","last_clean":"2024-09-15T22:38:25.241541+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:38:25.241541+0000","last_undegraded":"2024-09-15T22:38:25.241541+0000","last_fullsized":"2024-09-15T22:38:25.241541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124170+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:38:25.124170+0000","last_peered":"2024-09-15T22:38:25.124170+0000","last_clean":"2024-09-15T22:38:25.124170+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:38:25.124170+0000","last_undegraded":"2024-09-15T22:38:25.124170+0000","last_fullsized":"2024-09-15T22:38:25.124170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.110438+0000","last_change":"2024-09-15T22:38:14.242090+0000","last_active":"2024-09-15T22:38:25.110438+0000","last_peered":"2024-09-15T22:38:25.110438+0000","last_clean":"2024-09-15T22:38:25.110438+0000","last_became_active":"2024-09-15T22:38:14.241634+0000","last_became_peered":"2024-09-15T22:38:14.241634+0000","last_unstale":"2024-09-15T22:38:25.110438+0000","last_undegraded":"2024-09-15T22:38:25.110438+0000","last_fullsized":"2024-09-15T22:38:25.110438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:13.485506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112062+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:38:25.112062+0000","last_peered":"2024-09-15T22:38:25.112062+0000","last_clean":"2024-09-15T22:38:25.112062+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:38:25.112062+0000","last_undegraded":"2024-09-15T22:38:25.112062+0000","last_fullsized":"2024-09-15T22:38:25.112062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124297+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:38:25.124297+0000","last_peered":"2024-09-15T22:38:25.124297+0000","last_clean":"2024-09-15T22:38:25.124297+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:38:25.124297+0000","last_undegraded":"2024-09-15T22:38:25.124297+0000","last_fullsized":"2024-09-15T22:38:25.124297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010822+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:38:25.010822+0000","last_peered":"2024-09-15T22:38:25.010822+0000","last_clean":"2024-09-15T22:38:25.010822+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:38:25.010822+0000","last_undegraded":"2024-09-15T22:38:25.010822+0000","last_fullsized":"2024-09-15T22:38:25.010822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006453+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:38:25.006453+0000","last_peered":"2024-09-15T22:38:25.006453+0000","last_clean":"2024-09-15T22:38:25.006453+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:38:25.006453+0000","last_undegraded":"2024-09-15T22:38:25.006453+0000","last_fullsized":"2024-09-15T22:38:25.006453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009249+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:38:25.009249+0000","last_peered":"2024-09-15T22:38:25.009249+0000","last_clean":"2024-09-15T22:38:25.009249+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:38:25.009249+0000","last_undegraded":"2024-09-15T22:38:25.009249+0000","last_fullsized":"2024-09-15T22:38:25.009249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006157+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:38:25.006157+0000","last_peered":"2024-09-15T22:38:25.006157+0000","last_clean":"2024-09-15T22:38:25.006157+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:38:25.006157+0000","last_undegraded":"2024-09-15T22:38:25.006157+0000","last_fullsized":"2024-09-15T22:38:25.006157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006955+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:38:25.006955+0000","last_peered":"2024-09-15T22:38:25.006955+0000","last_clean":"2024-09-15T22:38:25.006955+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:38:25.006955+0000","last_undegraded":"2024-09-15T22:38:25.006955+0000","last_fullsized":"2024-09-15T22:38:25.006955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005813+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:38:25.005813+0000","last_peered":"2024-09-15T22:38:25.005813+0000","last_clean":"2024-09-15T22:38:25.005813+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:38:25.005813+0000","last_undegraded":"2024-09-15T22:38:25.005813+0000","last_fullsized":"2024-09-15T22:38:25.005813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010164+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:38:25.010164+0000","last_peered":"2024-09-15T22:38:25.010164+0000","last_clean":"2024-09-15T22:38:25.010164+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:38:25.010164+0000","last_undegraded":"2024-09-15T22:38:25.010164+0000","last_fullsized":"2024-09-15T22:38:25.010164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009598+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:38:25.009598+0000","last_peered":"2024-09-15T22:38:25.009598+0000","last_clean":"2024-09-15T22:38:25.009598+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:38:25.009598+0000","last_undegraded":"2024-09-15T22:38:25.009598+0000","last_fullsized":"2024-09-15T22:38:25.009598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113256+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:38:25.113256+0000","last_peered":"2024-09-15T22:38:25.113256+0000","last_clean":"2024-09-15T22:38:25.113256+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:38:25.113256+0000","last_undegraded":"2024-09-15T22:38:25.113256+0000","last_fullsized":"2024-09-15T22:38:25.113256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241238+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:38:25.241238+0000","last_peered":"2024-09-15T22:38:25.241238+0000","last_clean":"2024-09-15T22:38:25.241238+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:38:25.241238+0000","last_undegraded":"2024-09-15T22:38:25.241238+0000","last_fullsized":"2024-09-15T22:38:25.241238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010362+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:38:25.010362+0000","last_peered":"2024-09-15T22:38:25.010362+0000","last_clean":"2024-09-15T22:38:25.010362+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:38:25.010362+0000","last_undegraded":"2024-09-15T22:38:25.010362+0000","last_fullsized":"2024-09-15T22:38:25.010362+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010316+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:38:25.010316+0000","last_peered":"2024-09-15T22:38:25.010316+0000","last_clean":"2024-09-15T22:38:25.010316+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:38:25.010316+0000","last_undegraded":"2024-09-15T22:38:25.010316+0000","last_fullsized":"2024-09-15T22:38:25.010316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010530+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:38:25.010530+0000","last_peered":"2024-09-15T22:38:25.010530+0000","last_clean":"2024-09-15T22:38:25.010530+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:38:25.010530+0000","last_undegraded":"2024-09-15T22:38:25.010530+0000","last_fullsized":"2024-09-15T22:38:25.010530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241899+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:38:25.241899+0000","last_peered":"2024-09-15T22:38:25.241899+0000","last_clean":"2024-09-15T22:38:25.241899+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:38:25.241899+0000","last_undegraded":"2024-09-15T22:38:25.241899+0000","last_fullsized":"2024-09-15T22:38:25.241899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241629+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:38:25.241629+0000","last_peered":"2024-09-15T22:38:25.241629+0000","last_clean":"2024-09-15T22:38:25.241629+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:38:25.241629+0000","last_undegraded":"2024-09-15T22:38:25.241629+0000","last_fullsized":"2024-09-15T22:38:25.241629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010356+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:38:25.010356+0000","last_peered":"2024-09-15T22:38:25.010356+0000","last_clean":"2024-09-15T22:38:25.010356+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:38:25.010356+0000","last_undegraded":"2024-09-15T22:38:25.010356+0000","last_fullsized":"2024-09-15T22:38:25.010356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242363+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:38:25.242363+0000","last_peered":"2024-09-15T22:38:25.242363+0000","last_clean":"2024-09-15T22:38:25.242363+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:38:25.242363+0000","last_undegraded":"2024-09-15T22:38:25.242363+0000","last_fullsized":"2024-09-15T22:38:25.242363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248973+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:38:25.248973+0000","last_peered":"2024-09-15T22:38:25.248973+0000","last_clean":"2024-09-15T22:38:25.248973+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:38:25.248973+0000","last_undegraded":"2024-09-15T22:38:25.248973+0000","last_fullsized":"2024-09-15T22:38:25.248973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010508+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:38:25.010508+0000","last_peered":"2024-09-15T22:38:25.010508+0000","last_clean":"2024-09-15T22:38:25.010508+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:38:25.010508+0000","last_undegraded":"2024-09-15T22:38:25.010508+0000","last_fullsized":"2024-09-15T22:38:25.010508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006902+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:38:25.006902+0000","last_peered":"2024-09-15T22:38:25.006902+0000","last_clean":"2024-09-15T22:38:25.006902+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:38:25.006902+0000","last_undegraded":"2024-09-15T22:38:25.006902+0000","last_fullsized":"2024-09-15T22:38:25.006902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009045+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:38:25.009045+0000","last_peered":"2024-09-15T22:38:25.009045+0000","last_clean":"2024-09-15T22:38:25.009045+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:38:25.009045+0000","last_undegraded":"2024-09-15T22:38:25.009045+0000","last_fullsized":"2024-09-15T22:38:25.009045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242337+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:38:25.242337+0000","last_peered":"2024-09-15T22:38:25.242337+0000","last_clean":"2024-09-15T22:38:25.242337+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:38:25.242337+0000","last_undegraded":"2024-09-15T22:38:25.242337+0000","last_fullsized":"2024-09-15T22:38:25.242337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.395057+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:38:25.395057+0000","last_peered":"2024-09-15T22:38:25.395057+0000","last_clean":"2024-09-15T22:38:25.395057+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:38:25.395057+0000","last_undegraded":"2024-09-15T22:38:25.395057+0000","last_fullsized":"2024-09-15T22:38:25.395057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009192+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:38:25.009192+0000","last_peered":"2024-09-15T22:38:25.009192+0000","last_clean":"2024-09-15T22:38:25.009192+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:38:25.009192+0000","last_undegraded":"2024-09-15T22:38:25.009192+0000","last_fullsized":"2024-09-15T22:38:25.009192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009375+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:38:25.009375+0000","last_peered":"2024-09-15T22:38:25.009375+0000","last_clean":"2024-09-15T22:38:25.009375+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:38:25.009375+0000","last_undegraded":"2024-09-15T22:38:25.009375+0000","last_fullsized":"2024-09-15T22:38:25.009375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112094+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:38:25.112094+0000","last_peered":"2024-09-15T22:38:25.112094+0000","last_clean":"2024-09-15T22:38:25.112094+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:38:25.112094+0000","last_undegraded":"2024-09-15T22:38:25.112094+0000","last_fullsized":"2024-09-15T22:38:25.112094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006770+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:38:25.006770+0000","last_peered":"2024-09-15T22:38:25.006770+0000","last_clean":"2024-09-15T22:38:25.006770+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:38:25.006770+0000","last_undegraded":"2024-09-15T22:38:25.006770+0000","last_fullsized":"2024-09-15T22:38:25.006770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010086+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:38:25.010086+0000","last_peered":"2024-09-15T22:38:25.010086+0000","last_clean":"2024-09-15T22:38:25.010086+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:38:25.010086+0000","last_undegraded":"2024-09-15T22:38:25.010086+0000","last_fullsized":"2024-09-15T22:38:25.010086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.539508+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:38:25.539508+0000","last_peered":"2024-09-15T22:38:25.539508+0000","last_clean":"2024-09-15T22:38:25.539508+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:38:25.539508+0000","last_undegraded":"2024-09-15T22:38:25.539508+0000","last_fullsized":"2024-09-15T22:38:25.539508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108636+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:38:25.108636+0000","last_peered":"2024-09-15T22:38:25.108636+0000","last_clean":"2024-09-15T22:38:25.108636+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:38:25.108636+0000","last_undegraded":"2024-09-15T22:38:25.108636+0000","last_fullsized":"2024-09-15T22:38:25.108636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010264+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:38:25.010264+0000","last_peered":"2024-09-15T22:38:25.010264+0000","last_clean":"2024-09-15T22:38:25.010264+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:38:25.010264+0000","last_undegraded":"2024-09-15T22:38:25.010264+0000","last_fullsized":"2024-09-15T22:38:25.010264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009541+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:38:25.009541+0000","last_peered":"2024-09-15T22:38:25.009541+0000","last_clean":"2024-09-15T22:38:25.009541+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:38:25.009541+0000","last_undegraded":"2024-09-15T22:38:25.009541+0000","last_fullsized":"2024-09-15T22:38:25.009541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241984+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:38:25.241984+0000","last_peered":"2024-09-15T22:38:25.241984+0000","last_clean":"2024-09-15T22:38:25.241984+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:38:25.241984+0000","last_undegraded":"2024-09-15T22:38:25.241984+0000","last_fullsized":"2024-09-15T22:38:25.241984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005995+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:38:25.005995+0000","last_peered":"2024-09-15T22:38:25.005995+0000","last_clean":"2024-09-15T22:38:25.005995+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:38:25.005995+0000","last_undegraded":"2024-09-15T22:38:25.005995+0000","last_fullsized":"2024-09-15T22:38:25.005995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241792+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:38:25.241792+0000","last_peered":"2024-09-15T22:38:25.241792+0000","last_clean":"2024-09-15T22:38:25.241792+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:38:25.241792+0000","last_undegraded":"2024-09-15T22:38:25.241792+0000","last_fullsized":"2024-09-15T22:38:25.241792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009484+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:38:25.009484+0000","last_peered":"2024-09-15T22:38:25.009484+0000","last_clean":"2024-09-15T22:38:25.009484+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:38:25.009484+0000","last_undegraded":"2024-09-15T22:38:25.009484+0000","last_fullsized":"2024-09-15T22:38:25.009484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112389+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:38:25.112389+0000","last_peered":"2024-09-15T22:38:25.112389+0000","last_clean":"2024-09-15T22:38:25.112389+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:38:25.112389+0000","last_undegraded":"2024-09-15T22:38:25.112389+0000","last_fullsized":"2024-09-15T22:38:25.112389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.537439+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:38:25.537439+0000","last_peered":"2024-09-15T22:38:25.537439+0000","last_clean":"2024-09-15T22:38:25.537439+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:38:25.537439+0000","last_undegraded":"2024-09-15T22:38:25.537439+0000","last_fullsized":"2024-09-15T22:38:25.537439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69721,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":214,"num_read_kb":175,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107742+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:38:25.107742+0000","last_peered":"2024-09-15T22:38:25.107742+0000","last_clean":"2024-09-15T22:38:25.107742+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:38:25.107742+0000","last_undegraded":"2024-09-15T22:38:25.107742+0000","last_fullsized":"2024-09-15T22:38:25.107742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241717+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:38:25.241717+0000","last_peered":"2024-09-15T22:38:25.241717+0000","last_clean":"2024-09-15T22:38:25.241717+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:38:25.241717+0000","last_undegraded":"2024-09-15T22:38:25.241717+0000","last_fullsized":"2024-09-15T22:38:25.241717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009805+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:38:25.009805+0000","last_peered":"2024-09-15T22:38:25.009805+0000","last_clean":"2024-09-15T22:38:25.009805+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:38:25.009805+0000","last_undegraded":"2024-09-15T22:38:25.009805+0000","last_fullsized":"2024-09-15T22:38:25.009805+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241127+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:38:25.241127+0000","last_peered":"2024-09-15T22:38:25.241127+0000","last_clean":"2024-09-15T22:38:25.241127+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:38:25.241127+0000","last_undegraded":"2024-09-15T22:38:25.241127+0000","last_fullsized":"2024-09-15T22:38:25.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250624+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:38:25.250624+0000","last_peered":"2024-09-15T22:38:25.250624+0000","last_clean":"2024-09-15T22:38:25.250624+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:38:25.250624+0000","last_undegraded":"2024-09-15T22:38:25.250624+0000","last_fullsized":"2024-09-15T22:38:25.250624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010393+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:38:25.010393+0000","last_peered":"2024-09-15T22:38:25.010393+0000","last_clean":"2024-09-15T22:38:25.010393+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:38:25.010393+0000","last_undegraded":"2024-09-15T22:38:25.010393+0000","last_fullsized":"2024-09-15T22:38:25.010393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107495+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:38:25.107495+0000","last_peered":"2024-09-15T22:38:25.107495+0000","last_clean":"2024-09-15T22:38:25.107495+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:38:25.107495+0000","last_undegraded":"2024-09-15T22:38:25.107495+0000","last_fullsized":"2024-09-15T22:38:25.107495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124524+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:38:25.124524+0000","last_peered":"2024-09-15T22:38:25.124524+0000","last_clean":"2024-09-15T22:38:25.124524+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:38:25.124524+0000","last_undegraded":"2024-09-15T22:38:25.124524+0000","last_fullsized":"2024-09-15T22:38:25.124524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112430+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:38:25.112430+0000","last_peered":"2024-09-15T22:38:25.112430+0000","last_clean":"2024-09-15T22:38:25.112430+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:38:25.112430+0000","last_undegraded":"2024-09-15T22:38:25.112430+0000","last_fullsized":"2024-09-15T22:38:25.112430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249732+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:38:25.249732+0000","last_peered":"2024-09-15T22:38:25.249732+0000","last_clean":"2024-09-15T22:38:25.249732+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:38:25.249732+0000","last_undegraded":"2024-09-15T22:38:25.249732+0000","last_fullsized":"2024-09-15T22:38:25.249732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107744+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:38:25.107744+0000","last_peered":"2024-09-15T22:38:25.107744+0000","last_clean":"2024-09-15T22:38:25.107744+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:38:25.107744+0000","last_undegraded":"2024-09-15T22:38:25.107744+0000","last_fullsized":"2024-09-15T22:38:25.107744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009479+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:38:25.009479+0000","last_peered":"2024-09-15T22:38:25.009479+0000","last_clean":"2024-09-15T22:38:25.009479+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:38:25.009479+0000","last_undegraded":"2024-09-15T22:38:25.009479+0000","last_fullsized":"2024-09-15T22:38:25.009479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242387+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:38:25.242387+0000","last_peered":"2024-09-15T22:38:25.242387+0000","last_clean":"2024-09-15T22:38:25.242387+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:38:25.242387+0000","last_undegraded":"2024-09-15T22:38:25.242387+0000","last_fullsized":"2024-09-15T22:38:25.242387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112126+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:38:25.112126+0000","last_peered":"2024-09-15T22:38:25.112126+0000","last_clean":"2024-09-15T22:38:25.112126+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:38:25.112126+0000","last_undegraded":"2024-09-15T22:38:25.112126+0000","last_fullsized":"2024-09-15T22:38:25.112126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005971+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:38:25.005971+0000","last_peered":"2024-09-15T22:38:25.005971+0000","last_clean":"2024-09-15T22:38:25.005971+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:38:25.005971+0000","last_undegraded":"2024-09-15T22:38:25.005971+0000","last_fullsized":"2024-09-15T22:38:25.005971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124202+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:38:25.124202+0000","last_peered":"2024-09-15T22:38:25.124202+0000","last_clean":"2024-09-15T22:38:25.124202+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:38:25.124202+0000","last_undegraded":"2024-09-15T22:38:25.124202+0000","last_fullsized":"2024-09-15T22:38:25.124202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009565+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:38:25.009565+0000","last_peered":"2024-09-15T22:38:25.009565+0000","last_clean":"2024-09-15T22:38:25.009565+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:38:25.009565+0000","last_undegraded":"2024-09-15T22:38:25.009565+0000","last_fullsized":"2024-09-15T22:38:25.009565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241674+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:38:25.241674+0000","last_peered":"2024-09-15T22:38:25.241674+0000","last_clean":"2024-09-15T22:38:25.241674+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:38:25.241674+0000","last_undegraded":"2024-09-15T22:38:25.241674+0000","last_fullsized":"2024-09-15T22:38:25.241674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009806+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:38:25.009806+0000","last_peered":"2024-09-15T22:38:25.009806+0000","last_clean":"2024-09-15T22:38:25.009806+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:38:25.009806+0000","last_undegraded":"2024-09-15T22:38:25.009806+0000","last_fullsized":"2024-09-15T22:38:25.009806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112093+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:38:25.112093+0000","last_peered":"2024-09-15T22:38:25.112093+0000","last_clean":"2024-09-15T22:38:25.112093+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:38:25.112093+0000","last_undegraded":"2024-09-15T22:38:25.112093+0000","last_fullsized":"2024-09-15T22:38:25.112093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241736+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:38:25.241736+0000","last_peered":"2024-09-15T22:38:25.241736+0000","last_clean":"2024-09-15T22:38:25.241736+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:38:25.241736+0000","last_undegraded":"2024-09-15T22:38:25.241736+0000","last_fullsized":"2024-09-15T22:38:25.241736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241163+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:38:25.241163+0000","last_peered":"2024-09-15T22:38:25.241163+0000","last_clean":"2024-09-15T22:38:25.241163+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:38:25.241163+0000","last_undegraded":"2024-09-15T22:38:25.241163+0000","last_fullsized":"2024-09-15T22:38:25.241163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241675+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:38:25.241675+0000","last_peered":"2024-09-15T22:38:25.241675+0000","last_clean":"2024-09-15T22:38:25.241675+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:38:25.241675+0000","last_undegraded":"2024-09-15T22:38:25.241675+0000","last_fullsized":"2024-09-15T22:38:25.241675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241517+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:38:25.241517+0000","last_peered":"2024-09-15T22:38:25.241517+0000","last_clean":"2024-09-15T22:38:25.241517+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:38:25.241517+0000","last_undegraded":"2024-09-15T22:38:25.241517+0000","last_fullsized":"2024-09-15T22:38:25.241517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249023+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:38:25.249023+0000","last_peered":"2024-09-15T22:38:25.249023+0000","last_clean":"2024-09-15T22:38:25.249023+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:38:25.249023+0000","last_undegraded":"2024-09-15T22:38:25.249023+0000","last_fullsized":"2024-09-15T22:38:25.249023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006558+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:38:25.006558+0000","last_peered":"2024-09-15T22:38:25.006558+0000","last_clean":"2024-09-15T22:38:25.006558+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:38:25.006558+0000","last_undegraded":"2024-09-15T22:38:25.006558+0000","last_fullsized":"2024-09-15T22:38:25.006558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242143+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:38:25.242143+0000","last_peered":"2024-09-15T22:38:25.242143+0000","last_clean":"2024-09-15T22:38:25.242143+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:38:25.242143+0000","last_undegraded":"2024-09-15T22:38:25.242143+0000","last_fullsized":"2024-09-15T22:38:25.242143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241544+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:38:25.241544+0000","last_peered":"2024-09-15T22:38:25.241544+0000","last_clean":"2024-09-15T22:38:25.241544+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:38:25.241544+0000","last_undegraded":"2024-09-15T22:38:25.241544+0000","last_fullsized":"2024-09-15T22:38:25.241544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115695+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:38:25.115695+0000","last_peered":"2024-09-15T22:38:25.115695+0000","last_clean":"2024-09-15T22:38:25.115695+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:38:25.115695+0000","last_undegraded":"2024-09-15T22:38:25.115695+0000","last_fullsized":"2024-09-15T22:38:25.115695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009267+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:38:25.009267+0000","last_peered":"2024-09-15T22:38:25.009267+0000","last_clean":"2024-09-15T22:38:25.009267+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:38:25.009267+0000","last_undegraded":"2024-09-15T22:38:25.009267+0000","last_fullsized":"2024-09-15T22:38:25.009267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112660+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:38:25.112660+0000","last_peered":"2024-09-15T22:38:25.112660+0000","last_clean":"2024-09-15T22:38:25.112660+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:38:25.112660+0000","last_undegraded":"2024-09-15T22:38:25.112660+0000","last_fullsized":"2024-09-15T22:38:25.112660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241756+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:38:25.241756+0000","last_peered":"2024-09-15T22:38:25.241756+0000","last_clean":"2024-09-15T22:38:25.241756+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:38:25.241756+0000","last_undegraded":"2024-09-15T22:38:25.241756+0000","last_fullsized":"2024-09-15T22:38:25.241756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006219+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:38:25.006219+0000","last_peered":"2024-09-15T22:38:25.006219+0000","last_clean":"2024-09-15T22:38:25.006219+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:38:25.006219+0000","last_undegraded":"2024-09-15T22:38:25.006219+0000","last_fullsized":"2024-09-15T22:38:25.006219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242055+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:38:25.242055+0000","last_peered":"2024-09-15T22:38:25.242055+0000","last_clean":"2024-09-15T22:38:25.242055+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:38:25.242055+0000","last_undegraded":"2024-09-15T22:38:25.242055+0000","last_fullsized":"2024-09-15T22:38:25.242055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241921+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:38:25.241921+0000","last_peered":"2024-09-15T22:38:25.241921+0000","last_clean":"2024-09-15T22:38:25.241921+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:38:25.241921+0000","last_undegraded":"2024-09-15T22:38:25.241921+0000","last_fullsized":"2024-09-15T22:38:25.241921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.109165+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:38:25.109165+0000","last_peered":"2024-09-15T22:38:25.109165+0000","last_clean":"2024-09-15T22:38:25.109165+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:38:25.109165+0000","last_undegraded":"2024-09-15T22:38:25.109165+0000","last_fullsized":"2024-09-15T22:38:25.109165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242550+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:38:25.242550+0000","last_peered":"2024-09-15T22:38:25.242550+0000","last_clean":"2024-09-15T22:38:25.242550+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:38:25.242550+0000","last_undegraded":"2024-09-15T22:38:25.242550+0000","last_fullsized":"2024-09-15T22:38:25.242550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241031+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:38:25.241031+0000","last_peered":"2024-09-15T22:38:25.241031+0000","last_clean":"2024-09-15T22:38:25.241031+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:38:25.241031+0000","last_undegraded":"2024-09-15T22:38:25.241031+0000","last_fullsized":"2024-09-15T22:38:25.241031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248274+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:38:25.248274+0000","last_peered":"2024-09-15T22:38:25.248274+0000","last_clean":"2024-09-15T22:38:25.248274+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:38:25.248274+0000","last_undegraded":"2024-09-15T22:38:25.248274+0000","last_fullsized":"2024-09-15T22:38:25.248274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248580+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:38:25.248580+0000","last_peered":"2024-09-15T22:38:25.248580+0000","last_clean":"2024-09-15T22:38:25.248580+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:38:25.248580+0000","last_undegraded":"2024-09-15T22:38:25.248580+0000","last_fullsized":"2024-09-15T22:38:25.248580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006155+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:38:25.006155+0000","last_peered":"2024-09-15T22:38:25.006155+0000","last_clean":"2024-09-15T22:38:25.006155+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:38:25.006155+0000","last_undegraded":"2024-09-15T22:38:25.006155+0000","last_fullsized":"2024-09-15T22:38:25.006155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241453+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:38:25.241453+0000","last_peered":"2024-09-15T22:38:25.241453+0000","last_clean":"2024-09-15T22:38:25.241453+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:38:25.241453+0000","last_undegraded":"2024-09-15T22:38:25.241453+0000","last_fullsized":"2024-09-15T22:38:25.241453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010699+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:38:25.010699+0000","last_peered":"2024-09-15T22:38:25.010699+0000","last_clean":"2024-09-15T22:38:25.010699+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:38:25.010699+0000","last_undegraded":"2024-09-15T22:38:25.010699+0000","last_fullsized":"2024-09-15T22:38:25.010699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124494+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:38:25.124494+0000","last_peered":"2024-09-15T22:38:25.124494+0000","last_clean":"2024-09-15T22:38:25.124494+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:38:25.124494+0000","last_undegraded":"2024-09-15T22:38:25.124494+0000","last_fullsized":"2024-09-15T22:38:25.124494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112097+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:38:25.112097+0000","last_peered":"2024-09-15T22:38:25.112097+0000","last_clean":"2024-09-15T22:38:25.112097+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:38:25.112097+0000","last_undegraded":"2024-09-15T22:38:25.112097+0000","last_fullsized":"2024-09-15T22:38:25.112097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010114+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:38:25.010114+0000","last_peered":"2024-09-15T22:38:25.010114+0000","last_clean":"2024-09-15T22:38:25.010114+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:38:25.010114+0000","last_undegraded":"2024-09-15T22:38:25.010114+0000","last_fullsized":"2024-09-15T22:38:25.010114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124146+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:38:25.124146+0000","last_peered":"2024-09-15T22:38:25.124146+0000","last_clean":"2024-09-15T22:38:25.124146+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:38:25.124146+0000","last_undegraded":"2024-09-15T22:38:25.124146+0000","last_fullsized":"2024-09-15T22:38:25.124146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112085+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:38:25.112085+0000","last_peered":"2024-09-15T22:38:25.112085+0000","last_clean":"2024-09-15T22:38:25.112085+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:38:25.112085+0000","last_undegraded":"2024-09-15T22:38:25.112085+0000","last_fullsized":"2024-09-15T22:38:25.112085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108245+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:38:25.108245+0000","last_peered":"2024-09-15T22:38:25.108245+0000","last_clean":"2024-09-15T22:38:25.108245+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:38:25.108245+0000","last_undegraded":"2024-09-15T22:38:25.108245+0000","last_fullsized":"2024-09-15T22:38:25.108245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249437+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:38:25.249437+0000","last_peered":"2024-09-15T22:38:25.249437+0000","last_clean":"2024-09-15T22:38:25.249437+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:38:25.249437+0000","last_undegraded":"2024-09-15T22:38:25.249437+0000","last_fullsized":"2024-09-15T22:38:25.249437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249830+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:38:25.249830+0000","last_peered":"2024-09-15T22:38:25.249830+0000","last_clean":"2024-09-15T22:38:25.249830+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:38:25.249830+0000","last_undegraded":"2024-09-15T22:38:25.249830+0000","last_fullsized":"2024-09-15T22:38:25.249830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242437+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:38:25.242437+0000","last_peered":"2024-09-15T22:38:25.242437+0000","last_clean":"2024-09-15T22:38:25.242437+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:38:25.242437+0000","last_undegraded":"2024-09-15T22:38:25.242437+0000","last_fullsized":"2024-09-15T22:38:25.242437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599775,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":730,"num_read_kb":476,"num_write":298,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1799325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":519982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841581,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28852,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716300,"statfs":{"total":95995035648,"available":95965491200,"internally_reserved":0,"allocated":1871872,"data_stored":1309665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005107,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36768,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708384,"statfs":{"total":95995035648,"available":95957385216,"internally_reserved":0,"allocated":1454080,"data_stored":917530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36572,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708580,"statfs":{"total":95995035648,"available":95957585920,"internally_reserved":0,"allocated":1253376,"data_stored":716746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1650688,"data_stored":1116498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":515394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855431,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1794048,"data_stored":1243378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":521250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1040404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":586797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":584239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":516782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:38:28.194 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-15T22:38:28.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:38:28.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: [15/Sep/2024:22:38:26] ENGINE Bus STARTING 2024-09-15T22:38:28.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:28.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-15T22:38:28.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: [15/Sep/2024:22:38:27] ENGINE Serving on https://172.21.15.16:7150 2024-09-15T22:38:28.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: [15/Sep/2024:22:38:27] ENGINE Error in HTTPServer.serve 2024-09-15T22:38:28.279 INFO:journalctl@ceph.mon.c.smithi130.stdout: Traceback (most recent call last): 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: self._run(expiration_interval) 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: s = self.context.wrap_socket( 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: return self.sslsocket_class._create( 2024-09-15T22:38:28.280 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout: self.do_handshake() 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout: self._sslobj.do_handshake() 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: [15/Sep/2024:22:38:27] ENGINE Serving on http://172.21.15.16:8765 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: [15/Sep/2024:22:38:27] ENGINE Bus STARTED 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.281 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi130 to 176.0M 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: Unable to set osd_memory_target on smithi130 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.282 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:28 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 -- 172.21.15.16:0/843618408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe460100a10 msgr2=0x7fe460100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 --2- 172.21.15.16:0/843618408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe460100a10 0x7fe460100e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe450009900 tx=0x7fe45002a410 comp rx=0 tx=0).stop 2024-09-15T22:38:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 -- 172.21.15.16:0/843618408 shutdown_connections 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 --2- 172.21.15.16:0/843618408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4601025c0 0x7fe460108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 --2- 172.21.15.16:0/843618408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe460101c00 0x7fe460102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 --2- 172.21.15.16:0/843618408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe460100a10 0x7fe460100e10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 -- 172.21.15.16:0/843618408 >> 172.21.15.16:0/843618408 conn(0x7fe4600fc170 msgr2=0x7fe4600fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 -- 172.21.15.16:0/843618408 shutdown_connections 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.336+0000 7fe4663f6640 1 -- 172.21.15.16:0/843618408 wait complete. 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 Processor -- start 2024-09-15T22:38:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 -- start start 2024-09-15T22:38:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe460100a10 0x7fe46010b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe460101c00 0x7fe4601099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4601025c0 0x7fe460109f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010d840 con 0x7fe4601025c0 2024-09-15T22:38:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010d6c0 con 0x7fe460101c00 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe4663f6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010d9c0 con 0x7fe460100a10 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe464993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4601025c0 0x7fe460109f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe464993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4601025c0 0x7fe460109f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46868/0 (socket says 172.21.15.16:46868) 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe464993640 1 -- 172.21.15.16:0/495844694 learned_addr learned my addr 172.21.15.16:0/495844694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.337+0000 7fe45ffff640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe460100a10 0x7fe46010b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe45f7fe640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe460101c00 0x7fe4601099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe464993640 1 -- 172.21.15.16:0/495844694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe460100a10 msgr2=0x7fe46010b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe464993640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe460100a10 0x7fe46010b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe464993640 1 -- 172.21.15.16:0/495844694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe460101c00 msgr2=0x7fe4601099d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe464993640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe460101c00 0x7fe4601099d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe464993640 1 -- 172.21.15.16:0/495844694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46010a7a0 con 0x7fe4601025c0 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe45f7fe640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe460101c00 0x7fe4601099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe464993640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4601025c0 0x7fe460109f10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe458009fd0 tx=0x7fe45800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe45d7fa640 1 -- 172.21.15.16:0/495844694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458017070 con 0x7fe4601025c0 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe45d7fa640 1 -- 172.21.15.16:0/495844694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe458003e50 con 0x7fe4601025c0 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4601a1fc0 con 0x7fe4601025c0 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe45d7fa640 1 -- 172.21.15.16:0/495844694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4580106c0 con 0x7fe4601025c0 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.338+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4601a23d0 con 0x7fe4601025c0 2024-09-15T22:38:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.339+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe460107e20 con 0x7fe4601025c0 2024-09-15T22:38:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.341+0000 7fe45d7fa640 1 -- 172.21.15.16:0/495844694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 138475+0+0 (secure 0 0 0) 0x7fe458003970 con 0x7fe4601025c0 2024-09-15T22:38:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.341+0000 7fe45d7fa640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4480a33c0 0x7fe4480a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.342+0000 7fe45d7fa640 1 -- 172.21.15.16:0/495844694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe4580cfd00 con 0x7fe4601025c0 2024-09-15T22:38:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.342+0000 7fe45ffff640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4480a33c0 0x7fe4480a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.342+0000 7fe45ffff640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4480a33c0 0x7fe4480a5880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe45002aa00 tx=0x7fe4500315b0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:28.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.345+0000 7fe45d7fa640 1 -- 172.21.15.16:0/495844694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe45809c380 con 0x7fe4601025c0 2024-09-15T22:38:28.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: [15/Sep/2024:22:38:26] ENGINE Bus STARTING 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: [15/Sep/2024:22:38:27] ENGINE Serving on https://172.21.15.16:7150 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: [15/Sep/2024:22:38:27] ENGINE Error in HTTPServer.serve 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout: Traceback (most recent call last): 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout: self._run(expiration_interval) 2024-09-15T22:38:28.445 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: s = self.context.wrap_socket( 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: return self.sslsocket_class._create( 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: self.do_handshake() 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:38:28.446 INFO:journalctl@ceph.mon.b.smithi059.stdout: self._sslobj.do_handshake() 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: [15/Sep/2024:22:38:27] ENGINE Serving on http://172.21.15.16:8765 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: [15/Sep/2024:22:38:27] ENGINE Bus STARTED 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.447 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi130 to 176.0M 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: Unable to set osd_memory_target on smithi130 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='client.25037 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.448 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.449 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.449 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:28 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.531+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe460062630 con 0x7fe4480a33c0 2024-09-15T22:38:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.540+0000 7fe45d7fa640 1 -- 172.21.15.16:0/495844694 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339380 (secure 0 0 0) 0x7fe460062630 con 0x7fe4480a33c0 2024-09-15T22:38:28.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:38:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.543+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4480a33c0 msgr2=0x7fe4480a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.543+0000 7fe4663f6640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4480a33c0 0x7fe4480a5880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe45002aa00 tx=0x7fe4500315b0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.543+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4601025c0 msgr2=0x7fe460109f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.543+0000 7fe4663f6640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4601025c0 0x7fe460109f10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe458009fd0 tx=0x7fe45800b260 comp rx=0 tx=0).stop 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 shutdown_connections 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4601025c0 0x7fe460109f10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe460101c00 0x7fe4601099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4480a33c0 0x7fe4480a5880 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 --2- 172.21.15.16:0/495844694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe460100a10 0x7fe46010b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 >> 172.21.15.16:0/495844694 conn(0x7fe4600fc170 msgr2=0x7fe460104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 shutdown_connections 2024-09-15T22:38:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:28.544+0000 7fe4663f6640 1 -- 172.21.15.16:0/495844694 wait complete. 2024-09-15T22:38:28.557 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-15T22:38:27.019577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190377,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":866,"num_read_kb":595,"num_write":481,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":414860,"kb_used_data":15656,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124526964,"statfs":{"total":1151940427776,"available":1151515611136,"internally_reserved":0,"allocated":16031744,"data_stored":9588665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992200"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108773+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:38:25.108773+0000","last_peered":"2024-09-15T22:38:25.108773+0000","last_clean":"2024-09-15T22:38:25.108773+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:38:25.108773+0000","last_undegraded":"2024-09-15T22:38:25.108773+0000","last_fullsized":"2024-09-15T22:38:25.108773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114924+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:38:25.114924+0000","last_peered":"2024-09-15T22:38:25.114924+0000","last_clean":"2024-09-15T22:38:25.114924+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:38:25.114924+0000","last_undegraded":"2024-09-15T22:38:25.114924+0000","last_fullsized":"2024-09-15T22:38:25.114924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241365+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:38:25.241365+0000","last_peered":"2024-09-15T22:38:25.241365+0000","last_clean":"2024-09-15T22:38:25.241365+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:38:25.241365+0000","last_undegraded":"2024-09-15T22:38:25.241365+0000","last_fullsized":"2024-09-15T22:38:25.241365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115300+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:38:25.115300+0000","last_peered":"2024-09-15T22:38:25.115300+0000","last_clean":"2024-09-15T22:38:25.115300+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:38:25.115300+0000","last_undegraded":"2024-09-15T22:38:25.115300+0000","last_fullsized":"2024-09-15T22:38:25.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250490+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:38:25.250490+0000","last_peered":"2024-09-15T22:38:25.250490+0000","last_clean":"2024-09-15T22:38:25.250490+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:38:25.250490+0000","last_undegraded":"2024-09-15T22:38:25.250490+0000","last_fullsized":"2024-09-15T22:38:25.250490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240846+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:38:25.240846+0000","last_peered":"2024-09-15T22:38:25.240846+0000","last_clean":"2024-09-15T22:38:25.240846+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:38:25.240846+0000","last_undegraded":"2024-09-15T22:38:25.240846+0000","last_fullsized":"2024-09-15T22:38:25.240846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112496+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:38:25.112496+0000","last_peered":"2024-09-15T22:38:25.112496+0000","last_clean":"2024-09-15T22:38:25.112496+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:38:25.112496+0000","last_undegraded":"2024-09-15T22:38:25.112496+0000","last_fullsized":"2024-09-15T22:38:25.112496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241710+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:38:25.241710+0000","last_peered":"2024-09-15T22:38:25.241710+0000","last_clean":"2024-09-15T22:38:25.241710+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:38:25.241710+0000","last_undegraded":"2024-09-15T22:38:25.241710+0000","last_fullsized":"2024-09-15T22:38:25.241710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241001+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:38:25.241001+0000","last_peered":"2024-09-15T22:38:25.241001+0000","last_clean":"2024-09-15T22:38:25.241001+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:38:25.241001+0000","last_undegraded":"2024-09-15T22:38:25.241001+0000","last_fullsized":"2024-09-15T22:38:25.241001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112329+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:38:25.112329+0000","last_peered":"2024-09-15T22:38:25.112329+0000","last_clean":"2024-09-15T22:38:25.112329+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:38:25.112329+0000","last_undegraded":"2024-09-15T22:38:25.112329+0000","last_fullsized":"2024-09-15T22:38:25.112329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009573+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:38:25.009573+0000","last_peered":"2024-09-15T22:38:25.009573+0000","last_clean":"2024-09-15T22:38:25.009573+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:38:25.009573+0000","last_undegraded":"2024-09-15T22:38:25.009573+0000","last_fullsized":"2024-09-15T22:38:25.009573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240840+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:38:25.240840+0000","last_peered":"2024-09-15T22:38:25.240840+0000","last_clean":"2024-09-15T22:38:25.240840+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:38:25.240840+0000","last_undegraded":"2024-09-15T22:38:25.240840+0000","last_fullsized":"2024-09-15T22:38:25.240840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010739+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:38:25.010739+0000","last_peered":"2024-09-15T22:38:25.010739+0000","last_clean":"2024-09-15T22:38:25.010739+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:38:25.010739+0000","last_undegraded":"2024-09-15T22:38:25.010739+0000","last_fullsized":"2024-09-15T22:38:25.010739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241387+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:38:25.241387+0000","last_peered":"2024-09-15T22:38:25.241387+0000","last_clean":"2024-09-15T22:38:25.241387+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:38:25.241387+0000","last_undegraded":"2024-09-15T22:38:25.241387+0000","last_fullsized":"2024-09-15T22:38:25.241387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241192+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:38:25.241192+0000","last_peered":"2024-09-15T22:38:25.241192+0000","last_clean":"2024-09-15T22:38:25.241192+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:38:25.241192+0000","last_undegraded":"2024-09-15T22:38:25.241192+0000","last_fullsized":"2024-09-15T22:38:25.241192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113745+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:38:25.113745+0000","last_peered":"2024-09-15T22:38:25.113745+0000","last_clean":"2024-09-15T22:38:25.113745+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:38:25.113745+0000","last_undegraded":"2024-09-15T22:38:25.113745+0000","last_fullsized":"2024-09-15T22:38:25.113745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241575+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:38:25.241575+0000","last_peered":"2024-09-15T22:38:25.241575+0000","last_clean":"2024-09-15T22:38:25.241575+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:38:25.241575+0000","last_undegraded":"2024-09-15T22:38:25.241575+0000","last_fullsized":"2024-09-15T22:38:25.241575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241190+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:38:25.241190+0000","last_peered":"2024-09-15T22:38:25.241190+0000","last_clean":"2024-09-15T22:38:25.241190+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:38:25.241190+0000","last_undegraded":"2024-09-15T22:38:25.241190+0000","last_fullsized":"2024-09-15T22:38:25.241190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241234+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:38:25.241234+0000","last_peered":"2024-09-15T22:38:25.241234+0000","last_clean":"2024-09-15T22:38:25.241234+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:38:25.241234+0000","last_undegraded":"2024-09-15T22:38:25.241234+0000","last_fullsized":"2024-09-15T22:38:25.241234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114246+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:38:25.114246+0000","last_peered":"2024-09-15T22:38:25.114246+0000","last_clean":"2024-09-15T22:38:25.114246+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:38:25.114246+0000","last_undegraded":"2024-09-15T22:38:25.114246+0000","last_fullsized":"2024-09-15T22:38:25.114246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108613+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:38:25.108613+0000","last_peered":"2024-09-15T22:38:25.108613+0000","last_clean":"2024-09-15T22:38:25.108613+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:38:25.108613+0000","last_undegraded":"2024-09-15T22:38:25.108613+0000","last_fullsized":"2024-09-15T22:38:25.108613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112056+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:38:25.112056+0000","last_peered":"2024-09-15T22:38:25.112056+0000","last_clean":"2024-09-15T22:38:25.112056+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:38:25.112056+0000","last_undegraded":"2024-09-15T22:38:25.112056+0000","last_fullsized":"2024-09-15T22:38:25.112056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241262+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:38:25.241262+0000","last_peered":"2024-09-15T22:38:25.241262+0000","last_clean":"2024-09-15T22:38:25.241262+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:38:25.241262+0000","last_undegraded":"2024-09-15T22:38:25.241262+0000","last_fullsized":"2024-09-15T22:38:25.241262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241248+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:38:25.241248+0000","last_peered":"2024-09-15T22:38:25.241248+0000","last_clean":"2024-09-15T22:38:25.241248+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:38:25.241248+0000","last_undegraded":"2024-09-15T22:38:25.241248+0000","last_fullsized":"2024-09-15T22:38:25.241248+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249966+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:38:25.249966+0000","last_peered":"2024-09-15T22:38:25.249966+0000","last_clean":"2024-09-15T22:38:25.249966+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:38:25.249966+0000","last_undegraded":"2024-09-15T22:38:25.249966+0000","last_fullsized":"2024-09-15T22:38:25.249966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241375+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:38:25.241375+0000","last_peered":"2024-09-15T22:38:25.241375+0000","last_clean":"2024-09-15T22:38:25.241375+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:38:25.241375+0000","last_undegraded":"2024-09-15T22:38:25.241375+0000","last_fullsized":"2024-09-15T22:38:25.241375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241418+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:38:25.241418+0000","last_peered":"2024-09-15T22:38:25.241418+0000","last_clean":"2024-09-15T22:38:25.241418+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:38:25.241418+0000","last_undegraded":"2024-09-15T22:38:25.241418+0000","last_fullsized":"2024-09-15T22:38:25.241418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115826+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:38:25.115826+0000","last_peered":"2024-09-15T22:38:25.115826+0000","last_clean":"2024-09-15T22:38:25.115826+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:38:25.115826+0000","last_undegraded":"2024-09-15T22:38:25.115826+0000","last_fullsized":"2024-09-15T22:38:25.115826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'48","reported_seq":112,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.239768+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:38:25.239768+0000","last_peered":"2024-09-15T22:38:25.239768+0000","last_clean":"2024-09-15T22:38:25.239768+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:38:25.239768+0000","last_undegraded":"2024-09-15T22:38:25.239768+0000","last_fullsized":"2024-09-15T22:38:25.239768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.111983+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:38:25.111983+0000","last_peered":"2024-09-15T22:38:25.111983+0000","last_clean":"2024-09-15T22:38:25.111983+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:38:25.111983+0000","last_undegraded":"2024-09-15T22:38:25.111983+0000","last_fullsized":"2024-09-15T22:38:25.111983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006514+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:38:25.006514+0000","last_peered":"2024-09-15T22:38:25.006514+0000","last_clean":"2024-09-15T22:38:25.006514+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:38:25.006514+0000","last_undegraded":"2024-09-15T22:38:25.006514+0000","last_fullsized":"2024-09-15T22:38:25.006514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112492+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:38:25.112492+0000","last_peered":"2024-09-15T22:38:25.112492+0000","last_clean":"2024-09-15T22:38:25.112492+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:38:25.112492+0000","last_undegraded":"2024-09-15T22:38:25.112492+0000","last_fullsized":"2024-09-15T22:38:25.112492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241541+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:38:25.241541+0000","last_peered":"2024-09-15T22:38:25.241541+0000","last_clean":"2024-09-15T22:38:25.241541+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:38:25.241541+0000","last_undegraded":"2024-09-15T22:38:25.241541+0000","last_fullsized":"2024-09-15T22:38:25.241541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124170+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:38:25.124170+0000","last_peered":"2024-09-15T22:38:25.124170+0000","last_clean":"2024-09-15T22:38:25.124170+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:38:25.124170+0000","last_undegraded":"2024-09-15T22:38:25.124170+0000","last_fullsized":"2024-09-15T22:38:25.124170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.110438+0000","last_change":"2024-09-15T22:38:14.242090+0000","last_active":"2024-09-15T22:38:25.110438+0000","last_peered":"2024-09-15T22:38:25.110438+0000","last_clean":"2024-09-15T22:38:25.110438+0000","last_became_active":"2024-09-15T22:38:14.241634+0000","last_became_peered":"2024-09-15T22:38:14.241634+0000","last_unstale":"2024-09-15T22:38:25.110438+0000","last_undegraded":"2024-09-15T22:38:25.110438+0000","last_fullsized":"2024-09-15T22:38:25.110438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:13.485506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112062+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:38:25.112062+0000","last_peered":"2024-09-15T22:38:25.112062+0000","last_clean":"2024-09-15T22:38:25.112062+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:38:25.112062+0000","last_undegraded":"2024-09-15T22:38:25.112062+0000","last_fullsized":"2024-09-15T22:38:25.112062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124297+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:38:25.124297+0000","last_peered":"2024-09-15T22:38:25.124297+0000","last_clean":"2024-09-15T22:38:25.124297+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:38:25.124297+0000","last_undegraded":"2024-09-15T22:38:25.124297+0000","last_fullsized":"2024-09-15T22:38:25.124297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010822+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:38:25.010822+0000","last_peered":"2024-09-15T22:38:25.010822+0000","last_clean":"2024-09-15T22:38:25.010822+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:38:25.010822+0000","last_undegraded":"2024-09-15T22:38:25.010822+0000","last_fullsized":"2024-09-15T22:38:25.010822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006453+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:38:25.006453+0000","last_peered":"2024-09-15T22:38:25.006453+0000","last_clean":"2024-09-15T22:38:25.006453+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:38:25.006453+0000","last_undegraded":"2024-09-15T22:38:25.006453+0000","last_fullsized":"2024-09-15T22:38:25.006453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009249+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:38:25.009249+0000","last_peered":"2024-09-15T22:38:25.009249+0000","last_clean":"2024-09-15T22:38:25.009249+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:38:25.009249+0000","last_undegraded":"2024-09-15T22:38:25.009249+0000","last_fullsized":"2024-09-15T22:38:25.009249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006157+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:38:25.006157+0000","last_peered":"2024-09-15T22:38:25.006157+0000","last_clean":"2024-09-15T22:38:25.006157+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:38:25.006157+0000","last_undegraded":"2024-09-15T22:38:25.006157+0000","last_fullsized":"2024-09-15T22:38:25.006157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006955+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:38:25.006955+0000","last_peered":"2024-09-15T22:38:25.006955+0000","last_clean":"2024-09-15T22:38:25.006955+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:38:25.006955+0000","last_undegraded":"2024-09-15T22:38:25.006955+0000","last_fullsized":"2024-09-15T22:38:25.006955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005813+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:38:25.005813+0000","last_peered":"2024-09-15T22:38:25.005813+0000","last_clean":"2024-09-15T22:38:25.005813+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:38:25.005813+0000","last_undegraded":"2024-09-15T22:38:25.005813+0000","last_fullsized":"2024-09-15T22:38:25.005813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010164+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:38:25.010164+0000","last_peered":"2024-09-15T22:38:25.010164+0000","last_clean":"2024-09-15T22:38:25.010164+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:38:25.010164+0000","last_undegraded":"2024-09-15T22:38:25.010164+0000","last_fullsized":"2024-09-15T22:38:25.010164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009598+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:38:25.009598+0000","last_peered":"2024-09-15T22:38:25.009598+0000","last_clean":"2024-09-15T22:38:25.009598+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:38:25.009598+0000","last_undegraded":"2024-09-15T22:38:25.009598+0000","last_fullsized":"2024-09-15T22:38:25.009598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113256+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:38:25.113256+0000","last_peered":"2024-09-15T22:38:25.113256+0000","last_clean":"2024-09-15T22:38:25.113256+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:38:25.113256+0000","last_undegraded":"2024-09-15T22:38:25.113256+0000","last_fullsized":"2024-09-15T22:38:25.113256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241238+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:38:25.241238+0000","last_peered":"2024-09-15T22:38:25.241238+0000","last_clean":"2024-09-15T22:38:25.241238+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:38:25.241238+0000","last_undegraded":"2024-09-15T22:38:25.241238+0000","last_fullsized":"2024-09-15T22:38:25.241238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":360,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010362+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:38:25.010362+0000","last_peered":"2024-09-15T22:38:25.010362+0000","last_clean":"2024-09-15T22:38:25.010362+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:38:25.010362+0000","last_undegraded":"2024-09-15T22:38:25.010362+0000","last_fullsized":"2024-09-15T22:38:25.010362+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010316+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:38:25.010316+0000","last_peered":"2024-09-15T22:38:25.010316+0000","last_clean":"2024-09-15T22:38:25.010316+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:38:25.010316+0000","last_undegraded":"2024-09-15T22:38:25.010316+0000","last_fullsized":"2024-09-15T22:38:25.010316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010530+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:38:25.010530+0000","last_peered":"2024-09-15T22:38:25.010530+0000","last_clean":"2024-09-15T22:38:25.010530+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:38:25.010530+0000","last_undegraded":"2024-09-15T22:38:25.010530+0000","last_fullsized":"2024-09-15T22:38:25.010530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241899+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:38:25.241899+0000","last_peered":"2024-09-15T22:38:25.241899+0000","last_clean":"2024-09-15T22:38:25.241899+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:38:25.241899+0000","last_undegraded":"2024-09-15T22:38:25.241899+0000","last_fullsized":"2024-09-15T22:38:25.241899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241629+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:38:25.241629+0000","last_peered":"2024-09-15T22:38:25.241629+0000","last_clean":"2024-09-15T22:38:25.241629+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:38:25.241629+0000","last_undegraded":"2024-09-15T22:38:25.241629+0000","last_fullsized":"2024-09-15T22:38:25.241629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010356+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:38:25.010356+0000","last_peered":"2024-09-15T22:38:25.010356+0000","last_clean":"2024-09-15T22:38:25.010356+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:38:25.010356+0000","last_undegraded":"2024-09-15T22:38:25.010356+0000","last_fullsized":"2024-09-15T22:38:25.010356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242363+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:38:25.242363+0000","last_peered":"2024-09-15T22:38:25.242363+0000","last_clean":"2024-09-15T22:38:25.242363+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:38:25.242363+0000","last_undegraded":"2024-09-15T22:38:25.242363+0000","last_fullsized":"2024-09-15T22:38:25.242363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248973+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:38:25.248973+0000","last_peered":"2024-09-15T22:38:25.248973+0000","last_clean":"2024-09-15T22:38:25.248973+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:38:25.248973+0000","last_undegraded":"2024-09-15T22:38:25.248973+0000","last_fullsized":"2024-09-15T22:38:25.248973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010508+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:38:25.010508+0000","last_peered":"2024-09-15T22:38:25.010508+0000","last_clean":"2024-09-15T22:38:25.010508+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:38:25.010508+0000","last_undegraded":"2024-09-15T22:38:25.010508+0000","last_fullsized":"2024-09-15T22:38:25.010508+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006902+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:38:25.006902+0000","last_peered":"2024-09-15T22:38:25.006902+0000","last_clean":"2024-09-15T22:38:25.006902+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:38:25.006902+0000","last_undegraded":"2024-09-15T22:38:25.006902+0000","last_fullsized":"2024-09-15T22:38:25.006902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009045+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:38:25.009045+0000","last_peered":"2024-09-15T22:38:25.009045+0000","last_clean":"2024-09-15T22:38:25.009045+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:38:25.009045+0000","last_undegraded":"2024-09-15T22:38:25.009045+0000","last_fullsized":"2024-09-15T22:38:25.009045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242337+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:38:25.242337+0000","last_peered":"2024-09-15T22:38:25.242337+0000","last_clean":"2024-09-15T22:38:25.242337+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:38:25.242337+0000","last_undegraded":"2024-09-15T22:38:25.242337+0000","last_fullsized":"2024-09-15T22:38:25.242337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.395057+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:38:25.395057+0000","last_peered":"2024-09-15T22:38:25.395057+0000","last_clean":"2024-09-15T22:38:25.395057+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:38:25.395057+0000","last_undegraded":"2024-09-15T22:38:25.395057+0000","last_fullsized":"2024-09-15T22:38:25.395057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009192+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:38:25.009192+0000","last_peered":"2024-09-15T22:38:25.009192+0000","last_clean":"2024-09-15T22:38:25.009192+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:38:25.009192+0000","last_undegraded":"2024-09-15T22:38:25.009192+0000","last_fullsized":"2024-09-15T22:38:25.009192+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009375+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:38:25.009375+0000","last_peered":"2024-09-15T22:38:25.009375+0000","last_clean":"2024-09-15T22:38:25.009375+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:38:25.009375+0000","last_undegraded":"2024-09-15T22:38:25.009375+0000","last_fullsized":"2024-09-15T22:38:25.009375+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112094+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:38:25.112094+0000","last_peered":"2024-09-15T22:38:25.112094+0000","last_clean":"2024-09-15T22:38:25.112094+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:38:25.112094+0000","last_undegraded":"2024-09-15T22:38:25.112094+0000","last_fullsized":"2024-09-15T22:38:25.112094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006770+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:38:25.006770+0000","last_peered":"2024-09-15T22:38:25.006770+0000","last_clean":"2024-09-15T22:38:25.006770+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:38:25.006770+0000","last_undegraded":"2024-09-15T22:38:25.006770+0000","last_fullsized":"2024-09-15T22:38:25.006770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010086+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:38:25.010086+0000","last_peered":"2024-09-15T22:38:25.010086+0000","last_clean":"2024-09-15T22:38:25.010086+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:38:25.010086+0000","last_undegraded":"2024-09-15T22:38:25.010086+0000","last_fullsized":"2024-09-15T22:38:25.010086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.539508+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:38:25.539508+0000","last_peered":"2024-09-15T22:38:25.539508+0000","last_clean":"2024-09-15T22:38:25.539508+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:38:25.539508+0000","last_undegraded":"2024-09-15T22:38:25.539508+0000","last_fullsized":"2024-09-15T22:38:25.539508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108636+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:38:25.108636+0000","last_peered":"2024-09-15T22:38:25.108636+0000","last_clean":"2024-09-15T22:38:25.108636+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:38:25.108636+0000","last_undegraded":"2024-09-15T22:38:25.108636+0000","last_fullsized":"2024-09-15T22:38:25.108636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010264+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:38:25.010264+0000","last_peered":"2024-09-15T22:38:25.010264+0000","last_clean":"2024-09-15T22:38:25.010264+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:38:25.010264+0000","last_undegraded":"2024-09-15T22:38:25.010264+0000","last_fullsized":"2024-09-15T22:38:25.010264+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009541+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:38:25.009541+0000","last_peered":"2024-09-15T22:38:25.009541+0000","last_clean":"2024-09-15T22:38:25.009541+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:38:25.009541+0000","last_undegraded":"2024-09-15T22:38:25.009541+0000","last_fullsized":"2024-09-15T22:38:25.009541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241984+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:38:25.241984+0000","last_peered":"2024-09-15T22:38:25.241984+0000","last_clean":"2024-09-15T22:38:25.241984+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:38:25.241984+0000","last_undegraded":"2024-09-15T22:38:25.241984+0000","last_fullsized":"2024-09-15T22:38:25.241984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005995+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:38:25.005995+0000","last_peered":"2024-09-15T22:38:25.005995+0000","last_clean":"2024-09-15T22:38:25.005995+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:38:25.005995+0000","last_undegraded":"2024-09-15T22:38:25.005995+0000","last_fullsized":"2024-09-15T22:38:25.005995+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241792+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:38:25.241792+0000","last_peered":"2024-09-15T22:38:25.241792+0000","last_clean":"2024-09-15T22:38:25.241792+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:38:25.241792+0000","last_undegraded":"2024-09-15T22:38:25.241792+0000","last_fullsized":"2024-09-15T22:38:25.241792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009484+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:38:25.009484+0000","last_peered":"2024-09-15T22:38:25.009484+0000","last_clean":"2024-09-15T22:38:25.009484+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:38:25.009484+0000","last_undegraded":"2024-09-15T22:38:25.009484+0000","last_fullsized":"2024-09-15T22:38:25.009484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112389+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:38:25.112389+0000","last_peered":"2024-09-15T22:38:25.112389+0000","last_clean":"2024-09-15T22:38:25.112389+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:38:25.112389+0000","last_undegraded":"2024-09-15T22:38:25.112389+0000","last_fullsized":"2024-09-15T22:38:25.112389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":294,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.537439+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:38:25.537439+0000","last_peered":"2024-09-15T22:38:25.537439+0000","last_clean":"2024-09-15T22:38:25.537439+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:38:25.537439+0000","last_undegraded":"2024-09-15T22:38:25.537439+0000","last_fullsized":"2024-09-15T22:38:25.537439+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69721,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":214,"num_read_kb":175,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107742+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:38:25.107742+0000","last_peered":"2024-09-15T22:38:25.107742+0000","last_clean":"2024-09-15T22:38:25.107742+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:38:25.107742+0000","last_undegraded":"2024-09-15T22:38:25.107742+0000","last_fullsized":"2024-09-15T22:38:25.107742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241717+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:38:25.241717+0000","last_peered":"2024-09-15T22:38:25.241717+0000","last_clean":"2024-09-15T22:38:25.241717+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:38:25.241717+0000","last_undegraded":"2024-09-15T22:38:25.241717+0000","last_fullsized":"2024-09-15T22:38:25.241717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009805+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:38:25.009805+0000","last_peered":"2024-09-15T22:38:25.009805+0000","last_clean":"2024-09-15T22:38:25.009805+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:38:25.009805+0000","last_undegraded":"2024-09-15T22:38:25.009805+0000","last_fullsized":"2024-09-15T22:38:25.009805+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241127+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:38:25.241127+0000","last_peered":"2024-09-15T22:38:25.241127+0000","last_clean":"2024-09-15T22:38:25.241127+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:38:25.241127+0000","last_undegraded":"2024-09-15T22:38:25.241127+0000","last_fullsized":"2024-09-15T22:38:25.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250624+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:38:25.250624+0000","last_peered":"2024-09-15T22:38:25.250624+0000","last_clean":"2024-09-15T22:38:25.250624+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:38:25.250624+0000","last_undegraded":"2024-09-15T22:38:25.250624+0000","last_fullsized":"2024-09-15T22:38:25.250624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010393+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:38:25.010393+0000","last_peered":"2024-09-15T22:38:25.010393+0000","last_clean":"2024-09-15T22:38:25.010393+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:38:25.010393+0000","last_undegraded":"2024-09-15T22:38:25.010393+0000","last_fullsized":"2024-09-15T22:38:25.010393+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107495+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:38:25.107495+0000","last_peered":"2024-09-15T22:38:25.107495+0000","last_clean":"2024-09-15T22:38:25.107495+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:38:25.107495+0000","last_undegraded":"2024-09-15T22:38:25.107495+0000","last_fullsized":"2024-09-15T22:38:25.107495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124524+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:38:25.124524+0000","last_peered":"2024-09-15T22:38:25.124524+0000","last_clean":"2024-09-15T22:38:25.124524+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:38:25.124524+0000","last_undegraded":"2024-09-15T22:38:25.124524+0000","last_fullsized":"2024-09-15T22:38:25.124524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112430+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:38:25.112430+0000","last_peered":"2024-09-15T22:38:25.112430+0000","last_clean":"2024-09-15T22:38:25.112430+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:38:25.112430+0000","last_undegraded":"2024-09-15T22:38:25.112430+0000","last_fullsized":"2024-09-15T22:38:25.112430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249732+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:38:25.249732+0000","last_peered":"2024-09-15T22:38:25.249732+0000","last_clean":"2024-09-15T22:38:25.249732+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:38:25.249732+0000","last_undegraded":"2024-09-15T22:38:25.249732+0000","last_fullsized":"2024-09-15T22:38:25.249732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107744+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:38:25.107744+0000","last_peered":"2024-09-15T22:38:25.107744+0000","last_clean":"2024-09-15T22:38:25.107744+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:38:25.107744+0000","last_undegraded":"2024-09-15T22:38:25.107744+0000","last_fullsized":"2024-09-15T22:38:25.107744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009479+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:38:25.009479+0000","last_peered":"2024-09-15T22:38:25.009479+0000","last_clean":"2024-09-15T22:38:25.009479+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:38:25.009479+0000","last_undegraded":"2024-09-15T22:38:25.009479+0000","last_fullsized":"2024-09-15T22:38:25.009479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242387+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:38:25.242387+0000","last_peered":"2024-09-15T22:38:25.242387+0000","last_clean":"2024-09-15T22:38:25.242387+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:38:25.242387+0000","last_undegraded":"2024-09-15T22:38:25.242387+0000","last_fullsized":"2024-09-15T22:38:25.242387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112126+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:38:25.112126+0000","last_peered":"2024-09-15T22:38:25.112126+0000","last_clean":"2024-09-15T22:38:25.112126+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:38:25.112126+0000","last_undegraded":"2024-09-15T22:38:25.112126+0000","last_fullsized":"2024-09-15T22:38:25.112126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005971+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:38:25.005971+0000","last_peered":"2024-09-15T22:38:25.005971+0000","last_clean":"2024-09-15T22:38:25.005971+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:38:25.005971+0000","last_undegraded":"2024-09-15T22:38:25.005971+0000","last_fullsized":"2024-09-15T22:38:25.005971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124202+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:38:25.124202+0000","last_peered":"2024-09-15T22:38:25.124202+0000","last_clean":"2024-09-15T22:38:25.124202+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:38:25.124202+0000","last_undegraded":"2024-09-15T22:38:25.124202+0000","last_fullsized":"2024-09-15T22:38:25.124202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009565+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:38:25.009565+0000","last_peered":"2024-09-15T22:38:25.009565+0000","last_clean":"2024-09-15T22:38:25.009565+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:38:25.009565+0000","last_undegraded":"2024-09-15T22:38:25.009565+0000","last_fullsized":"2024-09-15T22:38:25.009565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241674+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:38:25.241674+0000","last_peered":"2024-09-15T22:38:25.241674+0000","last_clean":"2024-09-15T22:38:25.241674+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:38:25.241674+0000","last_undegraded":"2024-09-15T22:38:25.241674+0000","last_fullsized":"2024-09-15T22:38:25.241674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009806+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:38:25.009806+0000","last_peered":"2024-09-15T22:38:25.009806+0000","last_clean":"2024-09-15T22:38:25.009806+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:38:25.009806+0000","last_undegraded":"2024-09-15T22:38:25.009806+0000","last_fullsized":"2024-09-15T22:38:25.009806+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112093+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:38:25.112093+0000","last_peered":"2024-09-15T22:38:25.112093+0000","last_clean":"2024-09-15T22:38:25.112093+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:38:25.112093+0000","last_undegraded":"2024-09-15T22:38:25.112093+0000","last_fullsized":"2024-09-15T22:38:25.112093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241736+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:38:25.241736+0000","last_peered":"2024-09-15T22:38:25.241736+0000","last_clean":"2024-09-15T22:38:25.241736+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:38:25.241736+0000","last_undegraded":"2024-09-15T22:38:25.241736+0000","last_fullsized":"2024-09-15T22:38:25.241736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241163+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:38:25.241163+0000","last_peered":"2024-09-15T22:38:25.241163+0000","last_clean":"2024-09-15T22:38:25.241163+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:38:25.241163+0000","last_undegraded":"2024-09-15T22:38:25.241163+0000","last_fullsized":"2024-09-15T22:38:25.241163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241675+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:38:25.241675+0000","last_peered":"2024-09-15T22:38:25.241675+0000","last_clean":"2024-09-15T22:38:25.241675+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:38:25.241675+0000","last_undegraded":"2024-09-15T22:38:25.241675+0000","last_fullsized":"2024-09-15T22:38:25.241675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241517+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:38:25.241517+0000","last_peered":"2024-09-15T22:38:25.241517+0000","last_clean":"2024-09-15T22:38:25.241517+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:38:25.241517+0000","last_undegraded":"2024-09-15T22:38:25.241517+0000","last_fullsized":"2024-09-15T22:38:25.241517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249023+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:38:25.249023+0000","last_peered":"2024-09-15T22:38:25.249023+0000","last_clean":"2024-09-15T22:38:25.249023+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:38:25.249023+0000","last_undegraded":"2024-09-15T22:38:25.249023+0000","last_fullsized":"2024-09-15T22:38:25.249023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006558+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:38:25.006558+0000","last_peered":"2024-09-15T22:38:25.006558+0000","last_clean":"2024-09-15T22:38:25.006558+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:38:25.006558+0000","last_undegraded":"2024-09-15T22:38:25.006558+0000","last_fullsized":"2024-09-15T22:38:25.006558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242143+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:38:25.242143+0000","last_peered":"2024-09-15T22:38:25.242143+0000","last_clean":"2024-09-15T22:38:25.242143+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:38:25.242143+0000","last_undegraded":"2024-09-15T22:38:25.242143+0000","last_fullsized":"2024-09-15T22:38:25.242143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241544+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:38:25.241544+0000","last_peered":"2024-09-15T22:38:25.241544+0000","last_clean":"2024-09-15T22:38:25.241544+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:38:25.241544+0000","last_undegraded":"2024-09-15T22:38:25.241544+0000","last_fullsized":"2024-09-15T22:38:25.241544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115695+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:38:25.115695+0000","last_peered":"2024-09-15T22:38:25.115695+0000","last_clean":"2024-09-15T22:38:25.115695+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:38:25.115695+0000","last_undegraded":"2024-09-15T22:38:25.115695+0000","last_fullsized":"2024-09-15T22:38:25.115695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009267+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:38:25.009267+0000","last_peered":"2024-09-15T22:38:25.009267+0000","last_clean":"2024-09-15T22:38:25.009267+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:38:25.009267+0000","last_undegraded":"2024-09-15T22:38:25.009267+0000","last_fullsized":"2024-09-15T22:38:25.009267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112660+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:38:25.112660+0000","last_peered":"2024-09-15T22:38:25.112660+0000","last_clean":"2024-09-15T22:38:25.112660+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:38:25.112660+0000","last_undegraded":"2024-09-15T22:38:25.112660+0000","last_fullsized":"2024-09-15T22:38:25.112660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241756+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:38:25.241756+0000","last_peered":"2024-09-15T22:38:25.241756+0000","last_clean":"2024-09-15T22:38:25.241756+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:38:25.241756+0000","last_undegraded":"2024-09-15T22:38:25.241756+0000","last_fullsized":"2024-09-15T22:38:25.241756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006219+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:38:25.006219+0000","last_peered":"2024-09-15T22:38:25.006219+0000","last_clean":"2024-09-15T22:38:25.006219+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:38:25.006219+0000","last_undegraded":"2024-09-15T22:38:25.006219+0000","last_fullsized":"2024-09-15T22:38:25.006219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242055+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:38:25.242055+0000","last_peered":"2024-09-15T22:38:25.242055+0000","last_clean":"2024-09-15T22:38:25.242055+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:38:25.242055+0000","last_undegraded":"2024-09-15T22:38:25.242055+0000","last_fullsized":"2024-09-15T22:38:25.242055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241921+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:38:25.241921+0000","last_peered":"2024-09-15T22:38:25.241921+0000","last_clean":"2024-09-15T22:38:25.241921+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:38:25.241921+0000","last_undegraded":"2024-09-15T22:38:25.241921+0000","last_fullsized":"2024-09-15T22:38:25.241921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.109165+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:38:25.109165+0000","last_peered":"2024-09-15T22:38:25.109165+0000","last_clean":"2024-09-15T22:38:25.109165+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:38:25.109165+0000","last_undegraded":"2024-09-15T22:38:25.109165+0000","last_fullsized":"2024-09-15T22:38:25.109165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242550+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:38:25.242550+0000","last_peered":"2024-09-15T22:38:25.242550+0000","last_clean":"2024-09-15T22:38:25.242550+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:38:25.242550+0000","last_undegraded":"2024-09-15T22:38:25.242550+0000","last_fullsized":"2024-09-15T22:38:25.242550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241031+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:38:25.241031+0000","last_peered":"2024-09-15T22:38:25.241031+0000","last_clean":"2024-09-15T22:38:25.241031+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:38:25.241031+0000","last_undegraded":"2024-09-15T22:38:25.241031+0000","last_fullsized":"2024-09-15T22:38:25.241031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248274+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:38:25.248274+0000","last_peered":"2024-09-15T22:38:25.248274+0000","last_clean":"2024-09-15T22:38:25.248274+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:38:25.248274+0000","last_undegraded":"2024-09-15T22:38:25.248274+0000","last_fullsized":"2024-09-15T22:38:25.248274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248580+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:38:25.248580+0000","last_peered":"2024-09-15T22:38:25.248580+0000","last_clean":"2024-09-15T22:38:25.248580+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:38:25.248580+0000","last_undegraded":"2024-09-15T22:38:25.248580+0000","last_fullsized":"2024-09-15T22:38:25.248580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006155+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:38:25.006155+0000","last_peered":"2024-09-15T22:38:25.006155+0000","last_clean":"2024-09-15T22:38:25.006155+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:38:25.006155+0000","last_undegraded":"2024-09-15T22:38:25.006155+0000","last_fullsized":"2024-09-15T22:38:25.006155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241453+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:38:25.241453+0000","last_peered":"2024-09-15T22:38:25.241453+0000","last_clean":"2024-09-15T22:38:25.241453+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:38:25.241453+0000","last_undegraded":"2024-09-15T22:38:25.241453+0000","last_fullsized":"2024-09-15T22:38:25.241453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010699+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:38:25.010699+0000","last_peered":"2024-09-15T22:38:25.010699+0000","last_clean":"2024-09-15T22:38:25.010699+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:38:25.010699+0000","last_undegraded":"2024-09-15T22:38:25.010699+0000","last_fullsized":"2024-09-15T22:38:25.010699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124494+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:38:25.124494+0000","last_peered":"2024-09-15T22:38:25.124494+0000","last_clean":"2024-09-15T22:38:25.124494+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:38:25.124494+0000","last_undegraded":"2024-09-15T22:38:25.124494+0000","last_fullsized":"2024-09-15T22:38:25.124494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112097+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:38:25.112097+0000","last_peered":"2024-09-15T22:38:25.112097+0000","last_clean":"2024-09-15T22:38:25.112097+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:38:25.112097+0000","last_undegraded":"2024-09-15T22:38:25.112097+0000","last_fullsized":"2024-09-15T22:38:25.112097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010114+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:38:25.010114+0000","last_peered":"2024-09-15T22:38:25.010114+0000","last_clean":"2024-09-15T22:38:25.010114+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:38:25.010114+0000","last_undegraded":"2024-09-15T22:38:25.010114+0000","last_fullsized":"2024-09-15T22:38:25.010114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124146+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:38:25.124146+0000","last_peered":"2024-09-15T22:38:25.124146+0000","last_clean":"2024-09-15T22:38:25.124146+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:38:25.124146+0000","last_undegraded":"2024-09-15T22:38:25.124146+0000","last_fullsized":"2024-09-15T22:38:25.124146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112085+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:38:25.112085+0000","last_peered":"2024-09-15T22:38:25.112085+0000","last_clean":"2024-09-15T22:38:25.112085+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:38:25.112085+0000","last_undegraded":"2024-09-15T22:38:25.112085+0000","last_fullsized":"2024-09-15T22:38:25.112085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108245+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:38:25.108245+0000","last_peered":"2024-09-15T22:38:25.108245+0000","last_clean":"2024-09-15T22:38:25.108245+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:38:25.108245+0000","last_undegraded":"2024-09-15T22:38:25.108245+0000","last_fullsized":"2024-09-15T22:38:25.108245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249437+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:38:25.249437+0000","last_peered":"2024-09-15T22:38:25.249437+0000","last_clean":"2024-09-15T22:38:25.249437+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:38:25.249437+0000","last_undegraded":"2024-09-15T22:38:25.249437+0000","last_fullsized":"2024-09-15T22:38:25.249437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249830+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:38:25.249830+0000","last_peered":"2024-09-15T22:38:25.249830+0000","last_clean":"2024-09-15T22:38:25.249830+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:38:25.249830+0000","last_undegraded":"2024-09-15T22:38:25.249830+0000","last_fullsized":"2024-09-15T22:38:25.249830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242437+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:38:25.242437+0000","last_peered":"2024-09-15T22:38:25.242437+0000","last_clean":"2024-09-15T22:38:25.242437+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:38:25.242437+0000","last_undegraded":"2024-09-15T22:38:25.242437+0000","last_fullsized":"2024-09-15T22:38:25.242437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599775,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":730,"num_read_kb":476,"num_write":298,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1799325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612648,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":519982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841581,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28852,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716300,"statfs":{"total":95995035648,"available":95965491200,"internally_reserved":0,"allocated":1871872,"data_stored":1309665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005107,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36768,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708384,"statfs":{"total":95995035648,"available":95957385216,"internally_reserved":0,"allocated":1454080,"data_stored":917530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36572,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708580,"statfs":{"total":95995035648,"available":95957585920,"internally_reserved":0,"allocated":1253376,"data_stored":716746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1650688,"data_stored":1116498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":515394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855431,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1794048,"data_stored":1243378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986252,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":521250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1040404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":586797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":584239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":516782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:38:28.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:28.858 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 -- 172.21.15.16:0/4049142324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794101360 msgr2=0x7f479410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 --2- 172.21.15.16:0/4049142324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794101360 0x7f479410df90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4784009900 tx=0x7f478402a410 comp rx=0 tx=0).stop 2024-09-15T22:38:29.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 -- 172.21.15.16:0/4049142324 shutdown_connections 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 --2- 172.21.15.16:0/4049142324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f479410e4d0 0x7f47941108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 --2- 172.21.15.16:0/4049142324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4794101360 0x7f479410df90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 --2- 172.21.15.16:0/4049142324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f47941009b0 0x7f4794100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 -- 172.21.15.16:0/4049142324 >> 172.21.15.16:0/4049142324 conn(0x7f47940fc7d0 msgr2=0x7f47940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 -- 172.21.15.16:0/4049142324 shutdown_connections 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.366+0000 7f479b416640 1 -- 172.21.15.16:0/4049142324 wait complete. 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.367+0000 7f479b416640 1 Processor -- start 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.367+0000 7f479b416640 1 -- start start 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.367+0000 7f479b416640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47941009b0 0x7f479419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.367+0000 7f479b416640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4794101360 0x7f479419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f479b416640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479410e4d0 0x7f479419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f479b416640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4794114220 con 0x7f479410e4d0 2024-09-15T22:38:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f479b416640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f47941140a0 con 0x7f47941009b0 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f479b416640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f47941143a0 con 0x7f4794101360 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47999b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479410e4d0 0x7f479419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47991b2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47941009b0 0x7f479419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47999b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479410e4d0 0x7f479419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46898/0 (socket says 172.21.15.16:46898) 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47999b3640 1 -- 172.21.15.16:0/11780653 learned_addr learned my addr 172.21.15.16:0/11780653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47989b1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4794101360 0x7f479419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47991b2640 1 -- 172.21.15.16:0/11780653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4794101360 msgr2=0x7f479419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47991b2640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4794101360 0x7f479419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47991b2640 1 -- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479410e4d0 msgr2=0x7f479419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47991b2640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479410e4d0 0x7f479419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47991b2640 1 -- 172.21.15.16:0/11780653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479419f6f0 con 0x7f47941009b0 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47999b3640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479410e4d0 0x7f479419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.368+0000 7f47991b2640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47941009b0 0x7f479419a740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f479000e9f0 tx=0x7f479000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.369+0000 7f478a7fc640 1 -- 172.21.15.16:0/11780653 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479000ccc0 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.369+0000 7f478a7fc640 1 -- 172.21.15.16:0/11780653 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4790005020 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.369+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479419f9e0 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.369+0000 7f478a7fc640 1 -- 172.21.15.16:0/11780653 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47900049e0 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.369+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941a72c0 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.370+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4794100d90 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.371+0000 7f478a7fc640 1 -- 172.21.15.16:0/11780653 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f4790005560 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.372+0000 7f478a7fc640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f47800a3440 0x7f47800a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.372+0000 7f47989b1640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f47800a3440 0x7f47800a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.372+0000 7f478a7fc640 1 -- 172.21.15.16:0/11780653 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f47900d0430 con 0x7f47941009b0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.372+0000 7f47989b1640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f47800a3440 0x7f47800a5900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4784009230 tx=0x7f47840058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.375+0000 7f478a7fc640 1 -- 172.21.15.16:0/11780653 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47900051d0 con 0x7f47941009b0 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi059 to 177.0M 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: Unable to set osd_memory_target on smithi059 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:29.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:29 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi059 to 177.0M 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: Unable to set osd_memory_target on smithi059 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:29.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:29 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.594+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f479419baa0 con 0x7f47941009b0 2024-09-15T22:38:29.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.595+0000 7f478a7fc640 1 -- 172.21.15.16:0/11780653 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f47900bae20 con 0x7f47941009b0 2024-09-15T22:38:29.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:29.596 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:38:29.597 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:38:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f47800a3440 msgr2=0x7f47800a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:29.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f47800a3440 0x7f47800a5900 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4784009230 tx=0x7f47840058d0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47941009b0 msgr2=0x7f479419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47941009b0 0x7f479419a740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f479000e9f0 tx=0x7f479000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 shutdown_connections 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f479410e4d0 0x7f479419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f47800a3440 0x7f47800a5900 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4794101360 0x7f479419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 --2- 172.21.15.16:0/11780653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47941009b0 0x7f479419a740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 >> 172.21.15.16:0/11780653 conn(0x7f47940fc7d0 msgr2=0x7f47940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 shutdown_connections 2024-09-15T22:38:29.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:29.597+0000 7f479b416640 1 -- 172.21.15.16:0/11780653 wait complete. 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi059 to 177.0M 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: Unable to set osd_memory_target on smithi059 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:29.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:29.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:29.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:29 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:30.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Unable to set osd_memory_target on smithi016 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi130:/etc/ceph/ceph.conf 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/11780653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:30 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-15T22:38:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Unable to set osd_memory_target on smithi016 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-15T22:38:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-15T22:38:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-15T22:38:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi130:/etc/ceph/ceph.conf 2024-09-15T22:38:30.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/11780653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:30 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Unable to set osd_memory_target on smithi016 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi059:/etc/ceph/ceph.conf 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi130:/etc/ceph/ceph.conf 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/11780653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi130:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:30.697 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:30 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 -- 172.21.15.16:0/2987962428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc102790 msgr2=0x7f67dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 --2- 172.21.15.16:0/2987962428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc102790 0x7f67dc1067e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc02a410 comp rx=0 tx=0).stop 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 -- 172.21.15.16:0/2987962428 shutdown_connections 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 --2- 172.21.15.16:0/2987962428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc106f10 0x7f67dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 --2- 172.21.15.16:0/2987962428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc102790 0x7f67dc1067e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 --2- 172.21.15.16:0/2987962428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc101de0 0x7f67dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 -- 172.21.15.16:0/2987962428 >> 172.21.15.16:0/2987962428 conn(0x7f67dc0fd8c0 msgr2=0x7f67dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 -- 172.21.15.16:0/2987962428 shutdown_connections 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.186+0000 7f67e4732640 1 -- 172.21.15.16:0/2987962428 wait complete. 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 Processor -- start 2024-09-15T22:38:31.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 -- start start 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc101de0 0x7f67dc1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc102790 0x7f67dc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc111f70 con 0x7f67dc106f10 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc111df0 con 0x7f67dc102790 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e4732640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1120f0 con 0x7f67dc101de0 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e2ccf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e2ccf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46914/0 (socket says 172.21.15.16:46914) 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e2ccf640 1 -- 172.21.15.16:0/3825772064 learned_addr learned my addr 172.21.15.16:0/3825772064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:31.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e24ce640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc101de0 0x7f67dc1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e1ccd640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc102790 0x7f67dc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e2ccf640 1 -- 172.21.15.16:0/3825772064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc101de0 msgr2=0x7f67dc1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e2ccf640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc101de0 0x7f67dc1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.187+0000 7f67e2ccf640 1 -- 172.21.15.16:0/3825772064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc102790 msgr2=0x7f67dc1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67e2ccf640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc102790 0x7f67dc1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67e2ccf640 1 -- 172.21.15.16:0/3825772064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc19d460 con 0x7f67dc106f10 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67e2ccf640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f67d400eaa0 tx=0x7f67d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67d37fe640 1 -- 172.21.15.16:0/3825772064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400cdb0 con 0x7f67dc106f10 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67d37fe640 1 -- 172.21.15.16:0/3825772064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4005020 con 0x7f67dc106f10 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc19d750 con 0x7f67dc106f10 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67d37fe640 1 -- 172.21.15.16:0/3825772064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40116b0 con 0x7f67dc106f10 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.188+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1a5030 con 0x7f67dc106f10 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.189+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc1021c0 con 0x7f67dc106f10 2024-09-15T22:38:31.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.190+0000 7f67d37fe640 1 -- 172.21.15.16:0/3825772064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f67d4005560 con 0x7f67dc106f10 2024-09-15T22:38:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.191+0000 7f67d37fe640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a34c0 0x7f67c80a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.191+0000 7f67d37fe640 1 -- 172.21.15.16:0/3825772064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f67d40d02f0 con 0x7f67dc106f10 2024-09-15T22:38:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.191+0000 7f67e24ce640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a34c0 0x7f67c80a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.192+0000 7f67e24ce640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a34c0 0x7f67c80a5980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67d8005fd0 tx=0x7f67d800b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:31.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.195+0000 7f67d37fe640 1 -- 172.21.15.16:0/3825772064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67d4016030 con 0x7f67dc106f10 2024-09-15T22:38:31.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.377+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67dc10a180 con 0x7f67dc106f10 2024-09-15T22:38:31.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.378+0000 7f67d37fe640 1 -- 172.21.15.16:0/3825772064 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25138 (secure 0 0 0) 0x7f67d409c1b0 con 0x7f67dc106f10 2024-09-15T22:38:31.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:31.380 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:38:25.000286+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3275020767":"2024-09-16T22:38:25.000235+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3377264779":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3984482177":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/1746300554":"2024-09-16T22:38:25.000235+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:38:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a34c0 msgr2=0x7f67c80a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a34c0 0x7f67c80a5980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67d8005fd0 tx=0x7f67d800b390 comp rx=0 tx=0).stop 2024-09-15T22:38:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106f10 msgr2=0x7f67dc19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f67d400eaa0 tx=0x7f67d400ef60 comp rx=0 tx=0).stop 2024-09-15T22:38:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 shutdown_connections 2024-09-15T22:38:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc102790 0x7f67dc1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a34c0 0x7f67c80a5980 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 --2- 172.21.15.16:0/3825772064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc101de0 0x7f67dc1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 >> 172.21.15.16:0/3825772064 conn(0x7f67dc0fd8c0 msgr2=0x7f67dc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 shutdown_connections 2024-09-15T22:38:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:31.381+0000 7f67e4732640 1 -- 172.21.15.16:0/3825772064 wait complete. 2024-09-15T22:38:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:31 smithi130 ceph-mon[35742]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:31 smithi130 ceph-mon[35742]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:31 smithi130 ceph-mon[35742]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:31 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:31 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:31 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:31.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:31 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:31.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:31 smithi016 ceph-mon[31492]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:31 smithi016 ceph-mon[31492]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:31 smithi016 ceph-mon[31492]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:31 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:31.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:31 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:31.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:31 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:31.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:31 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:31 smithi059 ceph-mon[34872]: Updating smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:31 smithi059 ceph-mon[34872]: Updating smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:31 smithi059 ceph-mon[34872]: Updating smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:38:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:31 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:31 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:31 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:31.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:31 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:32.074 DEBUG:tasks.fwd_scrub:Yielding 2024-09-15T22:38:32.074 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:38:32.084 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:38:32.084 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:38:32.084 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-15T22:38:32.084 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-15T22:38:32.085 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:38:32.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:38:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 -- 172.21.15.16:0/1677683783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 msgr2=0x7fdb1c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 --2- 172.21.15.16:0/1677683783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c106da0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009930 tx=0x7fdb1802a420 comp rx=0 tx=0).stop 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 -- 172.21.15.16:0/1677683783 shutdown_connections 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 --2- 172.21.15.16:0/1677683783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c101350 0x7fdb1c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 --2- 172.21.15.16:0/1677683783 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb1c1009b0 0x7fdb1c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 --2- 172.21.15.16:0/1677683783 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c106da0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 -- 172.21.15.16:0/1677683783 >> 172.21.15.16:0/1677683783 conn(0x7fdb1c077990 msgr2=0x7fdb1c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 -- 172.21.15.16:0/1677683783 shutdown_connections 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 -- 172.21.15.16:0/1677683783 wait complete. 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 Processor -- start 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.193+0000 7fdb234ba640 1 -- start start 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb234ba640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb1c1009b0 0x7fdb1c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb234ba640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c101350 0x7fdb1c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb234ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb234ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c110e00 con 0x7fdb1c1069c0 2024-09-15T22:38:32.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb234ba640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c110c80 con 0x7fdb1c101350 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb234ba640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c110f80 con 0x7fdb1c1009b0 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54144/0 (socket says 172.21.15.16:54144) 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 -- 172.21.15.16:0/3045635466 learned_addr learned my addr 172.21.15.16:0/3045635466 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21256640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb1c1009b0 0x7fdb1c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb20a55640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c101350 0x7fdb1c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 -- 172.21.15.16:0/3045635466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb1c1009b0 msgr2=0x7fdb1c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb1c1009b0 0x7fdb1c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 -- 172.21.15.16:0/3045635466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c101350 msgr2=0x7fdb1c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c101350 0x7fdb1c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 -- 172.21.15.16:0/3045635466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c19f530 con 0x7fdb1c1069c0 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.194+0000 7fdb21a57640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c19ee00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400cce0 tx=0x7fdb1400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.195+0000 7fdb127fc640 1 -- 172.21.15.16:0/3045635466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400f450 con 0x7fdb1c1069c0 2024-09-15T22:38:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.195+0000 7fdb127fc640 1 -- 172.21.15.16:0/3045635466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb1400fad0 con 0x7fdb1c1069c0 2024-09-15T22:38:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.195+0000 7fdb127fc640 1 -- 172.21.15.16:0/3045635466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14018c10 con 0x7fdb1c1069c0 2024-09-15T22:38:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.195+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c19f820 con 0x7fdb1c1069c0 2024-09-15T22:38:32.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.195+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1a70b0 con 0x7fdb1c1069c0 2024-09-15T22:38:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.196+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c100e10 con 0x7fdb1c1069c0 2024-09-15T22:38:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.197+0000 7fdb127fc640 1 -- 172.21.15.16:0/3045635466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7fdb1400f5f0 con 0x7fdb1c1069c0 2024-09-15T22:38:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.198+0000 7fdb127fc640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdb080a3500 0x7fdb080a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.198+0000 7fdb21256640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdb080a3500 0x7fdb080a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.198+0000 7fdb127fc640 1 -- 172.21.15.16:0/3045635466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fdb140d0850 con 0x7fdb1c1069c0 2024-09-15T22:38:32.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.198+0000 7fdb21256640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdb080a3500 0x7fdb080a59c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009900 tx=0x7fdb180058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.202+0000 7fdb127fc640 1 -- 172.21.15.16:0/3045635466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb14014030 con 0x7fdb1c1069c0 2024-09-15T22:38:32.357 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:32.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.395+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fdb1c1a7350 con 0x7fdb1c1069c0 2024-09-15T22:38:32.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.399+0000 7fdb127fc640 1 -- 172.21.15.16:0/3045635466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v32)=0 v32) ==== 129+0+0 (secure 0 0 0) 0x7fdb1409cd00 con 0x7fdb1c1069c0 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdb080a3500 msgr2=0x7fdb080a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdb080a3500 0x7fdb080a59c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009900 tx=0x7fdb180058d0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 msgr2=0x7fdb1c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c19ee00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400cce0 tx=0x7fdb1400a620 comp rx=0 tx=0).stop 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 shutdown_connections 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb1c1069c0 0x7fdb1c19ee00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb1c101350 0x7fdb1c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdb080a3500 0x7fdb080a59c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 --2- 172.21.15.16:0/3045635466 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdb1c1009b0 0x7fdb1c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 >> 172.21.15.16:0/3045635466 conn(0x7fdb1c077990 msgr2=0x7fdb1c0698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 shutdown_connections 2024-09-15T22:38:32.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.401+0000 7fdb234ba640 1 -- 172.21.15.16:0/3045635466 wait complete. 2024-09-15T22:38:32.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-15T22:38:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.520+0000 7f0567c42640 1 -- 172.21.15.16:0/1945116813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 msgr2=0x7f0560100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.520+0000 7f0567c42640 1 --2- 172.21.15.16:0/1945116813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560100e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0550009930 tx=0x7f055002a420 comp rx=0 tx=0).stop 2024-09-15T22:38:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 -- 172.21.15.16:0/1945116813 shutdown_connections 2024-09-15T22:38:32.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 --2- 172.21.15.16:0/1945116813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0560101370 0x7f056010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 --2- 172.21.15.16:0/1945116813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560100e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 --2- 172.21.15.16:0/1945116813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05601069e0 0x7f0560106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 -- 172.21.15.16:0/1945116813 >> 172.21.15.16:0/1945116813 conn(0x7f05600fc6f0 msgr2=0x7f05600feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 -- 172.21.15.16:0/1945116813 shutdown_connections 2024-09-15T22:38:32.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 -- 172.21.15.16:0/1945116813 wait complete. 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 Processor -- start 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.521+0000 7f0567c42640 1 -- start start 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0567c42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0567c42640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0560101370 0x7f05601988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0567c42640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05601069e0 0x7f056019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0567c42640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0560110de0 con 0x7f05601009d0 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0567c42640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0560110c60 con 0x7f0560101370 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0567c42640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0560110f60 con 0x7f05601069e0 2024-09-15T22:38:32.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54162/0 (socket says 172.21.15.16:54162) 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 -- 172.21.15.16:0/4138255176 learned_addr learned my addr 172.21.15.16:0/4138255176 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05661df640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05601069e0 0x7f056019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05651dd640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0560101370 0x7f05601988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 -- 172.21.15.16:0/4138255176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05601069e0 msgr2=0x7f056019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05601069e0 0x7f056019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 -- 172.21.15.16:0/4138255176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0560101370 msgr2=0x7f05601988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0560101370 0x7f05601988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 -- 172.21.15.16:0/4138255176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056019d350 con 0x7f05601009d0 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05651dd640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0560101370 0x7f05601988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f05659de640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560198360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f055c00d950 tx=0x7f055c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0556ffd640 1 -- 172.21.15.16:0/4138255176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c017030 con 0x7f05601009d0 2024-09-15T22:38:32.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0556ffd640 1 -- 172.21.15.16:0/4138255176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f055c004020 con 0x7f05601009d0 2024-09-15T22:38:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056019d640 con 0x7f05601009d0 2024-09-15T22:38:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.522+0000 7f0556ffd640 1 -- 172.21.15.16:0/4138255176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c0047a0 con 0x7f05601009d0 2024-09-15T22:38:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.523+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05601a4ed0 con 0x7f05601009d0 2024-09-15T22:38:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.524+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0560102110 con 0x7f05601009d0 2024-09-15T22:38:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.525+0000 7f0556ffd640 1 -- 172.21.15.16:0/4138255176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f055c0041d0 con 0x7f05601009d0 2024-09-15T22:38:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.525+0000 7f0556ffd640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f054c0a3580 0x7f054c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.526+0000 7f0556ffd640 1 -- 172.21.15.16:0/4138255176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f055c0cfaf0 con 0x7f05601009d0 2024-09-15T22:38:32.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.526+0000 7f05651dd640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f054c0a3580 0x7f054c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.526+0000 7f05651dd640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f054c0a3580 0x7f054c0a5a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0550000c00 tx=0x7f0550032c70 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.530+0000 7f0556ffd640 1 -- 172.21.15.16:0/4138255176 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f055c09c000 con 0x7f05601009d0 2024-09-15T22:38:32.632 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:32 smithi016 ceph-mon[31492]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-09-15T22:38:32.633 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:32 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3825772064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:32.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:32 smithi059 ceph-mon[34872]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-09-15T22:38:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:32 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3825772064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.721+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f05601a51d0 con 0x7f05601009d0 2024-09-15T22:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.725+0000 7f0556ffd640 1 -- 172.21.15.16:0/4138255176 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7f055c09b900 con 0x7f05601009d0 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f054c0a3580 msgr2=0x7f054c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f054c0a3580 0x7f054c0a5a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0550000c00 tx=0x7f0550032c70 comp rx=0 tx=0).stop 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 msgr2=0x7f0560198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560198360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f055c00d950 tx=0x7f055c00de10 comp rx=0 tx=0).stop 2024-09-15T22:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 shutdown_connections 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f054c0a3580 0x7f054c0a5a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f05601069e0 0x7f056019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0560101370 0x7f05601988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 --2- 172.21.15.16:0/4138255176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05601009d0 0x7f0560198360 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 >> 172.21.15.16:0/4138255176 conn(0x7f05600fc6f0 msgr2=0x7f0560104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 shutdown_connections 2024-09-15T22:38:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.728+0000 7f0567c42640 1 -- 172.21.15.16:0/4138255176 wait complete. 2024-09-15T22:38:32.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-15T22:38:32.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:32 smithi130 ceph-mon[35742]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-09-15T22:38:32.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:32 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3825772064' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 -- 172.21.15.16:0/3098427014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9981009b0 msgr2=0x7fc998100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3098427014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9981009b0 0x7fc998100e10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc980009930 tx=0x7fc98002a420 comp rx=0 tx=0).stop 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 -- 172.21.15.16:0/3098427014 shutdown_connections 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3098427014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc998101350 0x7fc99810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3098427014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9981009b0 0x7fc998100e10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3098427014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9981069c0 0x7fc998106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 -- 172.21.15.16:0/3098427014 >> 172.21.15.16:0/3098427014 conn(0x7fc998077990 msgr2=0x7fc9980fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 -- 172.21.15.16:0/3098427014 shutdown_connections 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 -- 172.21.15.16:0/3098427014 wait complete. 2024-09-15T22:38:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.849+0000 7fc99d21e640 1 Processor -- start 2024-09-15T22:38:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99d21e640 1 -- start start 2024-09-15T22:38:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99d21e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9981009b0 0x7fc9981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99d21e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc998101350 0x7fc9981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99d21e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9981069c0 0x7fc99819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99d21e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc998110e00 con 0x7fc998101350 2024-09-15T22:38:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99d21e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc998110c80 con 0x7fc9981009b0 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99d21e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc998110f80 con 0x7fc9981069c0 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc998101350 0x7fc9981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc998101350 0x7fc9981988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54186/0 (socket says 172.21.15.16:54186) 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 -- 172.21.15.16:0/3675583531 learned_addr learned my addr 172.21.15.16:0/3675583531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99759e640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9981069c0 0x7fc99819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc996d9d640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9981009b0 0x7fc9981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 -- 172.21.15.16:0/3675583531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9981069c0 msgr2=0x7fc99819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9981069c0 0x7fc99819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 -- 172.21.15.16:0/3675583531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9981009b0 msgr2=0x7fc9981983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9981009b0 0x7fc9981983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 -- 172.21.15.16:0/3675583531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99819d3a0 con 0x7fc998101350 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.850+0000 7fc99659c640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc998101350 0x7fc9981988f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc980009230 tx=0x7fc980004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.851+0000 7fc98ffff640 1 -- 172.21.15.16:0/3675583531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98003f030 con 0x7fc998101350 2024-09-15T22:38:32.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.851+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99819d630 con 0x7fc998101350 2024-09-15T22:38:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.851+0000 7fc98ffff640 1 -- 172.21.15.16:0/3675583531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc980004440 con 0x7fc998101350 2024-09-15T22:38:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.851+0000 7fc98ffff640 1 -- 172.21.15.16:0/3675583531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980035070 con 0x7fc998101350 2024-09-15T22:38:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.851+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9981a4f20 con 0x7fc998101350 2024-09-15T22:38:32.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.852+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc998100e10 con 0x7fc998101350 2024-09-15T22:38:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.853+0000 7fc98ffff640 1 -- 172.21.15.16:0/3675583531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7fc980042030 con 0x7fc998101350 2024-09-15T22:38:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.854+0000 7fc98ffff640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc9840a3500 0x7fc9840a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.854+0000 7fc98ffff640 1 -- 172.21.15.16:0/3675583531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fc9800f05b0 con 0x7fc998101350 2024-09-15T22:38:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.854+0000 7fc996d9d640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc9840a3500 0x7fc9840a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.855+0000 7fc996d9d640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc9840a3500 0x7fc9840a59c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ae0 tx=0x7fc988005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.858+0000 7fc98ffff640 1 -- 172.21.15.16:0/3675583531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9800bcac0 con 0x7fc998101350 2024-09-15T22:38:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 -- 172.21.15.16:0/1303758447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f810c170 msgr2=0x7f40f810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 --2- 172.21.15.16:0/1303758447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f810c170 0x7f40f810e610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009be0 tx=0x7f40f002a750 comp rx=0 tx=0).stop 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 -- 172.21.15.16:0/1303758447 shutdown_connections 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 --2- 172.21.15.16:0/1303758447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f810c170 0x7f40f810e610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 --2- 172.21.15.16:0/1303758447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f8103660 0x7f40f810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 --2- 172.21.15.16:0/1303758447 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40f8102d40 0x7f40f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 -- 172.21.15.16:0/1303758447 >> 172.21.15.16:0/1303758447 conn(0x7f40f80fc7a0 msgr2=0x7f40f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 -- 172.21.15.16:0/1303758447 shutdown_connections 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.872+0000 7f40fe545640 1 -- 172.21.15.16:0/1303758447 wait complete. 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 Processor -- start 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 -- start start 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40f8102d40 0x7f40f81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f8103660 0x7f40f8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f810c170 0x7f40f819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8111ee0 con 0x7f40f810c170 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8111d60 con 0x7f40f8103660 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fe545640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8112060 con 0x7f40f8102d40 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fcae2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f810c170 0x7f40f819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fcae2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f810c170 0x7f40f819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54212/0 (socket says 172.21.15.16:54212) 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.873+0000 7f40fcae2640 1 -- 172.21.15.16:0/2859587904 learned_addr learned my addr 172.21.15.16:0/2859587904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40f7fff640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40f8102d40 0x7f40f81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40f77fe640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f8103660 0x7f40f8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fcae2640 1 -- 172.21.15.16:0/2859587904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40f8102d40 msgr2=0x7f40f81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fcae2640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40f8102d40 0x7f40f81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fcae2640 1 -- 172.21.15.16:0/2859587904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f8103660 msgr2=0x7f40f8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fcae2640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f8103660 0x7f40f8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fcae2640 1 -- 172.21.15.16:0/2859587904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f819d390 con 0x7f40f810c170 2024-09-15T22:38:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fcae2640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f810c170 0x7f40f819ccb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d00 tx=0x7f40f0030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40f57fa640 1 -- 172.21.15.16:0/2859587904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0041020 con 0x7f40f810c170 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40f57fa640 1 -- 172.21.15.16:0/2859587904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f0037040 con 0x7f40f810c170 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40f57fa640 1 -- 172.21.15.16:0/2859587904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0036070 con 0x7f40f810c170 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f819d620 con 0x7f40f810c170 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.874+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f819dab0 con 0x7f40f810c170 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.877+0000 7f410091e740 1 Processor -- start 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.877+0000 7f40f57fa640 1 -- 172.21.15.16:0/2859587904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f40f0044020 con 0x7f40f810c170 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.877+0000 7f410091e740 1 -- start start 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.877+0000 7f410091e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2128cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.877+0000 7f410091e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf21297800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f410091e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f410091e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf211f2280 con 0x55cf21297d40 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f410091e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cf21199b10 con 0x55cf2128cb60 2024-09-15T22:38:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f7fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf21297800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54228/0 (socket says 172.21.15.16:54228) 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 -- 172.21.15.16:0/4194870916 learned_addr learned my addr 172.21.15.16:0/4194870916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f57fa640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40e00a34c0 0x7f40e00a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f410091e740 1 -- 172.21.15.16:0/4194870916 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cf211ee240 con 0x55cf2128d480 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40fcae2640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40e00a34c0 0x7f40e00a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2128cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f57fa640 1 -- 172.21.15.16:0/2859587904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f40f00f0ca0 con 0x7f40f810c170 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 -- 172.21.15.16:0/4194870916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 msgr2=0x55cf21297800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf21297800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 -- 172.21.15.16:0/4194870916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 msgr2=0x55cf2128cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2128cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f77fe640 1 -- 172.21.15.16:0/4194870916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf211a61f0 con 0x55cf21297d40 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.878+0000 7f40f7fff640 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf21297800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.879+0000 7f40f77fe640 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129a130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005c80 tx=0x7f40ec005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c23ed1384b4e73f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.879+0000 7f40fcae2640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40e00a34c0 0x7f40e00a5980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40f00ee030 tx=0x7f40f00b8700 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.879+0000 7f40de7fc640 1 -- 172.21.15.16:0/4194870916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec013e10 con 0x55cf21297d40 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.879+0000 7f40de7fc640 1 -- 172.21.15.16:0/4194870916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ec0117e0 con 0x55cf21297d40 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.879+0000 7f40de7fc640 1 -- 172.21.15.16:0/4194870916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0036e0 con 0x55cf21297d40 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.879+0000 7f410091e740 1 -- 172.21.15.16:0/4194870916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 msgr2=0x55cf2129a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129a130 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005c80 tx=0x7f40ec005850 comp rx=0 tx=0).stop 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 -- 172.21.15.16:0/4194870916 shutdown_connections 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129a130 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf21297800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 --2- 172.21.15.16:0/4194870916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2128cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 -- 172.21.15.16:0/4194870916 >> 172.21.15.16:0/4194870916 conn(0x55cf212890f0 msgr2=0x55cf212894d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 -- 172.21.15.16:0/4194870916 shutdown_connections 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 -- 172.21.15.16:0/4194870916 wait complete. 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 -- start start 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.880+0000 7f410091e740 1 Processor -- start 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128cb60 0x55cf21294c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128d480 0x55cf21295180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf21297d40 0x55cf212956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cf2129a910 con 0x55cf2128cb60 2024-09-15T22:38:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cf211acf10 con 0x55cf2128d480 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cf2129aa90 con 0x55cf21297d40 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 10 client.0 _ll_register_callbacks cb 0x55cf21175a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128cb60 0x55cf21294c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128cb60 0x55cf21294c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54244/0 (socket says 172.21.15.16:54244) 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 -- 172.21.15.16:0/767803055 learned_addr learned my addr 172.21.15.16:0/767803055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f7fff640 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128d480 0x55cf21295180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 -- 172.21.15.16:0/767803055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf21297d40 msgr2=0x55cf212956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf21297d40 0x55cf212956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 -- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128d480 msgr2=0x55cf21295180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128d480 0x55cf21295180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 -- 172.21.15.16:0/767803055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf20f636a0 con 0x55cf2128cb60 2024-09-15T22:38:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f7fff640 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128d480 0x55cf21295180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40f77fe640 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128cb60 0x55cf21294c40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0044e0 tx=0x7f40ec005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c4b6982a4ffaac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40d77fe640 1 -- 172.21.15.16:0/767803055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec005240 con 0x55cf2128cb60 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f40d77fe640 1 -- 172.21.15.16:0/767803055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f40ec0053e0 con 0x55cf2128cb60 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.881+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40ec0117e0 con 0x55cf2128cb60 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f40d77fe640 1 -- 172.21.15.16:0/767803055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40ec011dd0 con 0x55cf2128cb60 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 10 client.15387 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf212aa950 con 0x55cf2128cb60 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f40d77fe640 1 -- 172.21.15.16:0/767803055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ec017070 con 0x55cf2128cb60 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f40d77fe640 1 -- 172.21.15.16:0/767803055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40ec0039d0 con 0x55cf2128cb60 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 10 client.15387 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 20 client.15387 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf212abd50 0x55cf212ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 4 client.15387 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f40ec0117e0 con 0x55cf212abd50 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.882+0000 7f40f7fff640 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf212abd50 0x55cf212ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.883+0000 7f40f7fff640 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf212abd50 0x55cf212ae140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f93b10b2c4a23e87 server_cookie=c5d26d985cebd157 in_seq=0 out_seq=0 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.883+0000 7f40d77fe640 10 client.15387 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f40d77fe640 1 -- 172.21.15.16:0/767803055 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f40ec0117e0 con 0x55cf212abd50 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f40d77fe640 10 client.15387 handle_command_reply: tid=0 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 client.15387 shutdown 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128cb60 msgr2=0x55cf21294c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128cb60 0x55cf21294c40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0044e0 tx=0x7f40ec005970 comp rx=0 tx=0).stop 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 shutdown_connections 2024-09-15T22:38:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf212abd50 0x55cf212ae140 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf21297d40 0x55cf212956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128d480 0x55cf21295180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f40d77fe640 0 client.15387 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 --2- 172.21.15.16:0/767803055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128cb60 0x55cf21294c40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 >> 172.21.15.16:0/767803055 conn(0x55cf212890f0 msgr2=0x55cf2128edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 shutdown_connections 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 1 -- 172.21.15.16:0/767803055 wait complete. 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.884+0000 7f410091e740 20 client.15387 trim_cache size 0 max 16384 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 Processor -- start 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 -- start start 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128cb60 0x55cf21295510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128d480 0x55cf21293af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf21297d40 0x55cf212b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81af5c0 con 0x55cf2128d480 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81ace90 con 0x55cf21297d40 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f410091e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81aace0 con 0x55cf2128cb60 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128d480 0x55cf21293af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128d480 0x55cf21293af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54248/0 (socket says 172.21.15.16:54248) 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 -- 172.21.15.16:0/476685297 learned_addr learned my addr 172.21.15.16:0/476685297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f77fe640 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128cb60 0x55cf21295510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 -- 172.21.15.16:0/476685297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128cb60 msgr2=0x55cf21295510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128cb60 0x55cf21295510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 -- 172.21.15.16:0/476685297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf21297d40 msgr2=0x55cf212b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf21297d40 0x55cf212b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f7fff640 1 -- 172.21.15.16:0/476685297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf21297220 con 0x55cf2128d480 2024-09-15T22:38:32.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.902+0000 7f40f77fe640 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128cb60 0x55cf21295510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.903+0000 7f40f7fff640 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128d480 0x55cf21293af0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005d00 tx=0x7f40e800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47718f51c34d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.903+0000 7f40ddffb640 1 -- 172.21.15.16:0/476685297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8005220 con 0x55cf2128d480 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.903+0000 7f40ddffb640 1 -- 172.21.15.16:0/476685297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8004c80 con 0x55cf2128d480 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.903+0000 7f40ddffb640 1 -- 172.21.15.16:0/476685297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e801dad0 con 0x55cf2128d480 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.903+0000 7f410091e740 1 -- 172.21.15.16:0/476685297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128d480 msgr2=0x55cf21293af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.903+0000 7f410091e740 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128d480 0x55cf21293af0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f40e8005d00 tx=0x7f40e800b410 comp rx=0 tx=0).stop 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- 172.21.15.16:0/476685297 shutdown_connections 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf21297d40 0x55cf212b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf2128d480 0x55cf21293af0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 --2- 172.21.15.16:0/476685297 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128cb60 0x55cf21295510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- 172.21.15.16:0/476685297 >> 172.21.15.16:0/476685297 conn(0x55cf212890f0 msgr2=0x55cf212933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- 172.21.15.16:0/476685297 shutdown_connections 2024-09-15T22:38:32.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- 172.21.15.16:0/476685297 wait complete. 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- start start 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 Processor -- start 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2129ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf2129bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81ab350 con 0x55cf21297d40 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81ac7c0 con 0x55cf2128cb60 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.904+0000 7f410091e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81aba20 con 0x55cf2128d480 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f410091e740 10 client.0 _ll_register_callbacks cb 0x7f40ec009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54252/0 (socket says 172.21.15.16:54252) 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 -- 172.21.15.16:0/1866736425 learned_addr learned my addr 172.21.15.16:0/1866736425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2129ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f7fff640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf2129bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 -- 172.21.15.16:0/1866736425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 msgr2=0x55cf2129bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf2129bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 -- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 msgr2=0x55cf2129ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2129ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 -- 172.21.15.16:0/1866736425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf213158f0 con 0x55cf21297d40 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f7fff640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf2129bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40f77fe640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129c4b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0094f0 tx=0x7f40ec009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29204c456a9b941 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40d77fe640 1 -- 172.21.15.16:0/1866736425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0117b0 con 0x55cf21297d40 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f40d77fe640 1 -- 172.21.15.16:0/1866736425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f40ec003a30 con 0x55cf21297d40 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.905+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e8004c80 con 0x55cf21297d40 2024-09-15T22:38:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f40d77fe640 1 -- 172.21.15.16:0/1866736425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40ec042050 con 0x55cf21297d40 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 10 client.15393 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf21316c10 con 0x55cf21297d40 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f40d77fe640 1 -- 172.21.15.16:0/1866736425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8004c80 con 0x55cf21297d40 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f40d77fe640 1 -- 172.21.15.16:0/1866736425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40ec0053a0 con 0x55cf21297d40 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 10 client.15393 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 20 client.15393 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf213172d0 0x55cf213196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 4 client.15393 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cf211ff7f0 con 0x55cf213172d0 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.906+0000 7f40f7fff640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf213172d0 0x55cf213196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.907+0000 7f40f7fff640 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf213172d0 0x55cf213196f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cef1536d4f7f84a server_cookie=e1b9163ff5e6a686 in_seq=0 out_seq=0 2024-09-15T22:38:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.907+0000 7f40d77fe640 10 client.15393 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f40d77fe640 1 -- 172.21.15.16:0/1866736425 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cf211ff7f0 con 0x55cf213172d0 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f40d77fe640 10 client.15393 handle_command_reply: tid=0 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 client.15393 shutdown 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 msgr2=0x55cf2129c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129c4b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0094f0 tx=0x7f40ec009dd0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 shutdown_connections 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55cf213172d0 0x55cf213196f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f40d77fe640 0 client.15393 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cf21297d40 0x55cf2129c4b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cf2128d480 0x55cf2129bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "98cccde3-85f6-4988-9a47-dd109b32513a", 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 --2- 172.21.15.16:0/1866736425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cf2128cb60 0x55cf2129ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.908+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 >> 172.21.15.16:0/1866736425 conn(0x55cf212890f0 msgr2=0x55cf21288270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 shutdown_connections 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f410091e740 1 -- 172.21.15.16:0/1866736425 wait complete. 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f410091e740 20 client.15393 trim_cache size 0 max 16384 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40e00a34c0 msgr2=0x7f40e00a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40e00a34c0 0x7f40e00a5980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f40f00ee030 tx=0x7f40f00b8700 comp rx=0 tx=0).stop 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f810c170 msgr2=0x7f40f819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f810c170 0x7f40f819ccb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d00 tx=0x7f40f0030650 comp rx=0 tx=0).stop 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 shutdown_connections 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40f810c170 0x7f40f819ccb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40f8103660 0x7f40f8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40e00a34c0 0x7f40e00a5980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 --2- 172.21.15.16:0/2859587904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40f8102d40 0x7f40f81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.909+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 >> 172.21.15.16:0/2859587904 conn(0x7f40f80fc7a0 msgr2=0x7f40f80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.910+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 shutdown_connections 2024-09-15T22:38:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:32.910+0000 7f40fe545640 1 -- 172.21.15.16:0/2859587904 wait complete. 2024-09-15T22:38:33.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.046+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc9981a5180 con 0x7fc998101350 2024-09-15T22:38:33.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.050+0000 7fc98ffff640 1 -- 172.21.15.16:0/3675583531 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7fc9800bc3c0 con 0x7fc998101350 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc9840a3500 msgr2=0x7fc9840a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc9840a3500 0x7fc9840a59c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ae0 tx=0x7fc988005f90 comp rx=0 tx=0).stop 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc998101350 msgr2=0x7fc9981988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc998101350 0x7fc9981988f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc980009230 tx=0x7fc980004040 comp rx=0 tx=0).stop 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 shutdown_connections 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc9840a3500 0x7fc9840a59c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc9981069c0 0x7fc99819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc998101350 0x7fc9981988f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 --2- 172.21.15.16:0/3675583531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9981009b0 0x7fc9981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 >> 172.21.15.16:0/3675583531 conn(0x7fc998077990 msgr2=0x7fc9980699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 shutdown_connections 2024-09-15T22:38:33.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.052+0000 7fc99d21e640 1 -- 172.21.15.16:0/3675583531 wait complete. 2024-09-15T22:38:33.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-15T22:38:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.172+0000 7f2094de4640 1 -- 172.21.15.16:0/702397906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 msgr2=0x7f20901030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.172+0000 7f2094de4640 1 --2- 172.21.15.16:0/702397906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f20901030c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2080009900 tx=0x7f208002a480 comp rx=0 tx=0).stop 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.172+0000 7f2094de4640 1 -- 172.21.15.16:0/702397906 shutdown_connections 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.172+0000 7f2094de4640 1 --2- 172.21.15.16:0/702397906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f209010c0b0 0x7f209010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.172+0000 7f2094de4640 1 --2- 172.21.15.16:0/702397906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2090103600 0x7f209010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.172+0000 7f2094de4640 1 --2- 172.21.15.16:0/702397906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f20901030c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.172+0000 7f2094de4640 1 -- 172.21.15.16:0/702397906 >> 172.21.15.16:0/702397906 conn(0x7f2090077990 msgr2=0x7f20900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.173+0000 7f2094de4640 1 -- 172.21.15.16:0/702397906 shutdown_connections 2024-09-15T22:38:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.173+0000 7f2094de4640 1 -- 172.21.15.16:0/702397906 wait complete. 2024-09-15T22:38:33.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.173+0000 7f2094de4640 1 Processor -- start 2024-09-15T22:38:33.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.173+0000 7f2094de4640 1 -- start start 2024-09-15T22:38:33.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f2094de4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f2090100930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f2094de4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2090103600 0x7f2090100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f2094de4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f209010c0b0 0x7f20901013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f2094de4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111eb0 con 0x7f2090102ce0 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f2094de4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111d30 con 0x7f209010c0b0 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f2094de4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2090112030 con 0x7f2090103600 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f2090100930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f2090100930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54268/0 (socket says 172.21.15.16:54268) 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 -- 172.21.15.16:0/2731167049 learned_addr learned my addr 172.21.15.16:0/2731167049 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208ed9d640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f209010c0b0 0x7f20901013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208dd9b640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2090103600 0x7f2090100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 -- 172.21.15.16:0/2731167049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2090103600 msgr2=0x7f2090100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2090103600 0x7f2090100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 -- 172.21.15.16:0/2731167049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f209010c0b0 msgr2=0x7f20901013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f209010c0b0 0x7f20901013b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 -- 172.21.15.16:0/2731167049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901a6860 con 0x7f2090102ce0 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208dd9b640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2090103600 0x7f2090100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f208e59c640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f2090100930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2080006010 tx=0x7f2080004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f207f7fe640 1 -- 172.21.15.16:0/2731167049 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208003f030 con 0x7f2090102ce0 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901a6af0 con 0x7f2090102ce0 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.174+0000 7f207f7fe640 1 -- 172.21.15.16:0/2731167049 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2080003ad0 con 0x7f2090102ce0 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.175+0000 7f207f7fe640 1 -- 172.21.15.16:0/2731167049 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080035070 con 0x7f2090102ce0 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.175+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a7030 con 0x7f2090102ce0 2024-09-15T22:38:33.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.176+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20901030c0 con 0x7f2090102ce0 2024-09-15T22:38:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.177+0000 7f207f7fe640 1 -- 172.21.15.16:0/2731167049 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f2080042030 con 0x7f2090102ce0 2024-09-15T22:38:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.178+0000 7f207f7fe640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20780a3500 0x7f20780a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.178+0000 7f207f7fe640 1 -- 172.21.15.16:0/2731167049 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f20800f0bb0 con 0x7f2090102ce0 2024-09-15T22:38:33.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.178+0000 7f208dd9b640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20780a3500 0x7f20780a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.178+0000 7f208dd9b640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20780a3500 0x7f20780a59c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20901023c0 tx=0x7f2084005e50 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.181+0000 7f207f7fe640 1 -- 172.21.15.16:0/2731167049 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20800bd180 con 0x7f2090102ce0 2024-09-15T22:38:33.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.378+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2090062630 con 0x7f20780a3500 2024-09-15T22:38:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.449+0000 7f207f7fe640 1 -- 172.21.15.16:0/2731167049 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2090062630 con 0x7f20780a3500 2024-09-15T22:38:33.451 INFO:teuthology.orchestra.run.smithi016.stdout:Schedule set for path / 2024-09-15T22:38:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20780a3500 msgr2=0x7f20780a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20780a3500 0x7f20780a59c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20901023c0 tx=0x7f2084005e50 comp rx=0 tx=0).stop 2024-09-15T22:38:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 msgr2=0x7f2090100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f2090100930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2080006010 tx=0x7f2080004270 comp rx=0 tx=0).stop 2024-09-15T22:38:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 shutdown_connections 2024-09-15T22:38:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f209010c0b0 0x7f20901013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20780a3500 0x7f20780a59c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2090103600 0x7f2090100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 --2- 172.21.15.16:0/2731167049 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2090102ce0 0x7f2090100930 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 >> 172.21.15.16:0/2731167049 conn(0x7f2090077990 msgr2=0x7f20900fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 shutdown_connections 2024-09-15T22:38:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.451+0000 7f2094de4640 1 -- 172.21.15.16:0/2731167049 wait complete. 2024-09-15T22:38:33.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-15T22:38:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.569+0000 7f67dbaae640 1 -- 172.21.15.16:0/2682660842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 msgr2=0x7f67d4100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.569+0000 7f67dbaae640 1 --2- 172.21.15.16:0/2682660842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d4100e70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009930 tx=0x7f67d002a4d0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- 172.21.15.16:0/2682660842 shutdown_connections 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 --2- 172.21.15.16:0/2682660842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d4105ec0 0x7f67d4106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 --2- 172.21.15.16:0/2682660842 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67d41013b0 0x7f67d4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 --2- 172.21.15.16:0/2682660842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d4100e70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- 172.21.15.16:0/2682660842 >> 172.21.15.16:0/2682660842 conn(0x7f67d40fc6f0 msgr2=0x7f67d40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- 172.21.15.16:0/2682660842 shutdown_connections 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- 172.21.15.16:0/2682660842 wait complete. 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 Processor -- start 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- start start 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d41a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67d41013b0 0x7f67d41a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d4105ec0 0x7f67d419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4118530 con 0x7f67d4100a90 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67d41183b0 con 0x7f67d4105ec0 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.570+0000 7f67dbaae640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67d41186b0 con 0x7f67d41013b0 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d41a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d41a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54292/0 (socket says 172.21.15.16:54292) 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 -- 172.21.15.16:0/3896751999 learned_addr learned my addr 172.21.15.16:0/3896751999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67da04b640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d4105ec0 0x7f67d419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d9049640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67d41013b0 0x7f67d41a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 -- 172.21.15.16:0/3896751999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67d41013b0 msgr2=0x7f67d41a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67d41013b0 0x7f67d41a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 -- 172.21.15.16:0/3896751999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d4105ec0 msgr2=0x7f67d419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d4105ec0 0x7f67d419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 -- 172.21.15.16:0/3896751999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d419ef50 con 0x7f67d4100a90 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67d984a640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d41a4610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009a50 tx=0x7f67d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67caffd640 1 -- 172.21.15.16:0/3896751999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d003f030 con 0x7f67d4100a90 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67caffd640 1 -- 172.21.15.16:0/3896751999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d0003ce0 con 0x7f67d4100a90 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d419f1e0 con 0x7f67d4100a90 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67caffd640 1 -- 172.21.15.16:0/3896751999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0035070 con 0x7f67d4100a90 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.571+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d41ab460 con 0x7f67d4100a90 2024-09-15T22:38:33.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.572+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d406abe0 con 0x7f67d4100a90 2024-09-15T22:38:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.573+0000 7f67caffd640 1 -- 172.21.15.16:0/3896751999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f67d0042030 con 0x7f67d4100a90 2024-09-15T22:38:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.574+0000 7f67caffd640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c00a3580 0x7f67c00a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.574+0000 7f67d9049640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c00a3580 0x7f67c00a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.574+0000 7f67caffd640 1 -- 172.21.15.16:0/3896751999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f67d00f03e0 con 0x7f67d4100a90 2024-09-15T22:38:33.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.575+0000 7f67d9049640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c00a3580 0x7f67c00a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67d40ffba0 tx=0x7f67c4008040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:33.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.578+0000 7f67caffd640 1 -- 172.21.15.16:0/3896751999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67d00bc8f0 con 0x7f67d4100a90 2024-09-15T22:38:33.608 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3045635466' entity='client.admin' 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4138255176' entity='client.admin' 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3675583531' entity='client.admin' 2024-09-15T22:38:33.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.695 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3045635466' entity='client.admin' 2024-09-15T22:38:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4138255176' entity='client.admin' 2024-09-15T22:38:33.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3675583531' entity='client.admin' 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.781+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f67d4104f60 con 0x7f67c00a3580 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3045635466' entity='client.admin' 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4138255176' entity='client.admin' 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3675583531' entity='client.admin' 2024-09-15T22:38:33.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:33.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:33.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:33.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:38:33.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.844+0000 7f67caffd640 1 -- 172.21.15.16:0/3896751999 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f67d4104f60 con 0x7f67c00a3580 2024-09-15T22:38:33.846 INFO:teuthology.orchestra.run.smithi016.stdout:Retention added to path / 2024-09-15T22:38:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c00a3580 msgr2=0x7f67c00a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c00a3580 0x7f67c00a5a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67d40ffba0 tx=0x7f67c4008040 comp rx=0 tx=0).stop 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 msgr2=0x7f67d41a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d41a4610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009a50 tx=0x7f67d0004270 comp rx=0 tx=0).stop 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 shutdown_connections 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d4105ec0 0x7f67d419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c00a3580 0x7f67c00a5a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67d41013b0 0x7f67d41a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 --2- 172.21.15.16:0/3896751999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d4100a90 0x7f67d41a4610 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 >> 172.21.15.16:0/3896751999 conn(0x7f67d40fc6f0 msgr2=0x7f67d4103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 shutdown_connections 2024-09-15T22:38:33.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.847+0000 7f67dbaae640 1 -- 172.21.15.16:0/3896751999 wait complete. 2024-09-15T22:38:33.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-15T22:38:33.874 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.972+0000 7f2771037640 1 -- 172.21.15.16:0/1847390917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c10c0f0 msgr2=0x7f276c10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.972+0000 7f2771037640 1 --2- 172.21.15.16:0/1847390917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c10c0f0 0x7f276c10e560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2764009ca0 tx=0x7f276402a950 comp rx=0 tx=0).stop 2024-09-15T22:38:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 -- 172.21.15.16:0/1847390917 shutdown_connections 2024-09-15T22:38:33.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 --2- 172.21.15.16:0/1847390917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c10c0f0 0x7f276c10e560 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 --2- 172.21.15.16:0/1847390917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c103640 0x7f276c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 --2- 172.21.15.16:0/1847390917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f276c102d20 0x7f276c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 -- 172.21.15.16:0/1847390917 >> 172.21.15.16:0/1847390917 conn(0x7f276c077990 msgr2=0x7f276c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 -- 172.21.15.16:0/1847390917 shutdown_connections 2024-09-15T22:38:33.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 -- 172.21.15.16:0/1847390917 wait complete. 2024-09-15T22:38:33.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 Processor -- start 2024-09-15T22:38:33.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.973+0000 7f2771037640 1 -- start start 2024-09-15T22:38:33.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f2771037640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c102d20 0x7f276c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f2771037640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c103640 0x7f276c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f2771037640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f276c10c0f0 0x7f276c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f2771037640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f276c111ec0 con 0x7f276c102d20 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f2771037640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f276c111d40 con 0x7f276c103640 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f2771037640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f276c112040 con 0x7f276c10c0f0 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c102d20 0x7f276c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c102d20 0x7f276c1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54302/0 (socket says 172.21.15.16:54302) 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 -- 172.21.15.16:0/3706305637 learned_addr learned my addr 172.21.15.16:0/3706305637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276b59e640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f276c10c0f0 0x7f276c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276a59c640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c103640 0x7f276c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 -- 172.21.15.16:0/3706305637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f276c10c0f0 msgr2=0x7f276c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f276c10c0f0 0x7f276c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 -- 172.21.15.16:0/3706305637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c103640 msgr2=0x7f276c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:33.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c103640 0x7f276c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 -- 172.21.15.16:0/3706305637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c19d2a0 con 0x7f276c102d20 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276a59c640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c103640 0x7f276c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.974+0000 7f276ad9d640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c102d20 0x7f276c1982b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f275400c970 tx=0x7f275400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.975+0000 7f2763fff640 1 -- 172.21.15.16:0/3706305637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754007bb0 con 0x7f276c102d20 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.975+0000 7f2763fff640 1 -- 172.21.15.16:0/3706305637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2754007d50 con 0x7f276c102d20 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.975+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c19d590 con 0x7f276c102d20 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.975+0000 7f2763fff640 1 -- 172.21.15.16:0/3706305637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275400f450 con 0x7f276c102d20 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.975+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c1a4e20 con 0x7f276c102d20 2024-09-15T22:38:33.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.976+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c103100 con 0x7f276c102d20 2024-09-15T22:38:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.977+0000 7f2763fff640 1 -- 172.21.15.16:0/3706305637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f2754004fe0 con 0x7f276c102d20 2024-09-15T22:38:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.977+0000 7f2763fff640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f27580a3580 0x7f27580a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.978+0000 7f276a59c640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f27580a3580 0x7f27580a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.978+0000 7f2763fff640 1 -- 172.21.15.16:0/3706305637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f27540d0420 con 0x7f276c102d20 2024-09-15T22:38:33.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.978+0000 7f276a59c640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f27580a3580 0x7f27580a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f275c005e00 tx=0x7f275c005c70 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:33.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:33.980+0000 7f2763fff640 1 -- 172.21.15.16:0/3706305637 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2754014030 con 0x7f276c102d20 2024-09-15T22:38:34.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.181+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f276c0fd5f0 con 0x7f27580a3580 2024-09-15T22:38:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.187+0000 7f2763fff640 1 -- 172.21.15.16:0/3706305637 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f276c0fd5f0 con 0x7f27580a3580 2024-09-15T22:38:34.189 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-15T00:00:00", "created": "2024-09-15T22:38:33", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-15T22:38:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.189+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f27580a3580 msgr2=0x7f27580a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.189+0000 7f2771037640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f27580a3580 0x7f27580a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f275c005e00 tx=0x7f275c005c70 comp rx=0 tx=0).stop 2024-09-15T22:38:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.189+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c102d20 msgr2=0x7f276c1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.189+0000 7f2771037640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c102d20 0x7f276c1982b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f275400c970 tx=0x7f275400ce30 comp rx=0 tx=0).stop 2024-09-15T22:38:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.189+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 shutdown_connections 2024-09-15T22:38:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.189+0000 7f2771037640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f27580a3580 0x7f27580a5a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.190+0000 7f2771037640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f276c10c0f0 0x7f276c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.190+0000 7f2771037640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f276c103640 0x7f276c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.190+0000 7f2771037640 1 --2- 172.21.15.16:0/3706305637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f276c102d20 0x7f276c1982b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.190+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 >> 172.21.15.16:0/3706305637 conn(0x7f276c077990 msgr2=0x7f276c0fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.190+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 shutdown_connections 2024-09-15T22:38:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.190+0000 7f2771037640 1 -- 172.21.15.16:0/3706305637 wait complete. 2024-09-15T22:38:34.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-15T22:38:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 -- 172.21.15.16:0/1041725225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1810c0b0 msgr2=0x7f2c1810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 --2- 172.21.15.16:0/1041725225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1810c0b0 0x7f2c1810e550 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009ca0 tx=0x7f2c1002a950 comp rx=0 tx=0).stop 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 -- 172.21.15.16:0/1041725225 shutdown_connections 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 --2- 172.21.15.16:0/1041725225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1810c0b0 0x7f2c1810e550 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 --2- 172.21.15.16:0/1041725225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c18103600 0x7f2c1810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 --2- 172.21.15.16:0/1041725225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c18102ce0 0x7f2c181030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 -- 172.21.15.16:0/1041725225 >> 172.21.15.16:0/1041725225 conn(0x7f2c18077990 msgr2=0x7f2c180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 -- 172.21.15.16:0/1041725225 shutdown_connections 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 -- 172.21.15.16:0/1041725225 wait complete. 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 Processor -- start 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.350+0000 7f2c20037640 1 -- start start 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c20037640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c18102ce0 0x7f2c181981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c20037640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c18103600 0x7f2c18198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c20037640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c1810c0b0 0x7f2c1819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c20037640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18111e70 con 0x7f2c18103600 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c20037640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18111cf0 con 0x7f2c18102ce0 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c20037640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18111ff0 con 0x7f2c1810c0b0 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c18103600 0x7f2c18198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c18103600 0x7f2c18198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54306/0 (socket says 172.21.15.16:54306) 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 -- 172.21.15.16:0/2424093537 learned_addr learned my addr 172.21.15.16:0/2424093537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1e5d4640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c1810c0b0 0x7f2c1819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1ddd3640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c18102ce0 0x7f2c181981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 -- 172.21.15.16:0/2424093537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c1810c0b0 msgr2=0x7f2c1819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c1810c0b0 0x7f2c1819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 -- 172.21.15.16:0/2424093537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c18102ce0 msgr2=0x7f2c181981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c18102ce0 0x7f2c181981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 -- 172.21.15.16:0/2424093537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1819d1d0 con 0x7f2c18103600 2024-09-15T22:38:34.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1e5d4640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c1810c0b0 0x7f2c1819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1ddd3640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c18102ce0 0x7f2c181981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.351+0000 7f2c1d5d2640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c18103600 0x7f2c18198720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00c920 tx=0x7f2c0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.352+0000 7f2c0affd640 1 -- 172.21.15.16:0/2424093537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c007c70 con 0x7f2c18103600 2024-09-15T22:38:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.352+0000 7f2c0affd640 1 -- 172.21.15.16:0/2424093537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c0c007e10 con 0x7f2c18103600 2024-09-15T22:38:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.352+0000 7f2c0affd640 1 -- 172.21.15.16:0/2424093537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c018890 con 0x7f2c18103600 2024-09-15T22:38:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.352+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1819d4c0 con 0x7f2c18103600 2024-09-15T22:38:34.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.352+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a4d50 con 0x7f2c18103600 2024-09-15T22:38:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.353+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c181030c0 con 0x7f2c18103600 2024-09-15T22:38:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.355+0000 7f2c0affd640 1 -- 172.21.15.16:0/2424093537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f2c0c018a70 con 0x7f2c18103600 2024-09-15T22:38:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.355+0000 7f2c0affd640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2c040a3580 0x7f2c040a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.356+0000 7f2c1ddd3640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2c040a3580 0x7f2c040a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.356+0000 7f2c0affd640 1 -- 172.21.15.16:0/2424093537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f2c0c0d0320 con 0x7f2c18103600 2024-09-15T22:38:34.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.356+0000 7f2c1ddd3640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2c040a3580 0x7f2c040a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400abe0 tx=0x7f2c1400a5c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.360+0000 7f2c0affd640 1 -- 172.21.15.16:0/2424093537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c0c014030 con 0x7f2c18103600 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 -- 172.21.15.16:0/3712141848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc102d30 msgr2=0x7f61bc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 --2- 172.21.15.16:0/3712141848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc102d30 0x7f61bc103110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009cd0 tx=0x7f61ac02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 -- 172.21.15.16:0/3712141848 shutdown_connections 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 --2- 172.21.15.16:0/3712141848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61bc1082a0 0x7f61bc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 --2- 172.21.15.16:0/3712141848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61bc1036e0 0x7f61bc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 --2- 172.21.15.16:0/3712141848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc102d30 0x7f61bc103110 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 -- 172.21.15.16:0/3712141848 >> 172.21.15.16:0/3712141848 conn(0x7f61bc0fc7d0 msgr2=0x7f61bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 -- 172.21.15.16:0/3712141848 shutdown_connections 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.386+0000 7f61c1166640 1 -- 172.21.15.16:0/3712141848 wait complete. 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 Processor -- start 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 -- start start 2024-09-15T22:38:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61bc102d30 0x7f61bc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc1036e0 0x7f61bc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61bc1082a0 0x7f61bc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1120a0 con 0x7f61bc1036e0 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc111f20 con 0x7f61bc1082a0 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61c1166640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc112220 con 0x7f61bc102d30 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.387+0000 7f61ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc1036e0 0x7f61bc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc1036e0 0x7f61bc198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54320/0 (socket says 172.21.15.16:54320) 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 -- 172.21.15.16:0/2102229067 learned_addr learned my addr 172.21.15.16:0/2102229067 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61bad9d640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61bc102d30 0x7f61bc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61bb59e640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61bc1082a0 0x7f61bc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 -- 172.21.15.16:0/2102229067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61bc102d30 msgr2=0x7f61bc1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61bc102d30 0x7f61bc1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 -- 172.21.15.16:0/2102229067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61bc1082a0 msgr2=0x7f61bc19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61bc1082a0 0x7f61bc19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 -- 172.21.15.16:0/2102229067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61bc19d590 con 0x7f61bc1036e0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61ba59c640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc1036e0 0x7f61bc198b20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f61b000d750 tx=0x7f61b000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61a3fff640 1 -- 172.21.15.16:0/2102229067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0017070 con 0x7f61bc1036e0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61bc19d880 con 0x7f61bc1036e0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61a3fff640 1 -- 172.21.15.16:0/2102229067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b0005500 con 0x7f61bc1036e0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61a3fff640 1 -- 172.21.15.16:0/2102229067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004e00 con 0x7f61bc1036e0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.388+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61bc1a5160 con 0x7f61bc1036e0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61a3fff640 1 -- 172.21.15.16:0/2102229067 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f61b0010460 con 0x7f61bc1036e0 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 Processor -- start 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 -- start start 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe336360 0x55dcbe336740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe345550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe345a90 0x55dcbe347e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dcbe29c2e0 con 0x55dcbe345a90 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcbe243ce0 con 0x55dcbe336c80 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.391+0000 7f61c353f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dcbe2982a0 con 0x55dcbe336360 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe345a90 0x55dcbe347e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe345550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe345550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:54846/0 (socket says 172.21.15.16:54846) 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe345a90 0x55dcbe347e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54336/0 (socket says 172.21.15.16:54336) 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 -- 172.21.15.16:0/268682405 learned_addr learned my addr 172.21.15.16:0/268682405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bb59e640 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe336360 0x55dcbe336740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61a3fff640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f61a40a3440 0x7f61a40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 -- 172.21.15.16:0/268682405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe336360 msgr2=0x55dcbe336740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe336360 0x55dcbe336740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 -- 172.21.15.16:0/268682405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe345a90 msgr2=0x55dcbe347e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe345a90 0x55dcbe347e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 -- 172.21.15.16:0/268682405 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbe228440 con 0x55dcbe336c80 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61ba59c640 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe345a90 0x55dcbe347e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bb59e640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f61a40a3440 0x7f61a40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61a3fff640 1 -- 172.21.15.16:0/2102229067 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f61b00cfb20 con 0x7f61bc1036e0 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.392+0000 7f61bad9d640 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe345550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009cd0 tx=0x7f61ac005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b819e4a07785fba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61a17fa640 1 -- 172.21.15.16:0/268682405 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac048070 con 0x55dcbe336c80 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61bb59e640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f61a40a3440 0x7f61a40a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005fd0 tx=0x7f61b40046c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61a17fa640 1 -- 172.21.15.16:0/268682405 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61ac036070 con 0x55dcbe336c80 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61a17fa640 1 -- 172.21.15.16:0/268682405 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac004520 con 0x55dcbe336c80 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 -- 172.21.15.16:0/268682405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 msgr2=0x55dcbe345550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe345550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009cd0 tx=0x7f61ac005fa0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 -- 172.21.15.16:0/268682405 shutdown_connections 2024-09-15T22:38:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe345a90 0x55dcbe347e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe345550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 --2- 172.21.15.16:0/268682405 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe336360 0x55dcbe336740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 -- 172.21.15.16:0/268682405 >> 172.21.15.16:0/268682405 conn(0x55dcbe332a30 msgr2=0x55dcbe332e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 -- 172.21.15.16:0/268682405 shutdown_connections 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.393+0000 7f61c353f740 1 -- 172.21.15.16:0/268682405 wait complete. 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 -- start start 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 Processor -- start 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336360 0x55dcbe3453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe342670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe342bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dcbe348600 con 0x55dcbe336360 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55dcbe257050 con 0x55dcbe336c80 2024-09-15T22:38:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55dcbe348780 con 0x55dcbe345a90 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61c353f740 10 client.0 _ll_register_callbacks cb 0x55dcbe297160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336360 0x55dcbe3453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61bad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336360 0x55dcbe3453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54346/0 (socket says 172.21.15.16:54346) 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61bad9d640 1 -- 172.21.15.16:0/805933904 learned_addr learned my addr 172.21.15.16:0/805933904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61ba59c640 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe342670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.394+0000 7f61bb59e640 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe342bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61bad9d640 1 -- 172.21.15.16:0/805933904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 msgr2=0x55dcbe342bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61bad9d640 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe342bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61bad9d640 1 -- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 msgr2=0x55dcbe342670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61bad9d640 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe342670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61bad9d640 1 -- 172.21.15.16:0/805933904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbe00d3c0 con 0x55dcbe336360 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61bad9d640 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336360 0x55dcbe3453c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f61ac046040 tx=0x7f61ac0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445fba213febfd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f619eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f619eff5640 1 -- 172.21.15.16:0/805933904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ac048040 con 0x55dcbe336360 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f619eff5640 1 -- 172.21.15.16:0/805933904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f61ac004730 con 0x55dcbe336360 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61ac036070 con 0x55dcbe336360 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f619eff5640 1 -- 172.21.15.16:0/805933904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61ac05c050 con 0x55dcbe336360 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61c353f740 10 client.15429 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dcbe348600 con 0x55dcbe336360 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.395+0000 7f619eff5640 1 -- 172.21.15.16:0/805933904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61ac036070 con 0x55dcbe336360 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f619eff5640 1 -- 172.21.15.16:0/805933904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f61ac04b030 con 0x55dcbe336360 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61c353f740 10 client.15429 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61c353f740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61c353f740 20 client.15429 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61c353f740 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe355170 0x55dcbe357560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61c353f740 4 client.15429 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcbe2a9850 con 0x55dcbe355170 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61ba59c640 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe355170 0x55dcbe357560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f619eff5640 10 client.15429 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.396+0000 7f61ba59c640 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe355170 0x55dcbe357560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0fff424c4035c1 server_cookie=f95bb2b780f96175 in_seq=0 out_seq=0 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f619eff5640 1 -- 172.21.15.16:0/805933904 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcbe2a9850 con 0x55dcbe355170 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f619eff5640 10 client.15429 handle_command_reply: tid=0 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 client.15429 shutdown 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336360 msgr2=0x55dcbe3453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336360 0x55dcbe3453c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f61ac046040 tx=0x7f61ac0040e0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 shutdown_connections 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe355170 0x55dcbe357560 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe342bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336c80 0x55dcbe342670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 --2- 172.21.15.16:0/805933904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336360 0x55dcbe3453c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 >> 172.21.15.16:0/805933904 conn(0x55dcbe332a30 msgr2=0x55dcbe338790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.397+0000 7f619eff5640 0 client.15429 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.398+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 shutdown_connections 2024-09-15T22:38:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.398+0000 7f61c353f740 1 -- 172.21.15.16:0/805933904 wait complete. 2024-09-15T22:38:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.398+0000 7f61c353f740 20 client.15429 trim_cache size 0 max 16384 2024-09-15T22:38:34.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:34 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:38:34.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:34 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:34.422 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:34 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:38:34.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:34 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:34.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:34 smithi016 ceph-mon[31492]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-15T22:38:34.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:34 smithi016 ceph-mon[31492]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:34.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:34 smithi016 ceph-mon[31492]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 Processor -- start 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 -- start start 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe3518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336c80 0x55dcbe3445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe35cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1af4e0 con 0x55dcbe336c80 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1acdb0 con 0x55dcbe336360 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61c353f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1aac00 con 0x55dcbe345a90 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336c80 0x55dcbe3445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61ba59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336c80 0x55dcbe3445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54362/0 (socket says 172.21.15.16:54362) 2024-09-15T22:38:34.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61ba59c640 1 -- 172.21.15.16:0/4236089073 learned_addr learned my addr 172.21.15.16:0/4236089073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61bad9d640 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe3518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.416+0000 7f61bb59e640 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe35cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61ba59c640 1 -- 172.21.15.16:0/4236089073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 msgr2=0x55dcbe35cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61ba59c640 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe35cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61ba59c640 1 -- 172.21.15.16:0/4236089073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 msgr2=0x55dcbe3518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61ba59c640 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe3518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61ba59c640 1 -- 172.21.15.16:0/4236089073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcbe340bc0 con 0x55dcbe336c80 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61bad9d640 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe3518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61bb59e640 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe35cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61ba59c640 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336c80 0x55dcbe3445e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f61b00983b0 tx=0x7f61b00d2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a3306f6af87e76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61a0ff9640 1 -- 172.21.15.16:0/4236089073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0020040 con 0x55dcbe336c80 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61a0ff9640 1 -- 172.21.15.16:0/4236089073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b00afe50 con 0x55dcbe336c80 2024-09-15T22:38:34.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.417+0000 7f61a0ff9640 1 -- 172.21.15.16:0/4236089073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b00d4040 con 0x55dcbe336c80 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 -- 172.21.15.16:0/4236089073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336c80 msgr2=0x55dcbe3445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336c80 0x55dcbe3445e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f61b00983b0 tx=0x7f61b00d2000 comp rx=0 tx=0).stop 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 -- 172.21.15.16:0/4236089073 shutdown_connections 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe35cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe336c80 0x55dcbe3445e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 --2- 172.21.15.16:0/4236089073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe3518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 -- 172.21.15.16:0/4236089073 >> 172.21.15.16:0/4236089073 conn(0x55dcbe332a30 msgr2=0x55dcbe343ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 -- 172.21.15.16:0/4236089073 shutdown_connections 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.418+0000 7f61c353f740 1 -- 172.21.15.16:0/4236089073 wait complete. 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 -- start start 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 Processor -- start 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe349a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe349fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe34a540 0x55dcbe3bed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1ab270 con 0x55dcbe34a540 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1ac6e0 con 0x55dcbe336360 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f61bc1ab940 con 0x55dcbe345a90 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61c353f740 10 client.0 _ll_register_callbacks cb 0x7f61ac046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61bb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe34a540 0x55dcbe3bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61bb59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe34a540 0x55dcbe3bed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54372/0 (socket says 172.21.15.16:54372) 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61bb59e640 1 -- 172.21.15.16:0/3257614149 learned_addr learned my addr 172.21.15.16:0/3257614149 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61bad9d640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe349a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.419+0000 7f61ba59c640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe349fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61bb59e640 1 -- 172.21.15.16:0/3257614149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 msgr2=0x55dcbe349fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61bb59e640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe349fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61bb59e640 1 -- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 msgr2=0x55dcbe349a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61bb59e640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe349a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61bb59e640 1 -- 172.21.15.16:0/3257614149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcbe3bf4b0 con 0x55dcbe34a540 2024-09-15T22:38:34.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61ba59c640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe349fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61bad9d640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe349a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61bb59e640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe34a540 0x55dcbe3bed40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f61b4004120 tx=0x7f61b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55aae5f3374c1db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f619eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f619eff5640 1 -- 172.21.15.16:0/3257614149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b401e440 con 0x55dcbe34a540 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f619eff5640 1 -- 172.21.15.16:0/3257614149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f61b401e5e0 con 0x55dcbe34a540 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.420+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b00afe50 con 0x55dcbe34a540 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f619eff5640 1 -- 172.21.15.16:0/3257614149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61b401e7e0 con 0x55dcbe34a540 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f61c353f740 10 client.15435 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dcbe3c0800 con 0x55dcbe34a540 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f619eff5640 1 -- 172.21.15.16:0/3257614149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61b00afe50 con 0x55dcbe34a540 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f619eff5640 1 -- 172.21.15.16:0/3257614149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f61b4010b30 con 0x55dcbe34a540 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f61c353f740 10 client.15435 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:34.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f61c353f740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f61c353f740 20 client.15435 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f61c353f740 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe3c0f30 0x55dcbe3c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.421+0000 7f61c353f740 4 client.15435 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.422+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dcbe2a3d60 con 0x55dcbe3c0f30 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.422+0000 7f61bad9d640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe3c0f30 0x55dcbe3c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.422+0000 7f61bad9d640 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe3c0f30 0x55dcbe3c3350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b9603fcfd696525 server_cookie=9c571612c17a8667 in_seq=0 out_seq=0 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.422+0000 7f619eff5640 10 client.15435 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f619eff5640 1 -- 172.21.15.16:0/3257614149 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dcbe2a3d60 con 0x55dcbe3c0f30 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f619eff5640 10 client.15435 handle_command_reply: tid=0 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 client.15435 shutdown 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe34a540 msgr2=0x55dcbe3bed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe34a540 0x55dcbe3bed40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f61b4004120 tx=0x7f61b4004730 comp rx=0 tx=0).stop 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 shutdown_connections 2024-09-15T22:38:34.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55dcbe3c0f30 0x55dcbe3c3350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcbe34a540 0x55dcbe3bed40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f619eff5640 0 client.15435 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55dcbe345a90 0x55dcbe349fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 --2- 172.21.15.16:0/3257614149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dcbe336360 0x55dcbe349a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 >> 172.21.15.16:0/3257614149 conn(0x55dcbe332a30 msgr2=0x55dcbe3446f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 shutdown_connections 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 1 -- 172.21.15.16:0/3257614149 wait complete. 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.423+0000 7f61c353f740 20 client.15435 trim_cache size 0 max 16384 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:34.434 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout: "98cccde3-85f6-4988-9a47-dd109b32513a": { 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "98cccde3-85f6-4988-9a47-dd109b32513a", 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:38:34.435 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f61a40a3440 msgr2=0x7f61a40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f61a40a3440 0x7f61a40a5900 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005fd0 tx=0x7f61b40046c0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc1036e0 msgr2=0x7f61bc198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc1036e0 0x7f61bc198b20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f61b000d750 tx=0x7f61b000dc10 comp rx=0 tx=0).stop 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 shutdown_connections 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61bc1082a0 0x7f61bc19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61bc1036e0 0x7f61bc198b20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f61a40a3440 0x7f61a40a5900 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 --2- 172.21.15.16:0/2102229067 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f61bc102d30 0x7f61bc1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 >> 172.21.15.16:0/2102229067 conn(0x7f61bc0fc7d0 msgr2=0x7f61bc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 shutdown_connections 2024-09-15T22:38:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.424+0000 7f61c1166640 1 -- 172.21.15.16:0/2102229067 wait complete. 2024-09-15T22:38:34.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.573+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2c180fd420 con 0x7f2c040a3580 2024-09-15T22:38:34.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.579+0000 7f2c0affd640 1 -- 172.21.15.16:0/2424093537 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2c180fd420 con 0x7f2c040a3580 2024-09-15T22:38:34.581 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-09-15T22:38:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.581+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2c040a3580 msgr2=0x7f2c040a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.581+0000 7f2c20037640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2c040a3580 0x7f2c040a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400abe0 tx=0x7f2c1400a5c0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.581+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c18103600 msgr2=0x7f2c18198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.581+0000 7f2c20037640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c18103600 0x7f2c18198720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00c920 tx=0x7f2c0c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 shutdown_connections 2024-09-15T22:38:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2c040a3580 0x7f2c040a5a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2c1810c0b0 0x7f2c1819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c18103600 0x7f2c18198720 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 --2- 172.21.15.16:0/2424093537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c18102ce0 0x7f2c181981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 >> 172.21.15.16:0/2424093537 conn(0x7f2c18077990 msgr2=0x7f2c180fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 shutdown_connections 2024-09-15T22:38:34.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:34.582+0000 7f2c20037640 1 -- 172.21.15.16:0/2424093537 wait complete. 2024-09-15T22:38:34.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-15T22:38:34.622 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-15T22:38:34.632 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-15T22:38:34.688 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:38:34.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:38:34.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:34 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:38:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:34 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:34 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:38:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:34 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:34 smithi059 ceph-mon[34872]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-15T22:38:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:34 smithi059 ceph-mon[34872]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:34 smithi059 ceph-mon[34872]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:34 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:38:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:34 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:34 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:38:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:34 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:38:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:34 smithi130 ceph-mon[35742]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-15T22:38:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:34 smithi130 ceph-mon[35742]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:34 smithi130 ceph-mon[35742]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:35.155 INFO:tasks.cephfs.filesystem:scrub status for tag:98cccde3-85f6-4988-9a47-dd109b32513a - {'path': '/', 'tag': '98cccde3-85f6-4988-9a47-dd109b32513a', 'options': 'recursive,force'} 2024-09-15T22:38:35.277 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:35.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:35 smithi016 ceph-mon[31492]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:35.664 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:35 smithi016 ceph-mon[31492]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:35.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:35 smithi059 ceph-mon[34872]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:35.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:35 smithi059 ceph-mon[34872]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 -- 172.21.15.16:0/1215171351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc4103640 msgr2=0x7fecc410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 --2- 172.21.15.16:0/1215171351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc4103640 0x7fecc410bb30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009900 tx=0x7fecb402a480 comp rx=0 tx=0).stop 2024-09-15T22:38:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 -- 172.21.15.16:0/1215171351 shutdown_connections 2024-09-15T22:38:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 --2- 172.21.15.16:0/1215171351 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecc410c150 0x7fecc410e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 --2- 172.21.15.16:0/1215171351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc4103640 0x7fecc410bb30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 --2- 172.21.15.16:0/1215171351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecc4102d20 0x7fecc4103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 -- 172.21.15.16:0/1215171351 >> 172.21.15.16:0/1215171351 conn(0x7fecc40fc780 msgr2=0x7fecc40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 -- 172.21.15.16:0/1215171351 shutdown_connections 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.746+0000 7feccc551640 1 -- 172.21.15.16:0/1215171351 wait complete. 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 Processor -- start 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 -- start start 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecc4102d20 0x7fecc4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecc4103640 0x7fecc41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc410c150 0x7fecc419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4111f50 con 0x7fecc410c150 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4111dd0 con 0x7fecc4103640 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccc551640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41120d0 con 0x7fecc4102d20 2024-09-15T22:38:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc410c150 0x7fecc419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc410c150 0x7fecc419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54384/0 (socket says 172.21.15.16:54384) 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 -- 172.21.15.16:0/3470450192 learned_addr learned my addr 172.21.15.16:0/3470450192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7fecc9aec640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecc4103640 0x7fecc41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7fecca2ed640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecc4102d20 0x7fecc4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 -- 172.21.15.16:0/3470450192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecc4102d20 msgr2=0x7fecc4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecc4102d20 0x7fecc4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 -- 172.21.15.16:0/3470450192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecc4103640 msgr2=0x7fecc41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecc4103640 0x7fecc41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.747+0000 7feccaaee640 1 -- 172.21.15.16:0/3470450192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc419d430 con 0x7fecc410c150 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.748+0000 7feccaaee640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc410c150 0x7fecc419cd50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00eaa0 tx=0x7fecbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.748+0000 7fecbb7fe640 1 -- 172.21.15.16:0/3470450192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00cdb0 con 0x7fecc410c150 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.748+0000 7fecbb7fe640 1 -- 172.21.15.16:0/3470450192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecbc005020 con 0x7fecc410c150 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.748+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc419d720 con 0x7fecc410c150 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.748+0000 7fecbb7fe640 1 -- 172.21.15.16:0/3470450192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0116b0 con 0x7fecc410c150 2024-09-15T22:38:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.748+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc41a5000 con 0x7fecc410c150 2024-09-15T22:38:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.749+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc4103100 con 0x7fecc410c150 2024-09-15T22:38:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.750+0000 7fecbb7fe640 1 -- 172.21.15.16:0/3470450192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7fecbc005560 con 0x7fecc410c150 2024-09-15T22:38:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.751+0000 7fecbb7fe640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fecb00a34c0 0x7fecb00a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.751+0000 7fecca2ed640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fecb00a34c0 0x7fecb00a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.751+0000 7fecbb7fe640 1 -- 172.21.15.16:0/3470450192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fecbc0d03a0 con 0x7fecc410c150 2024-09-15T22:38:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.751+0000 7fecca2ed640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fecb00a34c0 0x7fecb00a5980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecc0005fd0 tx=0x7fecc000b590 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.755+0000 7fecbb7fe640 1 -- 172.21.15.16:0/3470450192 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecbc016030 con 0x7fecc410c150 2024-09-15T22:38:35.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:35 smithi130 ceph-mon[35742]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:35.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:35 smithi130 ceph-mon[35742]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:35.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.929+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fecc4062630 con 0x7fecb00a34c0 2024-09-15T22:38:35.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.938+0000 7fecbb7fe640 1 -- 172.21.15.16:0/3470450192 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339385 (secure 0 0 0) 0x7fecc4062630 con 0x7fecb00a34c0 2024-09-15T22:38:35.942 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:38:35.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fecb00a34c0 msgr2=0x7fecb00a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fecb00a34c0 0x7fecb00a5980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fecc0005fd0 tx=0x7fecc000b590 comp rx=0 tx=0).stop 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc410c150 msgr2=0x7fecc419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc410c150 0x7fecc419cd50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00eaa0 tx=0x7fecbc00ef60 comp rx=0 tx=0).stop 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 shutdown_connections 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fecc410c150 0x7fecc419cd50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecc4103640 0x7fecc41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fecb00a34c0 0x7fecb00a5980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 --2- 172.21.15.16:0/3470450192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fecc4102d20 0x7fecc4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 >> 172.21.15.16:0/3470450192 conn(0x7fecc40fc780 msgr2=0x7fecc40fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 shutdown_connections 2024-09-15T22:38:35.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:35.942+0000 7feccc551640 1 -- 172.21.15.16:0/3470450192 wait complete. 2024-09-15T22:38:36.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:36 smithi016 ceph-mon[31492]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-15T22:38:36.686 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-15T22:38:35.021376+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190377,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":997,"num_read_kb":821,"num_write":501,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":883,"ondisk_log_size":883,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":415048,"kb_used_data":15848,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124526776,"statfs":{"total":1151940427776,"available":1151515418624,"internally_reserved":0,"allocated":16228352,"data_stored":9786845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":226,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993999"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108773+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:38:25.108773+0000","last_peered":"2024-09-15T22:38:25.108773+0000","last_clean":"2024-09-15T22:38:25.108773+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:38:25.108773+0000","last_undegraded":"2024-09-15T22:38:25.108773+0000","last_fullsized":"2024-09-15T22:38:25.108773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114924+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:38:25.114924+0000","last_peered":"2024-09-15T22:38:25.114924+0000","last_clean":"2024-09-15T22:38:25.114924+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:38:25.114924+0000","last_undegraded":"2024-09-15T22:38:25.114924+0000","last_fullsized":"2024-09-15T22:38:25.114924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241365+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:38:25.241365+0000","last_peered":"2024-09-15T22:38:25.241365+0000","last_clean":"2024-09-15T22:38:25.241365+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:38:25.241365+0000","last_undegraded":"2024-09-15T22:38:25.241365+0000","last_fullsized":"2024-09-15T22:38:25.241365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115300+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:38:25.115300+0000","last_peered":"2024-09-15T22:38:25.115300+0000","last_clean":"2024-09-15T22:38:25.115300+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:38:25.115300+0000","last_undegraded":"2024-09-15T22:38:25.115300+0000","last_fullsized":"2024-09-15T22:38:25.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250490+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:38:25.250490+0000","last_peered":"2024-09-15T22:38:25.250490+0000","last_clean":"2024-09-15T22:38:25.250490+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:38:25.250490+0000","last_undegraded":"2024-09-15T22:38:25.250490+0000","last_fullsized":"2024-09-15T22:38:25.250490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240846+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:38:25.240846+0000","last_peered":"2024-09-15T22:38:25.240846+0000","last_clean":"2024-09-15T22:38:25.240846+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:38:25.240846+0000","last_undegraded":"2024-09-15T22:38:25.240846+0000","last_fullsized":"2024-09-15T22:38:25.240846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112496+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:38:25.112496+0000","last_peered":"2024-09-15T22:38:25.112496+0000","last_clean":"2024-09-15T22:38:25.112496+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:38:25.112496+0000","last_undegraded":"2024-09-15T22:38:25.112496+0000","last_fullsized":"2024-09-15T22:38:25.112496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241710+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:38:25.241710+0000","last_peered":"2024-09-15T22:38:25.241710+0000","last_clean":"2024-09-15T22:38:25.241710+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:38:25.241710+0000","last_undegraded":"2024-09-15T22:38:25.241710+0000","last_fullsized":"2024-09-15T22:38:25.241710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241001+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:38:25.241001+0000","last_peered":"2024-09-15T22:38:25.241001+0000","last_clean":"2024-09-15T22:38:25.241001+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:38:25.241001+0000","last_undegraded":"2024-09-15T22:38:25.241001+0000","last_fullsized":"2024-09-15T22:38:25.241001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112329+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:38:25.112329+0000","last_peered":"2024-09-15T22:38:25.112329+0000","last_clean":"2024-09-15T22:38:25.112329+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:38:25.112329+0000","last_undegraded":"2024-09-15T22:38:25.112329+0000","last_fullsized":"2024-09-15T22:38:25.112329+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009573+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:38:25.009573+0000","last_peered":"2024-09-15T22:38:25.009573+0000","last_clean":"2024-09-15T22:38:25.009573+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:38:25.009573+0000","last_undegraded":"2024-09-15T22:38:25.009573+0000","last_fullsized":"2024-09-15T22:38:25.009573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240840+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:38:25.240840+0000","last_peered":"2024-09-15T22:38:25.240840+0000","last_clean":"2024-09-15T22:38:25.240840+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:38:25.240840+0000","last_undegraded":"2024-09-15T22:38:25.240840+0000","last_fullsized":"2024-09-15T22:38:25.240840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010739+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:38:25.010739+0000","last_peered":"2024-09-15T22:38:25.010739+0000","last_clean":"2024-09-15T22:38:25.010739+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:38:25.010739+0000","last_undegraded":"2024-09-15T22:38:25.010739+0000","last_fullsized":"2024-09-15T22:38:25.010739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241387+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:38:25.241387+0000","last_peered":"2024-09-15T22:38:25.241387+0000","last_clean":"2024-09-15T22:38:25.241387+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:38:25.241387+0000","last_undegraded":"2024-09-15T22:38:25.241387+0000","last_fullsized":"2024-09-15T22:38:25.241387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241362+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:38:25.241362+0000","last_peered":"2024-09-15T22:38:25.241362+0000","last_clean":"2024-09-15T22:38:25.241362+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:38:25.241362+0000","last_undegraded":"2024-09-15T22:38:25.241362+0000","last_fullsized":"2024-09-15T22:38:25.241362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113745+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:38:25.113745+0000","last_peered":"2024-09-15T22:38:25.113745+0000","last_clean":"2024-09-15T22:38:25.113745+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:38:25.113745+0000","last_undegraded":"2024-09-15T22:38:25.113745+0000","last_fullsized":"2024-09-15T22:38:25.113745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241575+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:38:25.241575+0000","last_peered":"2024-09-15T22:38:25.241575+0000","last_clean":"2024-09-15T22:38:25.241575+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:38:25.241575+0000","last_undegraded":"2024-09-15T22:38:25.241575+0000","last_fullsized":"2024-09-15T22:38:25.241575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241265+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:38:25.241265+0000","last_peered":"2024-09-15T22:38:25.241265+0000","last_clean":"2024-09-15T22:38:25.241265+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:38:25.241265+0000","last_undegraded":"2024-09-15T22:38:25.241265+0000","last_fullsized":"2024-09-15T22:38:25.241265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241234+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:38:25.241234+0000","last_peered":"2024-09-15T22:38:25.241234+0000","last_clean":"2024-09-15T22:38:25.241234+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:38:25.241234+0000","last_undegraded":"2024-09-15T22:38:25.241234+0000","last_fullsized":"2024-09-15T22:38:25.241234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114246+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:38:25.114246+0000","last_peered":"2024-09-15T22:38:25.114246+0000","last_clean":"2024-09-15T22:38:25.114246+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:38:25.114246+0000","last_undegraded":"2024-09-15T22:38:25.114246+0000","last_fullsized":"2024-09-15T22:38:25.114246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108613+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:38:25.108613+0000","last_peered":"2024-09-15T22:38:25.108613+0000","last_clean":"2024-09-15T22:38:25.108613+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:38:25.108613+0000","last_undegraded":"2024-09-15T22:38:25.108613+0000","last_fullsized":"2024-09-15T22:38:25.108613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112056+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:38:25.112056+0000","last_peered":"2024-09-15T22:38:25.112056+0000","last_clean":"2024-09-15T22:38:25.112056+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:38:25.112056+0000","last_undegraded":"2024-09-15T22:38:25.112056+0000","last_fullsized":"2024-09-15T22:38:25.112056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241262+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:38:25.241262+0000","last_peered":"2024-09-15T22:38:25.241262+0000","last_clean":"2024-09-15T22:38:25.241262+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:38:25.241262+0000","last_undegraded":"2024-09-15T22:38:25.241262+0000","last_fullsized":"2024-09-15T22:38:25.241262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241293+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:38:25.241293+0000","last_peered":"2024-09-15T22:38:25.241293+0000","last_clean":"2024-09-15T22:38:25.241293+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:38:25.241293+0000","last_undegraded":"2024-09-15T22:38:25.241293+0000","last_fullsized":"2024-09-15T22:38:25.241293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249966+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:38:25.249966+0000","last_peered":"2024-09-15T22:38:25.249966+0000","last_clean":"2024-09-15T22:38:25.249966+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:38:25.249966+0000","last_undegraded":"2024-09-15T22:38:25.249966+0000","last_fullsized":"2024-09-15T22:38:25.249966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241375+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:38:25.241375+0000","last_peered":"2024-09-15T22:38:25.241375+0000","last_clean":"2024-09-15T22:38:25.241375+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:38:25.241375+0000","last_undegraded":"2024-09-15T22:38:25.241375+0000","last_fullsized":"2024-09-15T22:38:25.241375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241418+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:38:25.241418+0000","last_peered":"2024-09-15T22:38:25.241418+0000","last_clean":"2024-09-15T22:38:25.241418+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:38:25.241418+0000","last_undegraded":"2024-09-15T22:38:25.241418+0000","last_fullsized":"2024-09-15T22:38:25.241418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115826+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:38:25.115826+0000","last_peered":"2024-09-15T22:38:25.115826+0000","last_clean":"2024-09-15T22:38:25.115826+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:38:25.115826+0000","last_undegraded":"2024-09-15T22:38:25.115826+0000","last_fullsized":"2024-09-15T22:38:25.115826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'48","reported_seq":112,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.239768+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:38:25.239768+0000","last_peered":"2024-09-15T22:38:25.239768+0000","last_clean":"2024-09-15T22:38:25.239768+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:38:25.239768+0000","last_undegraded":"2024-09-15T22:38:25.239768+0000","last_fullsized":"2024-09-15T22:38:25.239768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.111983+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:38:25.111983+0000","last_peered":"2024-09-15T22:38:25.111983+0000","last_clean":"2024-09-15T22:38:25.111983+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:38:25.111983+0000","last_undegraded":"2024-09-15T22:38:25.111983+0000","last_fullsized":"2024-09-15T22:38:25.111983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006514+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:38:25.006514+0000","last_peered":"2024-09-15T22:38:25.006514+0000","last_clean":"2024-09-15T22:38:25.006514+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:38:25.006514+0000","last_undegraded":"2024-09-15T22:38:25.006514+0000","last_fullsized":"2024-09-15T22:38:25.006514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112492+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:38:25.112492+0000","last_peered":"2024-09-15T22:38:25.112492+0000","last_clean":"2024-09-15T22:38:25.112492+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:38:25.112492+0000","last_undegraded":"2024-09-15T22:38:25.112492+0000","last_fullsized":"2024-09-15T22:38:25.112492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241541+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:38:25.241541+0000","last_peered":"2024-09-15T22:38:25.241541+0000","last_clean":"2024-09-15T22:38:25.241541+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:38:25.241541+0000","last_undegraded":"2024-09-15T22:38:25.241541+0000","last_fullsized":"2024-09-15T22:38:25.241541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124170+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:38:25.124170+0000","last_peered":"2024-09-15T22:38:25.124170+0000","last_clean":"2024-09-15T22:38:25.124170+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:38:25.124170+0000","last_undegraded":"2024-09-15T22:38:25.124170+0000","last_fullsized":"2024-09-15T22:38:25.124170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.110438+0000","last_change":"2024-09-15T22:38:14.242090+0000","last_active":"2024-09-15T22:38:25.110438+0000","last_peered":"2024-09-15T22:38:25.110438+0000","last_clean":"2024-09-15T22:38:25.110438+0000","last_became_active":"2024-09-15T22:38:14.241634+0000","last_became_peered":"2024-09-15T22:38:14.241634+0000","last_unstale":"2024-09-15T22:38:25.110438+0000","last_undegraded":"2024-09-15T22:38:25.110438+0000","last_fullsized":"2024-09-15T22:38:25.110438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:13.485506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112062+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:38:25.112062+0000","last_peered":"2024-09-15T22:38:25.112062+0000","last_clean":"2024-09-15T22:38:25.112062+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:38:25.112062+0000","last_undegraded":"2024-09-15T22:38:25.112062+0000","last_fullsized":"2024-09-15T22:38:25.112062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124297+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:38:25.124297+0000","last_peered":"2024-09-15T22:38:25.124297+0000","last_clean":"2024-09-15T22:38:25.124297+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:38:25.124297+0000","last_undegraded":"2024-09-15T22:38:25.124297+0000","last_fullsized":"2024-09-15T22:38:25.124297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010822+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:38:25.010822+0000","last_peered":"2024-09-15T22:38:25.010822+0000","last_clean":"2024-09-15T22:38:25.010822+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:38:25.010822+0000","last_undegraded":"2024-09-15T22:38:25.010822+0000","last_fullsized":"2024-09-15T22:38:25.010822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006453+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:38:25.006453+0000","last_peered":"2024-09-15T22:38:25.006453+0000","last_clean":"2024-09-15T22:38:25.006453+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:38:25.006453+0000","last_undegraded":"2024-09-15T22:38:25.006453+0000","last_fullsized":"2024-09-15T22:38:25.006453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009249+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:38:25.009249+0000","last_peered":"2024-09-15T22:38:25.009249+0000","last_clean":"2024-09-15T22:38:25.009249+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:38:25.009249+0000","last_undegraded":"2024-09-15T22:38:25.009249+0000","last_fullsized":"2024-09-15T22:38:25.009249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:29.543632+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:38:29.543632+0000","last_peered":"2024-09-15T22:38:29.543632+0000","last_clean":"2024-09-15T22:38:29.543632+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:38:29.543632+0000","last_undegraded":"2024-09-15T22:38:29.543632+0000","last_fullsized":"2024-09-15T22:38:29.543632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006955+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:38:25.006955+0000","last_peered":"2024-09-15T22:38:25.006955+0000","last_clean":"2024-09-15T22:38:25.006955+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:38:25.006955+0000","last_undegraded":"2024-09-15T22:38:25.006955+0000","last_fullsized":"2024-09-15T22:38:25.006955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005813+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:38:25.005813+0000","last_peered":"2024-09-15T22:38:25.005813+0000","last_clean":"2024-09-15T22:38:25.005813+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:38:25.005813+0000","last_undegraded":"2024-09-15T22:38:25.005813+0000","last_fullsized":"2024-09-15T22:38:25.005813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010164+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:38:25.010164+0000","last_peered":"2024-09-15T22:38:25.010164+0000","last_clean":"2024-09-15T22:38:25.010164+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:38:25.010164+0000","last_undegraded":"2024-09-15T22:38:25.010164+0000","last_fullsized":"2024-09-15T22:38:25.010164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009598+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:38:25.009598+0000","last_peered":"2024-09-15T22:38:25.009598+0000","last_clean":"2024-09-15T22:38:25.009598+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:38:25.009598+0000","last_undegraded":"2024-09-15T22:38:25.009598+0000","last_fullsized":"2024-09-15T22:38:25.009598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113256+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:38:25.113256+0000","last_peered":"2024-09-15T22:38:25.113256+0000","last_clean":"2024-09-15T22:38:25.113256+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:38:25.113256+0000","last_undegraded":"2024-09-15T22:38:25.113256+0000","last_fullsized":"2024-09-15T22:38:25.113256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241238+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:38:25.241238+0000","last_peered":"2024-09-15T22:38:25.241238+0000","last_clean":"2024-09-15T22:38:25.241238+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:38:25.241238+0000","last_undegraded":"2024-09-15T22:38:25.241238+0000","last_fullsized":"2024-09-15T22:38:25.241238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":404,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:27.116484+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:38:27.116484+0000","last_peered":"2024-09-15T22:38:27.116484+0000","last_clean":"2024-09-15T22:38:27.116484+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:38:27.116484+0000","last_undegraded":"2024-09-15T22:38:27.116484+0000","last_fullsized":"2024-09-15T22:38:27.116484+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010316+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:38:25.010316+0000","last_peered":"2024-09-15T22:38:25.010316+0000","last_clean":"2024-09-15T22:38:25.010316+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:38:25.010316+0000","last_undegraded":"2024-09-15T22:38:25.010316+0000","last_fullsized":"2024-09-15T22:38:25.010316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010530+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:38:25.010530+0000","last_peered":"2024-09-15T22:38:25.010530+0000","last_clean":"2024-09-15T22:38:25.010530+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:38:25.010530+0000","last_undegraded":"2024-09-15T22:38:25.010530+0000","last_fullsized":"2024-09-15T22:38:25.010530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241899+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:38:25.241899+0000","last_peered":"2024-09-15T22:38:25.241899+0000","last_clean":"2024-09-15T22:38:25.241899+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:38:25.241899+0000","last_undegraded":"2024-09-15T22:38:25.241899+0000","last_fullsized":"2024-09-15T22:38:25.241899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241629+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:38:25.241629+0000","last_peered":"2024-09-15T22:38:25.241629+0000","last_clean":"2024-09-15T22:38:25.241629+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:38:25.241629+0000","last_undegraded":"2024-09-15T22:38:25.241629+0000","last_fullsized":"2024-09-15T22:38:25.241629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010356+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:38:25.010356+0000","last_peered":"2024-09-15T22:38:25.010356+0000","last_clean":"2024-09-15T22:38:25.010356+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:38:25.010356+0000","last_undegraded":"2024-09-15T22:38:25.010356+0000","last_fullsized":"2024-09-15T22:38:25.010356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242363+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:38:25.242363+0000","last_peered":"2024-09-15T22:38:25.242363+0000","last_clean":"2024-09-15T22:38:25.242363+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:38:25.242363+0000","last_undegraded":"2024-09-15T22:38:25.242363+0000","last_fullsized":"2024-09-15T22:38:25.242363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248973+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:38:25.248973+0000","last_peered":"2024-09-15T22:38:25.248973+0000","last_clean":"2024-09-15T22:38:25.248973+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:38:25.248973+0000","last_undegraded":"2024-09-15T22:38:25.248973+0000","last_fullsized":"2024-09-15T22:38:25.248973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010556+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:38:25.010556+0000","last_peered":"2024-09-15T22:38:25.010556+0000","last_clean":"2024-09-15T22:38:25.010556+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:38:25.010556+0000","last_undegraded":"2024-09-15T22:38:25.010556+0000","last_fullsized":"2024-09-15T22:38:25.010556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006902+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:38:25.006902+0000","last_peered":"2024-09-15T22:38:25.006902+0000","last_clean":"2024-09-15T22:38:25.006902+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:38:25.006902+0000","last_undegraded":"2024-09-15T22:38:25.006902+0000","last_fullsized":"2024-09-15T22:38:25.006902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009099+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:38:25.009099+0000","last_peered":"2024-09-15T22:38:25.009099+0000","last_clean":"2024-09-15T22:38:25.009099+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:38:25.009099+0000","last_undegraded":"2024-09-15T22:38:25.009099+0000","last_fullsized":"2024-09-15T22:38:25.009099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242337+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:38:25.242337+0000","last_peered":"2024-09-15T22:38:25.242337+0000","last_clean":"2024-09-15T22:38:25.242337+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:38:25.242337+0000","last_undegraded":"2024-09-15T22:38:25.242337+0000","last_fullsized":"2024-09-15T22:38:25.242337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.395057+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:38:25.395057+0000","last_peered":"2024-09-15T22:38:25.395057+0000","last_clean":"2024-09-15T22:38:25.395057+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:38:25.395057+0000","last_undegraded":"2024-09-15T22:38:25.395057+0000","last_fullsized":"2024-09-15T22:38:25.395057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009234+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:38:25.009234+0000","last_peered":"2024-09-15T22:38:25.009234+0000","last_clean":"2024-09-15T22:38:25.009234+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:38:25.009234+0000","last_undegraded":"2024-09-15T22:38:25.009234+0000","last_fullsized":"2024-09-15T22:38:25.009234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009429+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:38:25.009429+0000","last_peered":"2024-09-15T22:38:25.009429+0000","last_clean":"2024-09-15T22:38:25.009429+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:38:25.009429+0000","last_undegraded":"2024-09-15T22:38:25.009429+0000","last_fullsized":"2024-09-15T22:38:25.009429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112094+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:38:25.112094+0000","last_peered":"2024-09-15T22:38:25.112094+0000","last_clean":"2024-09-15T22:38:25.112094+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:38:25.112094+0000","last_undegraded":"2024-09-15T22:38:25.112094+0000","last_fullsized":"2024-09-15T22:38:25.112094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006770+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:38:25.006770+0000","last_peered":"2024-09-15T22:38:25.006770+0000","last_clean":"2024-09-15T22:38:25.006770+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:38:25.006770+0000","last_undegraded":"2024-09-15T22:38:25.006770+0000","last_fullsized":"2024-09-15T22:38:25.006770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010143+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:38:25.010143+0000","last_peered":"2024-09-15T22:38:25.010143+0000","last_clean":"2024-09-15T22:38:25.010143+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:38:25.010143+0000","last_undegraded":"2024-09-15T22:38:25.010143+0000","last_fullsized":"2024-09-15T22:38:25.010143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:29.544141+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:38:29.544141+0000","last_peered":"2024-09-15T22:38:29.544141+0000","last_clean":"2024-09-15T22:38:29.544141+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:38:29.544141+0000","last_undegraded":"2024-09-15T22:38:29.544141+0000","last_fullsized":"2024-09-15T22:38:29.544141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108636+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:38:25.108636+0000","last_peered":"2024-09-15T22:38:25.108636+0000","last_clean":"2024-09-15T22:38:25.108636+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:38:25.108636+0000","last_undegraded":"2024-09-15T22:38:25.108636+0000","last_fullsized":"2024-09-15T22:38:25.108636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010315+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:38:25.010315+0000","last_peered":"2024-09-15T22:38:25.010315+0000","last_clean":"2024-09-15T22:38:25.010315+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:38:25.010315+0000","last_undegraded":"2024-09-15T22:38:25.010315+0000","last_fullsized":"2024-09-15T22:38:25.010315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009653+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:38:25.009653+0000","last_peered":"2024-09-15T22:38:25.009653+0000","last_clean":"2024-09-15T22:38:25.009653+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:38:25.009653+0000","last_undegraded":"2024-09-15T22:38:25.009653+0000","last_fullsized":"2024-09-15T22:38:25.009653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241984+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:38:25.241984+0000","last_peered":"2024-09-15T22:38:25.241984+0000","last_clean":"2024-09-15T22:38:25.241984+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:38:25.241984+0000","last_undegraded":"2024-09-15T22:38:25.241984+0000","last_fullsized":"2024-09-15T22:38:25.241984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006052+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:38:25.006052+0000","last_peered":"2024-09-15T22:38:25.006052+0000","last_clean":"2024-09-15T22:38:25.006052+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:38:25.006052+0000","last_undegraded":"2024-09-15T22:38:25.006052+0000","last_fullsized":"2024-09-15T22:38:25.006052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241792+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:38:25.241792+0000","last_peered":"2024-09-15T22:38:25.241792+0000","last_clean":"2024-09-15T22:38:25.241792+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:38:25.241792+0000","last_undegraded":"2024-09-15T22:38:25.241792+0000","last_fullsized":"2024-09-15T22:38:25.241792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009484+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:38:25.009484+0000","last_peered":"2024-09-15T22:38:25.009484+0000","last_clean":"2024-09-15T22:38:25.009484+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:38:25.009484+0000","last_undegraded":"2024-09-15T22:38:25.009484+0000","last_fullsized":"2024-09-15T22:38:25.009484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112389+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:38:25.112389+0000","last_peered":"2024-09-15T22:38:25.112389+0000","last_clean":"2024-09-15T22:38:25.112389+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:38:25.112389+0000","last_undegraded":"2024-09-15T22:38:25.112389+0000","last_fullsized":"2024-09-15T22:38:25.112389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":305,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:30.543340+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:38:30.543340+0000","last_peered":"2024-09-15T22:38:30.543340+0000","last_clean":"2024-09-15T22:38:30.543340+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:38:30.543340+0000","last_undegraded":"2024-09-15T22:38:30.543340+0000","last_fullsized":"2024-09-15T22:38:30.543340+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69721,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":180,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107742+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:38:25.107742+0000","last_peered":"2024-09-15T22:38:25.107742+0000","last_clean":"2024-09-15T22:38:25.107742+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:38:25.107742+0000","last_undegraded":"2024-09-15T22:38:25.107742+0000","last_fullsized":"2024-09-15T22:38:25.107742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241767+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:38:25.241767+0000","last_peered":"2024-09-15T22:38:25.241767+0000","last_clean":"2024-09-15T22:38:25.241767+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:38:25.241767+0000","last_undegraded":"2024-09-15T22:38:25.241767+0000","last_fullsized":"2024-09-15T22:38:25.241767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009858+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:38:25.009858+0000","last_peered":"2024-09-15T22:38:25.009858+0000","last_clean":"2024-09-15T22:38:25.009858+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:38:25.009858+0000","last_undegraded":"2024-09-15T22:38:25.009858+0000","last_fullsized":"2024-09-15T22:38:25.009858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241127+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:38:25.241127+0000","last_peered":"2024-09-15T22:38:25.241127+0000","last_clean":"2024-09-15T22:38:25.241127+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:38:25.241127+0000","last_undegraded":"2024-09-15T22:38:25.241127+0000","last_fullsized":"2024-09-15T22:38:25.241127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250624+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:38:25.250624+0000","last_peered":"2024-09-15T22:38:25.250624+0000","last_clean":"2024-09-15T22:38:25.250624+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:38:25.250624+0000","last_undegraded":"2024-09-15T22:38:25.250624+0000","last_fullsized":"2024-09-15T22:38:25.250624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010725+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:38:25.010725+0000","last_peered":"2024-09-15T22:38:25.010725+0000","last_clean":"2024-09-15T22:38:25.010725+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:38:25.010725+0000","last_undegraded":"2024-09-15T22:38:25.010725+0000","last_fullsized":"2024-09-15T22:38:25.010725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107495+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:38:25.107495+0000","last_peered":"2024-09-15T22:38:25.107495+0000","last_clean":"2024-09-15T22:38:25.107495+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:38:25.107495+0000","last_undegraded":"2024-09-15T22:38:25.107495+0000","last_fullsized":"2024-09-15T22:38:25.107495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:30.542988+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:38:30.542988+0000","last_peered":"2024-09-15T22:38:30.542988+0000","last_clean":"2024-09-15T22:38:30.542988+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:38:30.542988+0000","last_undegraded":"2024-09-15T22:38:30.542988+0000","last_fullsized":"2024-09-15T22:38:30.542988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112430+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:38:25.112430+0000","last_peered":"2024-09-15T22:38:25.112430+0000","last_clean":"2024-09-15T22:38:25.112430+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:38:25.112430+0000","last_undegraded":"2024-09-15T22:38:25.112430+0000","last_fullsized":"2024-09-15T22:38:25.112430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249732+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:38:25.249732+0000","last_peered":"2024-09-15T22:38:25.249732+0000","last_clean":"2024-09-15T22:38:25.249732+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:38:25.249732+0000","last_undegraded":"2024-09-15T22:38:25.249732+0000","last_fullsized":"2024-09-15T22:38:25.249732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107744+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:38:25.107744+0000","last_peered":"2024-09-15T22:38:25.107744+0000","last_clean":"2024-09-15T22:38:25.107744+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:38:25.107744+0000","last_undegraded":"2024-09-15T22:38:25.107744+0000","last_fullsized":"2024-09-15T22:38:25.107744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009479+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:38:25.009479+0000","last_peered":"2024-09-15T22:38:25.009479+0000","last_clean":"2024-09-15T22:38:25.009479+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:38:25.009479+0000","last_undegraded":"2024-09-15T22:38:25.009479+0000","last_fullsized":"2024-09-15T22:38:25.009479+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242387+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:38:25.242387+0000","last_peered":"2024-09-15T22:38:25.242387+0000","last_clean":"2024-09-15T22:38:25.242387+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:38:25.242387+0000","last_undegraded":"2024-09-15T22:38:25.242387+0000","last_fullsized":"2024-09-15T22:38:25.242387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112126+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:38:25.112126+0000","last_peered":"2024-09-15T22:38:25.112126+0000","last_clean":"2024-09-15T22:38:25.112126+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:38:25.112126+0000","last_undegraded":"2024-09-15T22:38:25.112126+0000","last_fullsized":"2024-09-15T22:38:25.112126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005971+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:38:25.005971+0000","last_peered":"2024-09-15T22:38:25.005971+0000","last_clean":"2024-09-15T22:38:25.005971+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:38:25.005971+0000","last_undegraded":"2024-09-15T22:38:25.005971+0000","last_fullsized":"2024-09-15T22:38:25.005971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124202+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:38:25.124202+0000","last_peered":"2024-09-15T22:38:25.124202+0000","last_clean":"2024-09-15T22:38:25.124202+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:38:25.124202+0000","last_undegraded":"2024-09-15T22:38:25.124202+0000","last_fullsized":"2024-09-15T22:38:25.124202+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009565+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:38:25.009565+0000","last_peered":"2024-09-15T22:38:25.009565+0000","last_clean":"2024-09-15T22:38:25.009565+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:38:25.009565+0000","last_undegraded":"2024-09-15T22:38:25.009565+0000","last_fullsized":"2024-09-15T22:38:25.009565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241674+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:38:25.241674+0000","last_peered":"2024-09-15T22:38:25.241674+0000","last_clean":"2024-09-15T22:38:25.241674+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:38:25.241674+0000","last_undegraded":"2024-09-15T22:38:25.241674+0000","last_fullsized":"2024-09-15T22:38:25.241674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:29.542268+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:38:29.542268+0000","last_peered":"2024-09-15T22:38:29.542268+0000","last_clean":"2024-09-15T22:38:29.542268+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:38:29.542268+0000","last_undegraded":"2024-09-15T22:38:29.542268+0000","last_fullsized":"2024-09-15T22:38:29.542268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112093+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:38:25.112093+0000","last_peered":"2024-09-15T22:38:25.112093+0000","last_clean":"2024-09-15T22:38:25.112093+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:38:25.112093+0000","last_undegraded":"2024-09-15T22:38:25.112093+0000","last_fullsized":"2024-09-15T22:38:25.112093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241736+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:38:25.241736+0000","last_peered":"2024-09-15T22:38:25.241736+0000","last_clean":"2024-09-15T22:38:25.241736+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:38:25.241736+0000","last_undegraded":"2024-09-15T22:38:25.241736+0000","last_fullsized":"2024-09-15T22:38:25.241736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241163+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:38:25.241163+0000","last_peered":"2024-09-15T22:38:25.241163+0000","last_clean":"2024-09-15T22:38:25.241163+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:38:25.241163+0000","last_undegraded":"2024-09-15T22:38:25.241163+0000","last_fullsized":"2024-09-15T22:38:25.241163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241675+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:38:25.241675+0000","last_peered":"2024-09-15T22:38:25.241675+0000","last_clean":"2024-09-15T22:38:25.241675+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:38:25.241675+0000","last_undegraded":"2024-09-15T22:38:25.241675+0000","last_fullsized":"2024-09-15T22:38:25.241675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241517+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:38:25.241517+0000","last_peered":"2024-09-15T22:38:25.241517+0000","last_clean":"2024-09-15T22:38:25.241517+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:38:25.241517+0000","last_undegraded":"2024-09-15T22:38:25.241517+0000","last_fullsized":"2024-09-15T22:38:25.241517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249023+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:38:25.249023+0000","last_peered":"2024-09-15T22:38:25.249023+0000","last_clean":"2024-09-15T22:38:25.249023+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:38:25.249023+0000","last_undegraded":"2024-09-15T22:38:25.249023+0000","last_fullsized":"2024-09-15T22:38:25.249023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006558+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:38:25.006558+0000","last_peered":"2024-09-15T22:38:25.006558+0000","last_clean":"2024-09-15T22:38:25.006558+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:38:25.006558+0000","last_undegraded":"2024-09-15T22:38:25.006558+0000","last_fullsized":"2024-09-15T22:38:25.006558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242143+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:38:25.242143+0000","last_peered":"2024-09-15T22:38:25.242143+0000","last_clean":"2024-09-15T22:38:25.242143+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:38:25.242143+0000","last_undegraded":"2024-09-15T22:38:25.242143+0000","last_fullsized":"2024-09-15T22:38:25.242143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241544+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:38:25.241544+0000","last_peered":"2024-09-15T22:38:25.241544+0000","last_clean":"2024-09-15T22:38:25.241544+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:38:25.241544+0000","last_undegraded":"2024-09-15T22:38:25.241544+0000","last_fullsized":"2024-09-15T22:38:25.241544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115695+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:38:25.115695+0000","last_peered":"2024-09-15T22:38:25.115695+0000","last_clean":"2024-09-15T22:38:25.115695+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:38:25.115695+0000","last_undegraded":"2024-09-15T22:38:25.115695+0000","last_fullsized":"2024-09-15T22:38:25.115695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009267+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:38:25.009267+0000","last_peered":"2024-09-15T22:38:25.009267+0000","last_clean":"2024-09-15T22:38:25.009267+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:38:25.009267+0000","last_undegraded":"2024-09-15T22:38:25.009267+0000","last_fullsized":"2024-09-15T22:38:25.009267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112660+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:38:25.112660+0000","last_peered":"2024-09-15T22:38:25.112660+0000","last_clean":"2024-09-15T22:38:25.112660+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:38:25.112660+0000","last_undegraded":"2024-09-15T22:38:25.112660+0000","last_fullsized":"2024-09-15T22:38:25.112660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241756+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:38:25.241756+0000","last_peered":"2024-09-15T22:38:25.241756+0000","last_clean":"2024-09-15T22:38:25.241756+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:38:25.241756+0000","last_undegraded":"2024-09-15T22:38:25.241756+0000","last_fullsized":"2024-09-15T22:38:25.241756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006219+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:38:25.006219+0000","last_peered":"2024-09-15T22:38:25.006219+0000","last_clean":"2024-09-15T22:38:25.006219+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:38:25.006219+0000","last_undegraded":"2024-09-15T22:38:25.006219+0000","last_fullsized":"2024-09-15T22:38:25.006219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242055+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:38:25.242055+0000","last_peered":"2024-09-15T22:38:25.242055+0000","last_clean":"2024-09-15T22:38:25.242055+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:38:25.242055+0000","last_undegraded":"2024-09-15T22:38:25.242055+0000","last_fullsized":"2024-09-15T22:38:25.242055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241921+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:38:25.241921+0000","last_peered":"2024-09-15T22:38:25.241921+0000","last_clean":"2024-09-15T22:38:25.241921+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:38:25.241921+0000","last_undegraded":"2024-09-15T22:38:25.241921+0000","last_fullsized":"2024-09-15T22:38:25.241921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.109165+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:38:25.109165+0000","last_peered":"2024-09-15T22:38:25.109165+0000","last_clean":"2024-09-15T22:38:25.109165+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:38:25.109165+0000","last_undegraded":"2024-09-15T22:38:25.109165+0000","last_fullsized":"2024-09-15T22:38:25.109165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242550+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:38:25.242550+0000","last_peered":"2024-09-15T22:38:25.242550+0000","last_clean":"2024-09-15T22:38:25.242550+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:38:25.242550+0000","last_undegraded":"2024-09-15T22:38:25.242550+0000","last_fullsized":"2024-09-15T22:38:25.242550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241084+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:38:25.241084+0000","last_peered":"2024-09-15T22:38:25.241084+0000","last_clean":"2024-09-15T22:38:25.241084+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:38:25.241084+0000","last_undegraded":"2024-09-15T22:38:25.241084+0000","last_fullsized":"2024-09-15T22:38:25.241084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248330+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:38:25.248330+0000","last_peered":"2024-09-15T22:38:25.248330+0000","last_clean":"2024-09-15T22:38:25.248330+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:38:25.248330+0000","last_undegraded":"2024-09-15T22:38:25.248330+0000","last_fullsized":"2024-09-15T22:38:25.248330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248630+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:38:25.248630+0000","last_peered":"2024-09-15T22:38:25.248630+0000","last_clean":"2024-09-15T22:38:25.248630+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:38:25.248630+0000","last_undegraded":"2024-09-15T22:38:25.248630+0000","last_fullsized":"2024-09-15T22:38:25.248630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006155+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:38:25.006155+0000","last_peered":"2024-09-15T22:38:25.006155+0000","last_clean":"2024-09-15T22:38:25.006155+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:38:25.006155+0000","last_undegraded":"2024-09-15T22:38:25.006155+0000","last_fullsized":"2024-09-15T22:38:25.006155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241453+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:38:25.241453+0000","last_peered":"2024-09-15T22:38:25.241453+0000","last_clean":"2024-09-15T22:38:25.241453+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:38:25.241453+0000","last_undegraded":"2024-09-15T22:38:25.241453+0000","last_fullsized":"2024-09-15T22:38:25.241453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010699+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:38:25.010699+0000","last_peered":"2024-09-15T22:38:25.010699+0000","last_clean":"2024-09-15T22:38:25.010699+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:38:25.010699+0000","last_undegraded":"2024-09-15T22:38:25.010699+0000","last_fullsized":"2024-09-15T22:38:25.010699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124494+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:38:25.124494+0000","last_peered":"2024-09-15T22:38:25.124494+0000","last_clean":"2024-09-15T22:38:25.124494+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:38:25.124494+0000","last_undegraded":"2024-09-15T22:38:25.124494+0000","last_fullsized":"2024-09-15T22:38:25.124494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241964+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:38:25.241964+0000","last_peered":"2024-09-15T22:38:25.241964+0000","last_clean":"2024-09-15T22:38:25.241964+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:38:25.241964+0000","last_undegraded":"2024-09-15T22:38:25.241964+0000","last_fullsized":"2024-09-15T22:38:25.241964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112097+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:38:25.112097+0000","last_peered":"2024-09-15T22:38:25.112097+0000","last_clean":"2024-09-15T22:38:25.112097+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:38:25.112097+0000","last_undegraded":"2024-09-15T22:38:25.112097+0000","last_fullsized":"2024-09-15T22:38:25.112097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010114+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:38:25.010114+0000","last_peered":"2024-09-15T22:38:25.010114+0000","last_clean":"2024-09-15T22:38:25.010114+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:38:25.010114+0000","last_undegraded":"2024-09-15T22:38:25.010114+0000","last_fullsized":"2024-09-15T22:38:25.010114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124146+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:38:25.124146+0000","last_peered":"2024-09-15T22:38:25.124146+0000","last_clean":"2024-09-15T22:38:25.124146+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:38:25.124146+0000","last_undegraded":"2024-09-15T22:38:25.124146+0000","last_fullsized":"2024-09-15T22:38:25.124146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112085+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:38:25.112085+0000","last_peered":"2024-09-15T22:38:25.112085+0000","last_clean":"2024-09-15T22:38:25.112085+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:38:25.112085+0000","last_undegraded":"2024-09-15T22:38:25.112085+0000","last_fullsized":"2024-09-15T22:38:25.112085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108245+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:38:25.108245+0000","last_peered":"2024-09-15T22:38:25.108245+0000","last_clean":"2024-09-15T22:38:25.108245+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:38:25.108245+0000","last_undegraded":"2024-09-15T22:38:25.108245+0000","last_fullsized":"2024-09-15T22:38:25.108245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249437+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:38:25.249437+0000","last_peered":"2024-09-15T22:38:25.249437+0000","last_clean":"2024-09-15T22:38:25.249437+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:38:25.249437+0000","last_undegraded":"2024-09-15T22:38:25.249437+0000","last_fullsized":"2024-09-15T22:38:25.249437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249830+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:38:25.249830+0000","last_peered":"2024-09-15T22:38:25.249830+0000","last_clean":"2024-09-15T22:38:25.249830+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:38:25.249830+0000","last_undegraded":"2024-09-15T22:38:25.249830+0000","last_fullsized":"2024-09-15T22:38:25.249830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242437+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:38:25.242437+0000","last_peered":"2024-09-15T22:38:25.242437+0000","last_clean":"2024-09-15T22:38:25.242437+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:38:25.242437+0000","last_undegraded":"2024-09-15T22:38:25.242437+0000","last_fullsized":"2024-09-15T22:38:25.242437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599775,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":760,"num_read_kb":491,"num_write":298,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1799325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612649,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":519982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841582,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28852,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716300,"statfs":{"total":95995035648,"available":95965491200,"internally_reserved":0,"allocated":1871872,"data_stored":1309665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005108,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36768,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708384,"statfs":{"total":95995035648,"available":95957385216,"internally_reserved":0,"allocated":1454080,"data_stored":917530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1449984,"data_stored":913354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462974,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1650688,"data_stored":1117284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":515394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855432,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28776,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716376,"statfs":{"total":95995035648,"available":95965569024,"internally_reserved":0,"allocated":1794048,"data_stored":1243378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":521250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215187,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36884,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708268,"statfs":{"total":95995035648,"available":95957266432,"internally_reserved":0,"allocated":1576960,"data_stored":1040404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":586797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":585025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":516782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:38:36.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:38:36.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:36 smithi059 ceph-mon[34872]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-15T22:38:36.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:36 smithi130 ceph-mon[35742]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-15T22:38:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.458+0000 7f844787c640 1 -- 172.21.15.16:0/414740668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 msgr2=0x7f844010df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.458+0000 7f844787c640 1 --2- 172.21.15.16:0/414740668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f844010df30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f843002a410 comp rx=0 tx=0).stop 2024-09-15T22:38:37.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.458+0000 7f844787c640 1 -- 172.21.15.16:0/414740668 shutdown_connections 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 --2- 172.21.15.16:0/414740668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f844010e500 0x7f8440110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 --2- 172.21.15.16:0/414740668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f844010df30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 --2- 172.21.15.16:0/414740668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8440100b20 0x7f8440100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 -- 172.21.15.16:0/414740668 >> 172.21.15.16:0/414740668 conn(0x7f84400fc7d0 msgr2=0x7f84400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 -- 172.21.15.16:0/414740668 shutdown_connections 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 -- 172.21.15.16:0/414740668 wait complete. 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 Processor -- start 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 -- start start 2024-09-15T22:38:37.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8440100b20 0x7f84401a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f84401a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f844010e500 0x7f844019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84401142a0 con 0x7f8440101440 2024-09-15T22:38:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8440114120 con 0x7f844010e500 2024-09-15T22:38:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.459+0000 7f844787c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8440114420 con 0x7f8440100b20 2024-09-15T22:38:37.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f84401a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f84401a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54396/0 (socket says 172.21.15.16:54396) 2024-09-15T22:38:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 -- 172.21.15.16:0/2825912364 learned_addr learned my addr 172.21.15.16:0/2825912364 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8445618640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8440100b20 0x7f84401a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8445e19640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f844010e500 0x7f844019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 -- 172.21.15.16:0/2825912364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8440100b20 msgr2=0x7f84401a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8440100b20 0x7f84401a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 -- 172.21.15.16:0/2825912364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f844010e500 msgr2=0x7f844019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f844010e500 0x7f844019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 -- 172.21.15.16:0/2825912364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844019acf0 con 0x7f8440101440 2024-09-15T22:38:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f8444e17640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f84401a0940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f843002a930 tx=0x7f8430003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f84367fc640 1 -- 172.21.15.16:0/2825912364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843003f070 con 0x7f8440101440 2024-09-15T22:38:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844019af80 con 0x7f8440101440 2024-09-15T22:38:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f84367fc640 1 -- 172.21.15.16:0/2825912364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8430031c90 con 0x7f8440101440 2024-09-15T22:38:37.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f84367fc640 1 -- 172.21.15.16:0/2825912364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035030 con 0x7f8440101440 2024-09-15T22:38:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.460+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84401a72d0 con 0x7f8440101440 2024-09-15T22:38:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.462+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8440100f00 con 0x7f8440101440 2024-09-15T22:38:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.463+0000 7f84367fc640 1 -- 172.21.15.16:0/2825912364 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f8430042070 con 0x7f8440101440 2024-09-15T22:38:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.463+0000 7f84367fc640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f842c0a34c0 0x7f842c0a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.464+0000 7f84367fc640 1 -- 172.21.15.16:0/2825912364 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f84300f0c70 con 0x7f8440101440 2024-09-15T22:38:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.464+0000 7f8445618640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f842c0a34c0 0x7f842c0a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:37.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.464+0000 7f8445618640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f842c0a34c0 0x7f842c0a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f843c0058c0 tx=0x7f843c00a2d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:37.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.467+0000 7f84367fc640 1 -- 172.21.15.16:0/2825912364 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84300bcc40 con 0x7f8440101440 2024-09-15T22:38:37.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:37 smithi016 ceph-mon[31492]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:37.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.640+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8440062630 con 0x7f842c0a34c0 2024-09-15T22:38:37.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.651+0000 7f84367fc640 1 -- 172.21.15.16:0/2825912364 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339406 (secure 0 0 0) 0x7f8440062630 con 0x7f842c0a34c0 2024-09-15T22:38:37.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:37.658 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:38:37.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.655+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f842c0a34c0 msgr2=0x7f842c0a5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.655+0000 7f844787c640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f842c0a34c0 0x7f842c0a5980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f843c0058c0 tx=0x7f843c00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.655+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 msgr2=0x7f84401a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.655+0000 7f844787c640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f84401a0940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f843002a930 tx=0x7f8430003b90 comp rx=0 tx=0).stop 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 shutdown_connections 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f844010e500 0x7f844019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8440101440 0x7f84401a0940 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f842c0a34c0 0x7f842c0a5980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 --2- 172.21.15.16:0/2825912364 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8440100b20 0x7f84401a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 >> 172.21.15.16:0/2825912364 conn(0x7f84400fc7d0 msgr2=0x7f84400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 shutdown_connections 2024-09-15T22:38:37.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:37.656+0000 7f844787c640 1 -- 172.21.15.16:0/2825912364 wait complete. 2024-09-15T22:38:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:37 smithi059 ceph-mon[34872]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:37.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:37 smithi130 ceph-mon[35742]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:38.391 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-15T22:38:37.021761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387009,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1282,"num_read_kb":1068,"num_write":627,"num_write_kb":5298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":962,"ondisk_log_size":962,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":415976,"kb_used_data":16424,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124525848,"statfs":{"total":1151940427776,"available":1151514468352,"internally_reserved":0,"allocated":16818176,"data_stored":10376741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":473,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.994384"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108817+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:38:25.108817+0000","last_peered":"2024-09-15T22:38:25.108817+0000","last_clean":"2024-09-15T22:38:25.108817+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:38:25.108817+0000","last_undegraded":"2024-09-15T22:38:25.108817+0000","last_fullsized":"2024-09-15T22:38:25.108817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114924+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:38:25.114924+0000","last_peered":"2024-09-15T22:38:25.114924+0000","last_clean":"2024-09-15T22:38:25.114924+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:38:25.114924+0000","last_undegraded":"2024-09-15T22:38:25.114924+0000","last_fullsized":"2024-09-15T22:38:25.114924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241365+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:38:25.241365+0000","last_peered":"2024-09-15T22:38:25.241365+0000","last_clean":"2024-09-15T22:38:25.241365+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:38:25.241365+0000","last_undegraded":"2024-09-15T22:38:25.241365+0000","last_fullsized":"2024-09-15T22:38:25.241365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115300+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:38:25.115300+0000","last_peered":"2024-09-15T22:38:25.115300+0000","last_clean":"2024-09-15T22:38:25.115300+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:38:25.115300+0000","last_undegraded":"2024-09-15T22:38:25.115300+0000","last_fullsized":"2024-09-15T22:38:25.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250490+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:38:25.250490+0000","last_peered":"2024-09-15T22:38:25.250490+0000","last_clean":"2024-09-15T22:38:25.250490+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:38:25.250490+0000","last_undegraded":"2024-09-15T22:38:25.250490+0000","last_fullsized":"2024-09-15T22:38:25.250490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240846+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:38:25.240846+0000","last_peered":"2024-09-15T22:38:25.240846+0000","last_clean":"2024-09-15T22:38:25.240846+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:38:25.240846+0000","last_undegraded":"2024-09-15T22:38:25.240846+0000","last_fullsized":"2024-09-15T22:38:25.240846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112544+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:38:25.112544+0000","last_peered":"2024-09-15T22:38:25.112544+0000","last_clean":"2024-09-15T22:38:25.112544+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:38:25.112544+0000","last_undegraded":"2024-09-15T22:38:25.112544+0000","last_fullsized":"2024-09-15T22:38:25.112544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241710+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:38:25.241710+0000","last_peered":"2024-09-15T22:38:25.241710+0000","last_clean":"2024-09-15T22:38:25.241710+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:38:25.241710+0000","last_undegraded":"2024-09-15T22:38:25.241710+0000","last_fullsized":"2024-09-15T22:38:25.241710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241001+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:38:25.241001+0000","last_peered":"2024-09-15T22:38:25.241001+0000","last_clean":"2024-09-15T22:38:25.241001+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:38:25.241001+0000","last_undegraded":"2024-09-15T22:38:25.241001+0000","last_fullsized":"2024-09-15T22:38:25.241001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112377+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:38:25.112377+0000","last_peered":"2024-09-15T22:38:25.112377+0000","last_clean":"2024-09-15T22:38:25.112377+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:38:25.112377+0000","last_undegraded":"2024-09-15T22:38:25.112377+0000","last_fullsized":"2024-09-15T22:38:25.112377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009573+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:38:25.009573+0000","last_peered":"2024-09-15T22:38:25.009573+0000","last_clean":"2024-09-15T22:38:25.009573+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:38:25.009573+0000","last_undegraded":"2024-09-15T22:38:25.009573+0000","last_fullsized":"2024-09-15T22:38:25.009573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240840+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:38:25.240840+0000","last_peered":"2024-09-15T22:38:25.240840+0000","last_clean":"2024-09-15T22:38:25.240840+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:38:25.240840+0000","last_undegraded":"2024-09-15T22:38:25.240840+0000","last_fullsized":"2024-09-15T22:38:25.240840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010739+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:38:25.010739+0000","last_peered":"2024-09-15T22:38:25.010739+0000","last_clean":"2024-09-15T22:38:25.010739+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:38:25.010739+0000","last_undegraded":"2024-09-15T22:38:25.010739+0000","last_fullsized":"2024-09-15T22:38:25.010739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241387+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:38:25.241387+0000","last_peered":"2024-09-15T22:38:25.241387+0000","last_clean":"2024-09-15T22:38:25.241387+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:38:25.241387+0000","last_undegraded":"2024-09-15T22:38:25.241387+0000","last_fullsized":"2024-09-15T22:38:25.241387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241362+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:38:25.241362+0000","last_peered":"2024-09-15T22:38:25.241362+0000","last_clean":"2024-09-15T22:38:25.241362+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:38:25.241362+0000","last_undegraded":"2024-09-15T22:38:25.241362+0000","last_fullsized":"2024-09-15T22:38:25.241362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113745+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:38:25.113745+0000","last_peered":"2024-09-15T22:38:25.113745+0000","last_clean":"2024-09-15T22:38:25.113745+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:38:25.113745+0000","last_undegraded":"2024-09-15T22:38:25.113745+0000","last_fullsized":"2024-09-15T22:38:25.113745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241575+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:38:25.241575+0000","last_peered":"2024-09-15T22:38:25.241575+0000","last_clean":"2024-09-15T22:38:25.241575+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:38:25.241575+0000","last_undegraded":"2024-09-15T22:38:25.241575+0000","last_fullsized":"2024-09-15T22:38:25.241575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241265+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:38:25.241265+0000","last_peered":"2024-09-15T22:38:25.241265+0000","last_clean":"2024-09-15T22:38:25.241265+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:38:25.241265+0000","last_undegraded":"2024-09-15T22:38:25.241265+0000","last_fullsized":"2024-09-15T22:38:25.241265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241234+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:38:25.241234+0000","last_peered":"2024-09-15T22:38:25.241234+0000","last_clean":"2024-09-15T22:38:25.241234+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:38:25.241234+0000","last_undegraded":"2024-09-15T22:38:25.241234+0000","last_fullsized":"2024-09-15T22:38:25.241234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114246+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:38:25.114246+0000","last_peered":"2024-09-15T22:38:25.114246+0000","last_clean":"2024-09-15T22:38:25.114246+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:38:25.114246+0000","last_undegraded":"2024-09-15T22:38:25.114246+0000","last_fullsized":"2024-09-15T22:38:25.114246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108657+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:38:25.108657+0000","last_peered":"2024-09-15T22:38:25.108657+0000","last_clean":"2024-09-15T22:38:25.108657+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:38:25.108657+0000","last_undegraded":"2024-09-15T22:38:25.108657+0000","last_fullsized":"2024-09-15T22:38:25.108657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112056+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:38:25.112056+0000","last_peered":"2024-09-15T22:38:25.112056+0000","last_clean":"2024-09-15T22:38:25.112056+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:38:25.112056+0000","last_undegraded":"2024-09-15T22:38:25.112056+0000","last_fullsized":"2024-09-15T22:38:25.112056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241262+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:38:25.241262+0000","last_peered":"2024-09-15T22:38:25.241262+0000","last_clean":"2024-09-15T22:38:25.241262+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:38:25.241262+0000","last_undegraded":"2024-09-15T22:38:25.241262+0000","last_fullsized":"2024-09-15T22:38:25.241262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241293+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:38:25.241293+0000","last_peered":"2024-09-15T22:38:25.241293+0000","last_clean":"2024-09-15T22:38:25.241293+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:38:25.241293+0000","last_undegraded":"2024-09-15T22:38:25.241293+0000","last_fullsized":"2024-09-15T22:38:25.241293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249966+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:38:25.249966+0000","last_peered":"2024-09-15T22:38:25.249966+0000","last_clean":"2024-09-15T22:38:25.249966+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:38:25.249966+0000","last_undegraded":"2024-09-15T22:38:25.249966+0000","last_fullsized":"2024-09-15T22:38:25.249966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241375+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:38:25.241375+0000","last_peered":"2024-09-15T22:38:25.241375+0000","last_clean":"2024-09-15T22:38:25.241375+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:38:25.241375+0000","last_undegraded":"2024-09-15T22:38:25.241375+0000","last_fullsized":"2024-09-15T22:38:25.241375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241418+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:38:25.241418+0000","last_peered":"2024-09-15T22:38:25.241418+0000","last_clean":"2024-09-15T22:38:25.241418+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:38:25.241418+0000","last_undegraded":"2024-09-15T22:38:25.241418+0000","last_fullsized":"2024-09-15T22:38:25.241418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115826+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:38:25.115826+0000","last_peered":"2024-09-15T22:38:25.115826+0000","last_clean":"2024-09-15T22:38:25.115826+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:38:25.115826+0000","last_undegraded":"2024-09-15T22:38:25.115826+0000","last_fullsized":"2024-09-15T22:38:25.115826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'101","reported_seq":234,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.578517+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:38:34.578517+0000","last_peered":"2024-09-15T22:38:34.578517+0000","last_clean":"2024-09-15T22:38:34.578517+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:38:34.578517+0000","last_undegraded":"2024-09-15T22:38:34.578517+0000","last_fullsized":"2024-09-15T22:38:34.578517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.111983+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:38:25.111983+0000","last_peered":"2024-09-15T22:38:25.111983+0000","last_clean":"2024-09-15T22:38:25.111983+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:38:25.111983+0000","last_undegraded":"2024-09-15T22:38:25.111983+0000","last_fullsized":"2024-09-15T22:38:25.111983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006514+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:38:25.006514+0000","last_peered":"2024-09-15T22:38:25.006514+0000","last_clean":"2024-09-15T22:38:25.006514+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:38:25.006514+0000","last_undegraded":"2024-09-15T22:38:25.006514+0000","last_fullsized":"2024-09-15T22:38:25.006514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112492+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:38:25.112492+0000","last_peered":"2024-09-15T22:38:25.112492+0000","last_clean":"2024-09-15T22:38:25.112492+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:38:25.112492+0000","last_undegraded":"2024-09-15T22:38:25.112492+0000","last_fullsized":"2024-09-15T22:38:25.112492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241541+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:38:25.241541+0000","last_peered":"2024-09-15T22:38:25.241541+0000","last_clean":"2024-09-15T22:38:25.241541+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:38:25.241541+0000","last_undegraded":"2024-09-15T22:38:25.241541+0000","last_fullsized":"2024-09-15T22:38:25.241541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124170+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:38:25.124170+0000","last_peered":"2024-09-15T22:38:25.124170+0000","last_clean":"2024-09-15T22:38:25.124170+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:38:25.124170+0000","last_undegraded":"2024-09-15T22:38:25.124170+0000","last_fullsized":"2024-09-15T22:38:25.124170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.110438+0000","last_change":"2024-09-15T22:38:14.242090+0000","last_active":"2024-09-15T22:38:25.110438+0000","last_peered":"2024-09-15T22:38:25.110438+0000","last_clean":"2024-09-15T22:38:25.110438+0000","last_became_active":"2024-09-15T22:38:14.241634+0000","last_became_peered":"2024-09-15T22:38:14.241634+0000","last_unstale":"2024-09-15T22:38:25.110438+0000","last_undegraded":"2024-09-15T22:38:25.110438+0000","last_fullsized":"2024-09-15T22:38:25.110438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:13.485506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112062+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:38:25.112062+0000","last_peered":"2024-09-15T22:38:25.112062+0000","last_clean":"2024-09-15T22:38:25.112062+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:38:25.112062+0000","last_undegraded":"2024-09-15T22:38:25.112062+0000","last_fullsized":"2024-09-15T22:38:25.112062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124297+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:38:25.124297+0000","last_peered":"2024-09-15T22:38:25.124297+0000","last_clean":"2024-09-15T22:38:25.124297+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:38:25.124297+0000","last_undegraded":"2024-09-15T22:38:25.124297+0000","last_fullsized":"2024-09-15T22:38:25.124297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010822+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:38:25.010822+0000","last_peered":"2024-09-15T22:38:25.010822+0000","last_clean":"2024-09-15T22:38:25.010822+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:38:25.010822+0000","last_undegraded":"2024-09-15T22:38:25.010822+0000","last_fullsized":"2024-09-15T22:38:25.010822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006453+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:38:25.006453+0000","last_peered":"2024-09-15T22:38:25.006453+0000","last_clean":"2024-09-15T22:38:25.006453+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:38:25.006453+0000","last_undegraded":"2024-09-15T22:38:25.006453+0000","last_fullsized":"2024-09-15T22:38:25.006453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009249+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:38:25.009249+0000","last_peered":"2024-09-15T22:38:25.009249+0000","last_clean":"2024-09-15T22:38:25.009249+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:38:25.009249+0000","last_undegraded":"2024-09-15T22:38:25.009249+0000","last_fullsized":"2024-09-15T22:38:25.009249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.549838+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:38:34.549838+0000","last_peered":"2024-09-15T22:38:34.549838+0000","last_clean":"2024-09-15T22:38:34.549838+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:38:34.549838+0000","last_undegraded":"2024-09-15T22:38:34.549838+0000","last_fullsized":"2024-09-15T22:38:34.549838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006955+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:38:25.006955+0000","last_peered":"2024-09-15T22:38:25.006955+0000","last_clean":"2024-09-15T22:38:25.006955+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:38:25.006955+0000","last_undegraded":"2024-09-15T22:38:25.006955+0000","last_fullsized":"2024-09-15T22:38:25.006955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005813+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:38:25.005813+0000","last_peered":"2024-09-15T22:38:25.005813+0000","last_clean":"2024-09-15T22:38:25.005813+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:38:25.005813+0000","last_undegraded":"2024-09-15T22:38:25.005813+0000","last_fullsized":"2024-09-15T22:38:25.005813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010164+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:38:25.010164+0000","last_peered":"2024-09-15T22:38:25.010164+0000","last_clean":"2024-09-15T22:38:25.010164+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:38:25.010164+0000","last_undegraded":"2024-09-15T22:38:25.010164+0000","last_fullsized":"2024-09-15T22:38:25.010164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009598+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:38:25.009598+0000","last_peered":"2024-09-15T22:38:25.009598+0000","last_clean":"2024-09-15T22:38:25.009598+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:38:25.009598+0000","last_undegraded":"2024-09-15T22:38:25.009598+0000","last_fullsized":"2024-09-15T22:38:25.009598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113256+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:38:25.113256+0000","last_peered":"2024-09-15T22:38:25.113256+0000","last_clean":"2024-09-15T22:38:25.113256+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:38:25.113256+0000","last_undegraded":"2024-09-15T22:38:25.113256+0000","last_fullsized":"2024-09-15T22:38:25.113256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241238+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:38:25.241238+0000","last_peered":"2024-09-15T22:38:25.241238+0000","last_clean":"2024-09-15T22:38:25.241238+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:38:25.241238+0000","last_undegraded":"2024-09-15T22:38:25.241238+0000","last_fullsized":"2024-09-15T22:38:25.241238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":404,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:27.116484+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:38:27.116484+0000","last_peered":"2024-09-15T22:38:27.116484+0000","last_clean":"2024-09-15T22:38:27.116484+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:38:27.116484+0000","last_undegraded":"2024-09-15T22:38:27.116484+0000","last_fullsized":"2024-09-15T22:38:27.116484+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010316+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:38:25.010316+0000","last_peered":"2024-09-15T22:38:25.010316+0000","last_clean":"2024-09-15T22:38:25.010316+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:38:25.010316+0000","last_undegraded":"2024-09-15T22:38:25.010316+0000","last_fullsized":"2024-09-15T22:38:25.010316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010530+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:38:25.010530+0000","last_peered":"2024-09-15T22:38:25.010530+0000","last_clean":"2024-09-15T22:38:25.010530+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:38:25.010530+0000","last_undegraded":"2024-09-15T22:38:25.010530+0000","last_fullsized":"2024-09-15T22:38:25.010530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241899+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:38:25.241899+0000","last_peered":"2024-09-15T22:38:25.241899+0000","last_clean":"2024-09-15T22:38:25.241899+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:38:25.241899+0000","last_undegraded":"2024-09-15T22:38:25.241899+0000","last_fullsized":"2024-09-15T22:38:25.241899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241629+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:38:25.241629+0000","last_peered":"2024-09-15T22:38:25.241629+0000","last_clean":"2024-09-15T22:38:25.241629+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:38:25.241629+0000","last_undegraded":"2024-09-15T22:38:25.241629+0000","last_fullsized":"2024-09-15T22:38:25.241629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010356+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:38:25.010356+0000","last_peered":"2024-09-15T22:38:25.010356+0000","last_clean":"2024-09-15T22:38:25.010356+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:38:25.010356+0000","last_undegraded":"2024-09-15T22:38:25.010356+0000","last_fullsized":"2024-09-15T22:38:25.010356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'65","reported_seq":137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.186113+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:38:34.186113+0000","last_peered":"2024-09-15T22:38:34.186113+0000","last_clean":"2024-09-15T22:38:34.186113+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:38:34.186113+0000","last_undegraded":"2024-09-15T22:38:34.186113+0000","last_fullsized":"2024-09-15T22:38:34.186113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248973+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:38:25.248973+0000","last_peered":"2024-09-15T22:38:25.248973+0000","last_clean":"2024-09-15T22:38:25.248973+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:38:25.248973+0000","last_undegraded":"2024-09-15T22:38:25.248973+0000","last_fullsized":"2024-09-15T22:38:25.248973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010556+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:38:25.010556+0000","last_peered":"2024-09-15T22:38:25.010556+0000","last_clean":"2024-09-15T22:38:25.010556+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:38:25.010556+0000","last_undegraded":"2024-09-15T22:38:25.010556+0000","last_fullsized":"2024-09-15T22:38:25.010556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006902+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:38:25.006902+0000","last_peered":"2024-09-15T22:38:25.006902+0000","last_clean":"2024-09-15T22:38:25.006902+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:38:25.006902+0000","last_undegraded":"2024-09-15T22:38:25.006902+0000","last_fullsized":"2024-09-15T22:38:25.006902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009099+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:38:25.009099+0000","last_peered":"2024-09-15T22:38:25.009099+0000","last_clean":"2024-09-15T22:38:25.009099+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:38:25.009099+0000","last_undegraded":"2024-09-15T22:38:25.009099+0000","last_fullsized":"2024-09-15T22:38:25.009099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242337+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:38:25.242337+0000","last_peered":"2024-09-15T22:38:25.242337+0000","last_clean":"2024-09-15T22:38:25.242337+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:38:25.242337+0000","last_undegraded":"2024-09-15T22:38:25.242337+0000","last_fullsized":"2024-09-15T22:38:25.242337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.395057+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:38:25.395057+0000","last_peered":"2024-09-15T22:38:25.395057+0000","last_clean":"2024-09-15T22:38:25.395057+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:38:25.395057+0000","last_undegraded":"2024-09-15T22:38:25.395057+0000","last_fullsized":"2024-09-15T22:38:25.395057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009234+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:38:25.009234+0000","last_peered":"2024-09-15T22:38:25.009234+0000","last_clean":"2024-09-15T22:38:25.009234+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:38:25.009234+0000","last_undegraded":"2024-09-15T22:38:25.009234+0000","last_fullsized":"2024-09-15T22:38:25.009234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009429+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:38:25.009429+0000","last_peered":"2024-09-15T22:38:25.009429+0000","last_clean":"2024-09-15T22:38:25.009429+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:38:25.009429+0000","last_undegraded":"2024-09-15T22:38:25.009429+0000","last_fullsized":"2024-09-15T22:38:25.009429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112094+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:38:25.112094+0000","last_peered":"2024-09-15T22:38:25.112094+0000","last_clean":"2024-09-15T22:38:25.112094+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:38:25.112094+0000","last_undegraded":"2024-09-15T22:38:25.112094+0000","last_fullsized":"2024-09-15T22:38:25.112094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006770+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:38:25.006770+0000","last_peered":"2024-09-15T22:38:25.006770+0000","last_clean":"2024-09-15T22:38:25.006770+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:38:25.006770+0000","last_undegraded":"2024-09-15T22:38:25.006770+0000","last_fullsized":"2024-09-15T22:38:25.006770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010143+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:38:25.010143+0000","last_peered":"2024-09-15T22:38:25.010143+0000","last_clean":"2024-09-15T22:38:25.010143+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:38:25.010143+0000","last_undegraded":"2024-09-15T22:38:25.010143+0000","last_fullsized":"2024-09-15T22:38:25.010143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.550674+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:38:34.550674+0000","last_peered":"2024-09-15T22:38:34.550674+0000","last_clean":"2024-09-15T22:38:34.550674+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:38:34.550674+0000","last_undegraded":"2024-09-15T22:38:34.550674+0000","last_fullsized":"2024-09-15T22:38:34.550674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108636+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:38:25.108636+0000","last_peered":"2024-09-15T22:38:25.108636+0000","last_clean":"2024-09-15T22:38:25.108636+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:38:25.108636+0000","last_undegraded":"2024-09-15T22:38:25.108636+0000","last_fullsized":"2024-09-15T22:38:25.108636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010315+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:38:25.010315+0000","last_peered":"2024-09-15T22:38:25.010315+0000","last_clean":"2024-09-15T22:38:25.010315+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:38:25.010315+0000","last_undegraded":"2024-09-15T22:38:25.010315+0000","last_fullsized":"2024-09-15T22:38:25.010315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009653+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:38:25.009653+0000","last_peered":"2024-09-15T22:38:25.009653+0000","last_clean":"2024-09-15T22:38:25.009653+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:38:25.009653+0000","last_undegraded":"2024-09-15T22:38:25.009653+0000","last_fullsized":"2024-09-15T22:38:25.009653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241984+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:38:25.241984+0000","last_peered":"2024-09-15T22:38:25.241984+0000","last_clean":"2024-09-15T22:38:25.241984+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:38:25.241984+0000","last_undegraded":"2024-09-15T22:38:25.241984+0000","last_fullsized":"2024-09-15T22:38:25.241984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006052+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:38:25.006052+0000","last_peered":"2024-09-15T22:38:25.006052+0000","last_clean":"2024-09-15T22:38:25.006052+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:38:25.006052+0000","last_undegraded":"2024-09-15T22:38:25.006052+0000","last_fullsized":"2024-09-15T22:38:25.006052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241792+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:38:25.241792+0000","last_peered":"2024-09-15T22:38:25.241792+0000","last_clean":"2024-09-15T22:38:25.241792+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:38:25.241792+0000","last_undegraded":"2024-09-15T22:38:25.241792+0000","last_fullsized":"2024-09-15T22:38:25.241792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009484+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:38:25.009484+0000","last_peered":"2024-09-15T22:38:25.009484+0000","last_clean":"2024-09-15T22:38:25.009484+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:38:25.009484+0000","last_undegraded":"2024-09-15T22:38:25.009484+0000","last_fullsized":"2024-09-15T22:38:25.009484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112424+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:38:25.112424+0000","last_peered":"2024-09-15T22:38:25.112424+0000","last_clean":"2024-09-15T22:38:25.112424+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:38:25.112424+0000","last_undegraded":"2024-09-15T22:38:25.112424+0000","last_fullsized":"2024-09-15T22:38:25.112424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":316,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:35.549922+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:38:35.549922+0000","last_peered":"2024-09-15T22:38:35.549922+0000","last_clean":"2024-09-15T22:38:35.549922+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:38:35.549922+0000","last_undegraded":"2024-09-15T22:38:35.549922+0000","last_fullsized":"2024-09-15T22:38:35.549922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69721,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":186,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107742+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:38:25.107742+0000","last_peered":"2024-09-15T22:38:25.107742+0000","last_clean":"2024-09-15T22:38:25.107742+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:38:25.107742+0000","last_undegraded":"2024-09-15T22:38:25.107742+0000","last_fullsized":"2024-09-15T22:38:25.107742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241767+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:38:25.241767+0000","last_peered":"2024-09-15T22:38:25.241767+0000","last_clean":"2024-09-15T22:38:25.241767+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:38:25.241767+0000","last_undegraded":"2024-09-15T22:38:25.241767+0000","last_fullsized":"2024-09-15T22:38:25.241767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009858+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:38:25.009858+0000","last_peered":"2024-09-15T22:38:25.009858+0000","last_clean":"2024-09-15T22:38:25.009858+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:38:25.009858+0000","last_undegraded":"2024-09-15T22:38:25.009858+0000","last_fullsized":"2024-09-15T22:38:25.009858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241176+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:38:25.241176+0000","last_peered":"2024-09-15T22:38:25.241176+0000","last_clean":"2024-09-15T22:38:25.241176+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:38:25.241176+0000","last_undegraded":"2024-09-15T22:38:25.241176+0000","last_fullsized":"2024-09-15T22:38:25.241176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250669+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:38:25.250669+0000","last_peered":"2024-09-15T22:38:25.250669+0000","last_clean":"2024-09-15T22:38:25.250669+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:38:25.250669+0000","last_undegraded":"2024-09-15T22:38:25.250669+0000","last_fullsized":"2024-09-15T22:38:25.250669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010725+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:38:25.010725+0000","last_peered":"2024-09-15T22:38:25.010725+0000","last_clean":"2024-09-15T22:38:25.010725+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:38:25.010725+0000","last_undegraded":"2024-09-15T22:38:25.010725+0000","last_fullsized":"2024-09-15T22:38:25.010725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107560+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:38:25.107560+0000","last_peered":"2024-09-15T22:38:25.107560+0000","last_clean":"2024-09-15T22:38:25.107560+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:38:25.107560+0000","last_undegraded":"2024-09-15T22:38:25.107560+0000","last_fullsized":"2024-09-15T22:38:25.107560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242031+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:38:25.242031+0000","last_peered":"2024-09-15T22:38:25.242031+0000","last_clean":"2024-09-15T22:38:25.242031+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:38:25.242031+0000","last_undegraded":"2024-09-15T22:38:25.242031+0000","last_fullsized":"2024-09-15T22:38:25.242031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.548331+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:38:34.548331+0000","last_peered":"2024-09-15T22:38:34.548331+0000","last_clean":"2024-09-15T22:38:34.548331+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:38:34.548331+0000","last_undegraded":"2024-09-15T22:38:34.548331+0000","last_fullsized":"2024-09-15T22:38:34.548331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112480+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:38:25.112480+0000","last_peered":"2024-09-15T22:38:25.112480+0000","last_clean":"2024-09-15T22:38:25.112480+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:38:25.112480+0000","last_undegraded":"2024-09-15T22:38:25.112480+0000","last_fullsized":"2024-09-15T22:38:25.112480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249732+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:38:25.249732+0000","last_peered":"2024-09-15T22:38:25.249732+0000","last_clean":"2024-09-15T22:38:25.249732+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:38:25.249732+0000","last_undegraded":"2024-09-15T22:38:25.249732+0000","last_fullsized":"2024-09-15T22:38:25.249732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107954+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:38:25.107954+0000","last_peered":"2024-09-15T22:38:25.107954+0000","last_clean":"2024-09-15T22:38:25.107954+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:38:25.107954+0000","last_undegraded":"2024-09-15T22:38:25.107954+0000","last_fullsized":"2024-09-15T22:38:25.107954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009527+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:38:25.009527+0000","last_peered":"2024-09-15T22:38:25.009527+0000","last_clean":"2024-09-15T22:38:25.009527+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:38:25.009527+0000","last_undegraded":"2024-09-15T22:38:25.009527+0000","last_fullsized":"2024-09-15T22:38:25.009527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242387+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:38:25.242387+0000","last_peered":"2024-09-15T22:38:25.242387+0000","last_clean":"2024-09-15T22:38:25.242387+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:38:25.242387+0000","last_undegraded":"2024-09-15T22:38:25.242387+0000","last_fullsized":"2024-09-15T22:38:25.242387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112180+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:38:25.112180+0000","last_peered":"2024-09-15T22:38:25.112180+0000","last_clean":"2024-09-15T22:38:25.112180+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:38:25.112180+0000","last_undegraded":"2024-09-15T22:38:25.112180+0000","last_fullsized":"2024-09-15T22:38:25.112180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006027+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:38:25.006027+0000","last_peered":"2024-09-15T22:38:25.006027+0000","last_clean":"2024-09-15T22:38:25.006027+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:38:25.006027+0000","last_undegraded":"2024-09-15T22:38:25.006027+0000","last_fullsized":"2024-09-15T22:38:25.006027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124268+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:38:25.124268+0000","last_peered":"2024-09-15T22:38:25.124268+0000","last_clean":"2024-09-15T22:38:25.124268+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:38:25.124268+0000","last_undegraded":"2024-09-15T22:38:25.124268+0000","last_fullsized":"2024-09-15T22:38:25.124268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009565+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:38:25.009565+0000","last_peered":"2024-09-15T22:38:25.009565+0000","last_clean":"2024-09-15T22:38:25.009565+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:38:25.009565+0000","last_undegraded":"2024-09-15T22:38:25.009565+0000","last_fullsized":"2024-09-15T22:38:25.009565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241674+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:38:25.241674+0000","last_peered":"2024-09-15T22:38:25.241674+0000","last_clean":"2024-09-15T22:38:25.241674+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:38:25.241674+0000","last_undegraded":"2024-09-15T22:38:25.241674+0000","last_fullsized":"2024-09-15T22:38:25.241674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.549003+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:38:34.549003+0000","last_peered":"2024-09-15T22:38:34.549003+0000","last_clean":"2024-09-15T22:38:34.549003+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:38:34.549003+0000","last_undegraded":"2024-09-15T22:38:34.549003+0000","last_fullsized":"2024-09-15T22:38:34.549003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112093+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:38:25.112093+0000","last_peered":"2024-09-15T22:38:25.112093+0000","last_clean":"2024-09-15T22:38:25.112093+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:38:25.112093+0000","last_undegraded":"2024-09-15T22:38:25.112093+0000","last_fullsized":"2024-09-15T22:38:25.112093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241736+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:38:25.241736+0000","last_peered":"2024-09-15T22:38:25.241736+0000","last_clean":"2024-09-15T22:38:25.241736+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:38:25.241736+0000","last_undegraded":"2024-09-15T22:38:25.241736+0000","last_fullsized":"2024-09-15T22:38:25.241736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241163+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:38:25.241163+0000","last_peered":"2024-09-15T22:38:25.241163+0000","last_clean":"2024-09-15T22:38:25.241163+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:38:25.241163+0000","last_undegraded":"2024-09-15T22:38:25.241163+0000","last_fullsized":"2024-09-15T22:38:25.241163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241675+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:38:25.241675+0000","last_peered":"2024-09-15T22:38:25.241675+0000","last_clean":"2024-09-15T22:38:25.241675+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:38:25.241675+0000","last_undegraded":"2024-09-15T22:38:25.241675+0000","last_fullsized":"2024-09-15T22:38:25.241675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241517+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:38:25.241517+0000","last_peered":"2024-09-15T22:38:25.241517+0000","last_clean":"2024-09-15T22:38:25.241517+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:38:25.241517+0000","last_undegraded":"2024-09-15T22:38:25.241517+0000","last_fullsized":"2024-09-15T22:38:25.241517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249023+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:38:25.249023+0000","last_peered":"2024-09-15T22:38:25.249023+0000","last_clean":"2024-09-15T22:38:25.249023+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:38:25.249023+0000","last_undegraded":"2024-09-15T22:38:25.249023+0000","last_fullsized":"2024-09-15T22:38:25.249023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006558+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:38:25.006558+0000","last_peered":"2024-09-15T22:38:25.006558+0000","last_clean":"2024-09-15T22:38:25.006558+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:38:25.006558+0000","last_undegraded":"2024-09-15T22:38:25.006558+0000","last_fullsized":"2024-09-15T22:38:25.006558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242143+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:38:25.242143+0000","last_peered":"2024-09-15T22:38:25.242143+0000","last_clean":"2024-09-15T22:38:25.242143+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:38:25.242143+0000","last_undegraded":"2024-09-15T22:38:25.242143+0000","last_fullsized":"2024-09-15T22:38:25.242143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241544+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:38:25.241544+0000","last_peered":"2024-09-15T22:38:25.241544+0000","last_clean":"2024-09-15T22:38:25.241544+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:38:25.241544+0000","last_undegraded":"2024-09-15T22:38:25.241544+0000","last_fullsized":"2024-09-15T22:38:25.241544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115695+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:38:25.115695+0000","last_peered":"2024-09-15T22:38:25.115695+0000","last_clean":"2024-09-15T22:38:25.115695+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:38:25.115695+0000","last_undegraded":"2024-09-15T22:38:25.115695+0000","last_fullsized":"2024-09-15T22:38:25.115695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009267+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:38:25.009267+0000","last_peered":"2024-09-15T22:38:25.009267+0000","last_clean":"2024-09-15T22:38:25.009267+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:38:25.009267+0000","last_undegraded":"2024-09-15T22:38:25.009267+0000","last_fullsized":"2024-09-15T22:38:25.009267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112660+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:38:25.112660+0000","last_peered":"2024-09-15T22:38:25.112660+0000","last_clean":"2024-09-15T22:38:25.112660+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:38:25.112660+0000","last_undegraded":"2024-09-15T22:38:25.112660+0000","last_fullsized":"2024-09-15T22:38:25.112660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241756+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:38:25.241756+0000","last_peered":"2024-09-15T22:38:25.241756+0000","last_clean":"2024-09-15T22:38:25.241756+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:38:25.241756+0000","last_undegraded":"2024-09-15T22:38:25.241756+0000","last_fullsized":"2024-09-15T22:38:25.241756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006219+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:38:25.006219+0000","last_peered":"2024-09-15T22:38:25.006219+0000","last_clean":"2024-09-15T22:38:25.006219+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:38:25.006219+0000","last_undegraded":"2024-09-15T22:38:25.006219+0000","last_fullsized":"2024-09-15T22:38:25.006219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242055+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:38:25.242055+0000","last_peered":"2024-09-15T22:38:25.242055+0000","last_clean":"2024-09-15T22:38:25.242055+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:38:25.242055+0000","last_undegraded":"2024-09-15T22:38:25.242055+0000","last_fullsized":"2024-09-15T22:38:25.242055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241921+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:38:25.241921+0000","last_peered":"2024-09-15T22:38:25.241921+0000","last_clean":"2024-09-15T22:38:25.241921+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:38:25.241921+0000","last_undegraded":"2024-09-15T22:38:25.241921+0000","last_fullsized":"2024-09-15T22:38:25.241921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.109165+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:38:25.109165+0000","last_peered":"2024-09-15T22:38:25.109165+0000","last_clean":"2024-09-15T22:38:25.109165+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:38:25.109165+0000","last_undegraded":"2024-09-15T22:38:25.109165+0000","last_fullsized":"2024-09-15T22:38:25.109165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242550+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:38:25.242550+0000","last_peered":"2024-09-15T22:38:25.242550+0000","last_clean":"2024-09-15T22:38:25.242550+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:38:25.242550+0000","last_undegraded":"2024-09-15T22:38:25.242550+0000","last_fullsized":"2024-09-15T22:38:25.242550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241084+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:38:25.241084+0000","last_peered":"2024-09-15T22:38:25.241084+0000","last_clean":"2024-09-15T22:38:25.241084+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:38:25.241084+0000","last_undegraded":"2024-09-15T22:38:25.241084+0000","last_fullsized":"2024-09-15T22:38:25.241084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248330+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:38:25.248330+0000","last_peered":"2024-09-15T22:38:25.248330+0000","last_clean":"2024-09-15T22:38:25.248330+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:38:25.248330+0000","last_undegraded":"2024-09-15T22:38:25.248330+0000","last_fullsized":"2024-09-15T22:38:25.248330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248630+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:38:25.248630+0000","last_peered":"2024-09-15T22:38:25.248630+0000","last_clean":"2024-09-15T22:38:25.248630+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:38:25.248630+0000","last_undegraded":"2024-09-15T22:38:25.248630+0000","last_fullsized":"2024-09-15T22:38:25.248630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006222+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:38:25.006222+0000","last_peered":"2024-09-15T22:38:25.006222+0000","last_clean":"2024-09-15T22:38:25.006222+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:38:25.006222+0000","last_undegraded":"2024-09-15T22:38:25.006222+0000","last_fullsized":"2024-09-15T22:38:25.006222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241501+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:38:25.241501+0000","last_peered":"2024-09-15T22:38:25.241501+0000","last_clean":"2024-09-15T22:38:25.241501+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:38:25.241501+0000","last_undegraded":"2024-09-15T22:38:25.241501+0000","last_fullsized":"2024-09-15T22:38:25.241501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010748+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:38:25.010748+0000","last_peered":"2024-09-15T22:38:25.010748+0000","last_clean":"2024-09-15T22:38:25.010748+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:38:25.010748+0000","last_undegraded":"2024-09-15T22:38:25.010748+0000","last_fullsized":"2024-09-15T22:38:25.010748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124521+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:38:25.124521+0000","last_peered":"2024-09-15T22:38:25.124521+0000","last_clean":"2024-09-15T22:38:25.124521+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:38:25.124521+0000","last_undegraded":"2024-09-15T22:38:25.124521+0000","last_fullsized":"2024-09-15T22:38:25.124521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242023+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:38:25.242023+0000","last_peered":"2024-09-15T22:38:25.242023+0000","last_clean":"2024-09-15T22:38:25.242023+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:38:25.242023+0000","last_undegraded":"2024-09-15T22:38:25.242023+0000","last_fullsized":"2024-09-15T22:38:25.242023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112144+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:38:25.112144+0000","last_peered":"2024-09-15T22:38:25.112144+0000","last_clean":"2024-09-15T22:38:25.112144+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:38:25.112144+0000","last_undegraded":"2024-09-15T22:38:25.112144+0000","last_fullsized":"2024-09-15T22:38:25.112144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010114+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:38:25.010114+0000","last_peered":"2024-09-15T22:38:25.010114+0000","last_clean":"2024-09-15T22:38:25.010114+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:38:25.010114+0000","last_undegraded":"2024-09-15T22:38:25.010114+0000","last_fullsized":"2024-09-15T22:38:25.010114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124205+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:38:25.124205+0000","last_peered":"2024-09-15T22:38:25.124205+0000","last_clean":"2024-09-15T22:38:25.124205+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:38:25.124205+0000","last_undegraded":"2024-09-15T22:38:25.124205+0000","last_fullsized":"2024-09-15T22:38:25.124205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112085+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:38:25.112085+0000","last_peered":"2024-09-15T22:38:25.112085+0000","last_clean":"2024-09-15T22:38:25.112085+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:38:25.112085+0000","last_undegraded":"2024-09-15T22:38:25.112085+0000","last_fullsized":"2024-09-15T22:38:25.112085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108245+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:38:25.108245+0000","last_peered":"2024-09-15T22:38:25.108245+0000","last_clean":"2024-09-15T22:38:25.108245+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:38:25.108245+0000","last_undegraded":"2024-09-15T22:38:25.108245+0000","last_fullsized":"2024-09-15T22:38:25.108245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249437+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:38:25.249437+0000","last_peered":"2024-09-15T22:38:25.249437+0000","last_clean":"2024-09-15T22:38:25.249437+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:38:25.249437+0000","last_undegraded":"2024-09-15T22:38:25.249437+0000","last_fullsized":"2024-09-15T22:38:25.249437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249830+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:38:25.249830+0000","last_peered":"2024-09-15T22:38:25.249830+0000","last_clean":"2024-09-15T22:38:25.249830+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:38:25.249830+0000","last_undegraded":"2024-09-15T22:38:25.249830+0000","last_fullsized":"2024-09-15T22:38:25.249830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242437+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:38:25.242437+0000","last_peered":"2024-09-15T22:38:25.242437+0000","last_clean":"2024-09-15T22:38:25.242437+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:38:25.242437+0000","last_undegraded":"2024-09-15T22:38:25.242437+0000","last_fullsized":"2024-09-15T22:38:25.242437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796407,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1045,"num_read_kb":738,"num_write":424,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2389221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":519982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841583,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29164,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715988,"statfs":{"total":95995035648,"available":95965171712,"internally_reserved":0,"allocated":2068480,"data_stored":1506297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005109,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1454080,"data_stored":917530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1449984,"data_stored":913354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1650688,"data_stored":1117284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626500,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":515394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29088,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716064,"statfs":{"total":95995035648,"available":95965249536,"internally_reserved":0,"allocated":1990656,"data_stored":1440010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":521250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37196,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707956,"statfs":{"total":95995035648,"available":95956946944,"internally_reserved":0,"allocated":1773568,"data_stored":1237036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":586797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":585025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":516782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":400625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:38:38.396 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:38:38.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:38:38.661 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:38.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:38 smithi059 ceph-mon[34872]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-15T22:38:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:38 smithi059 ceph-mon[34872]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:38.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:38 smithi130 ceph-mon[35742]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-15T22:38:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:38 smithi130 ceph-mon[35742]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:38.782 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:38 smithi016 ceph-mon[31492]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-15T22:38:38.782 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:38 smithi016 ceph-mon[31492]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 -- 172.21.15.16:0/332202485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e81034b0 msgr2=0x7f17e810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/332202485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e81034b0 0x7f17e810b9a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009900 tx=0x7f17dc02a480 comp rx=0 tx=0).stop 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 -- 172.21.15.16:0/332202485 shutdown_connections 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/332202485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17e810bfc0 0x7f17e810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/332202485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e81034b0 0x7f17e810b9a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/332202485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17e8102b90 0x7f17e8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 -- 172.21.15.16:0/332202485 >> 172.21.15.16:0/332202485 conn(0x7f17e80fc610 msgr2=0x7f17e80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:39.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 -- 172.21.15.16:0/332202485 shutdown_connections 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.141+0000 7f17ecd9e640 1 -- 172.21.15.16:0/332202485 wait complete. 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 Processor -- start 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 -- start start 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e8102b90 0x7f17e81007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17e81034b0 0x7f17e8100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17e810bfc0 0x7f17e8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17e8111e50 con 0x7f17e8102b90 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17e8111cd0 con 0x7f17e81034b0 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.142+0000 7f17ecd9e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f17e8111fd0 con 0x7f17e810bfc0 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e8102b90 0x7f17e81007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e8102b90 0x7f17e81007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54410/0 (socket says 172.21.15.16:54410) 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e6ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17e81034b0 0x7f17e8100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:39.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 -- 172.21.15.16:0/812470074 learned_addr learned my addr 172.21.15.16:0/812470074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e7fff640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17e810bfc0 0x7f17e8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 -- 172.21.15.16:0/812470074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17e810bfc0 msgr2=0x7f17e8101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17e810bfc0 0x7f17e8101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 -- 172.21.15.16:0/812470074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17e81034b0 msgr2=0x7f17e8100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17e81034b0 0x7f17e8100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 -- 172.21.15.16:0/812470074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e81a67f0 con 0x7f17e8102b90 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e6ffd640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17e81034b0 0x7f17e8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e77fe640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e8102b90 0x7f17e81007f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b4a0 tx=0x7f17d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e4ff9640 1 -- 172.21.15.16:0/812470074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8014070 con 0x7f17e8102b90 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e81a6ae0 con 0x7f17e8102b90 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e4ff9640 1 -- 172.21.15.16:0/812470074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17d8010af0 con 0x7f17e8102b90 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.143+0000 7f17e4ff9640 1 -- 172.21.15.16:0/812470074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8017d10 con 0x7f17e8102b90 2024-09-15T22:38:39.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.144+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e81a7040 con 0x7f17e8102b90 2024-09-15T22:38:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.144+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e8102f70 con 0x7f17e8102b90 2024-09-15T22:38:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.146+0000 7f17e4ff9640 1 -- 172.21.15.16:0/812470074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f17d8017070 con 0x7f17e8102b90 2024-09-15T22:38:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.146+0000 7f17e4ff9640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17d00a3440 0x7f17d00a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.146+0000 7f17e6ffd640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17d00a3440 0x7f17d00a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.146+0000 7f17e4ff9640 1 -- 172.21.15.16:0/812470074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f17d80cf3d0 con 0x7f17e8102b90 2024-09-15T22:38:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.147+0000 7f17e6ffd640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17d00a3440 0x7f17d00a5900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17dc000c00 tx=0x7f17dc031510 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:39.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.150+0000 7f17e4ff9640 1 -- 172.21.15.16:0/812470074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17d809b3a0 con 0x7f17e8102b90 2024-09-15T22:38:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.327+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17e8062630 con 0x7f17d00a3440 2024-09-15T22:38:39.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.336+0000 7f17e4ff9640 1 -- 172.21.15.16:0/812470074 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339407 (secure 0 0 0) 0x7f17e8062630 con 0x7f17d00a3440 2024-09-15T22:38:39.340 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17d00a3440 msgr2=0x7f17d00a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17d00a3440 0x7f17d00a5900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17dc000c00 tx=0x7f17dc031510 comp rx=0 tx=0).stop 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e8102b90 msgr2=0x7f17e81007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e8102b90 0x7f17e81007f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b4a0 tx=0x7f17d800b960 comp rx=0 tx=0).stop 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 shutdown_connections 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17d00a3440 0x7f17d00a5900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f17e810bfc0 0x7f17e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17e81034b0 0x7f17e8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 --2- 172.21.15.16:0/812470074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e8102b90 0x7f17e81007f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 >> 172.21.15.16:0/812470074 conn(0x7f17e80fc610 msgr2=0x7f17e80fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 shutdown_connections 2024-09-15T22:38:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:39.340+0000 7f17ecd9e640 1 -- 172.21.15.16:0/812470074 wait complete. 2024-09-15T22:38:40.003 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-15T22:38:39.022523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387009,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1282,"num_read_kb":1068,"num_write":627,"num_write_kb":5298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":962,"ondisk_log_size":962,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":415976,"kb_used_data":16424,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124525848,"statfs":{"total":1151940427776,"available":1151514468352,"internally_reserved":0,"allocated":16818176,"data_stored":10376741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":408532284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":416,"num_read_kb":473,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002945"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108817+0000","last_change":"2024-09-15T22:36:06.220762+0000","last_active":"2024-09-15T22:38:25.108817+0000","last_peered":"2024-09-15T22:38:25.108817+0000","last_clean":"2024-09-15T22:38:25.108817+0000","last_became_active":"2024-09-15T22:36:06.220464+0000","last_became_peered":"2024-09-15T22:36:06.220464+0000","last_unstale":"2024-09-15T22:38:25.108817+0000","last_undegraded":"2024-09-15T22:38:25.108817+0000","last_fullsized":"2024-09-15T22:38:25.108817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:40.096983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114924+0000","last_change":"2024-09-15T22:36:08.252700+0000","last_active":"2024-09-15T22:38:25.114924+0000","last_peered":"2024-09-15T22:38:25.114924+0000","last_clean":"2024-09-15T22:38:25.114924+0000","last_became_active":"2024-09-15T22:36:08.252420+0000","last_became_peered":"2024-09-15T22:36:08.252420+0000","last_unstale":"2024-09-15T22:38:25.114924+0000","last_undegraded":"2024-09-15T22:38:25.114924+0000","last_fullsized":"2024-09-15T22:38:25.114924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:36:57.344803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241365+0000","last_change":"2024-09-15T22:36:15.292198+0000","last_active":"2024-09-15T22:38:25.241365+0000","last_peered":"2024-09-15T22:38:25.241365+0000","last_clean":"2024-09-15T22:38:25.241365+0000","last_became_active":"2024-09-15T22:36:15.291789+0000","last_became_peered":"2024-09-15T22:36:15.291789+0000","last_unstale":"2024-09-15T22:38:25.241365+0000","last_undegraded":"2024-09-15T22:38:25.241365+0000","last_fullsized":"2024-09-15T22:38:25.241365+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:54.019630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115300+0000","last_change":"2024-09-15T22:36:06.335462+0000","last_active":"2024-09-15T22:38:25.115300+0000","last_peered":"2024-09-15T22:38:25.115300+0000","last_clean":"2024-09-15T22:38:25.115300+0000","last_became_active":"2024-09-15T22:36:06.335156+0000","last_became_peered":"2024-09-15T22:36:06.335156+0000","last_unstale":"2024-09-15T22:38:25.115300+0000","last_undegraded":"2024-09-15T22:38:25.115300+0000","last_fullsized":"2024-09-15T22:38:25.115300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:27.845899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250490+0000","last_change":"2024-09-15T22:36:08.247966+0000","last_active":"2024-09-15T22:38:25.250490+0000","last_peered":"2024-09-15T22:38:25.250490+0000","last_clean":"2024-09-15T22:38:25.250490+0000","last_became_active":"2024-09-15T22:36:08.247639+0000","last_became_peered":"2024-09-15T22:36:08.247639+0000","last_unstale":"2024-09-15T22:38:25.250490+0000","last_undegraded":"2024-09-15T22:38:25.250490+0000","last_fullsized":"2024-09-15T22:38:25.250490+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.534098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240846+0000","last_change":"2024-09-15T22:37:12.600028+0000","last_active":"2024-09-15T22:38:25.240846+0000","last_peered":"2024-09-15T22:38:25.240846+0000","last_clean":"2024-09-15T22:38:25.240846+0000","last_became_active":"2024-09-15T22:37:12.599585+0000","last_became_peered":"2024-09-15T22:37:12.599585+0000","last_unstale":"2024-09-15T22:38:25.240846+0000","last_undegraded":"2024-09-15T22:38:25.240846+0000","last_fullsized":"2024-09-15T22:38:25.240846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:21.145113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112544+0000","last_change":"2024-09-15T22:36:06.224719+0000","last_active":"2024-09-15T22:38:25.112544+0000","last_peered":"2024-09-15T22:38:25.112544+0000","last_clean":"2024-09-15T22:38:25.112544+0000","last_became_active":"2024-09-15T22:36:06.224307+0000","last_became_peered":"2024-09-15T22:36:06.224307+0000","last_unstale":"2024-09-15T22:38:25.112544+0000","last_undegraded":"2024-09-15T22:38:25.112544+0000","last_fullsized":"2024-09-15T22:38:25.112544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:48.397806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241710+0000","last_change":"2024-09-15T22:36:08.262745+0000","last_active":"2024-09-15T22:38:25.241710+0000","last_peered":"2024-09-15T22:38:25.241710+0000","last_clean":"2024-09-15T22:38:25.241710+0000","last_became_active":"2024-09-15T22:36:08.261329+0000","last_became_peered":"2024-09-15T22:36:08.261329+0000","last_unstale":"2024-09-15T22:38:25.241710+0000","last_undegraded":"2024-09-15T22:38:25.241710+0000","last_fullsized":"2024-09-15T22:38:25.241710+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:08.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241001+0000","last_change":"2024-09-15T22:36:15.303427+0000","last_active":"2024-09-15T22:38:25.241001+0000","last_peered":"2024-09-15T22:38:25.241001+0000","last_clean":"2024-09-15T22:38:25.241001+0000","last_became_active":"2024-09-15T22:36:15.302770+0000","last_became_peered":"2024-09-15T22:36:15.302770+0000","last_unstale":"2024-09-15T22:38:25.241001+0000","last_undegraded":"2024-09-15T22:38:25.241001+0000","last_fullsized":"2024-09-15T22:38:25.241001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:08.735900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112377+0000","last_change":"2024-09-15T22:36:06.224616+0000","last_active":"2024-09-15T22:38:25.112377+0000","last_peered":"2024-09-15T22:38:25.112377+0000","last_clean":"2024-09-15T22:38:25.112377+0000","last_became_active":"2024-09-15T22:36:06.224245+0000","last_became_peered":"2024-09-15T22:36:06.224245+0000","last_unstale":"2024-09-15T22:38:25.112377+0000","last_undegraded":"2024-09-15T22:38:25.112377+0000","last_fullsized":"2024-09-15T22:38:25.112377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:36.320690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009573+0000","last_change":"2024-09-15T22:36:08.248034+0000","last_active":"2024-09-15T22:38:25.009573+0000","last_peered":"2024-09-15T22:38:25.009573+0000","last_clean":"2024-09-15T22:38:25.009573+0000","last_became_active":"2024-09-15T22:36:08.247724+0000","last_became_peered":"2024-09-15T22:36:08.247724+0000","last_unstale":"2024-09-15T22:38:25.009573+0000","last_undegraded":"2024-09-15T22:38:25.009573+0000","last_fullsized":"2024-09-15T22:38:25.009573+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:06.474455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.240840+0000","last_change":"2024-09-15T22:36:15.303571+0000","last_active":"2024-09-15T22:38:25.240840+0000","last_peered":"2024-09-15T22:38:25.240840+0000","last_clean":"2024-09-15T22:38:25.240840+0000","last_became_active":"2024-09-15T22:36:15.302671+0000","last_became_peered":"2024-09-15T22:36:15.302671+0000","last_unstale":"2024-09-15T22:38:25.240840+0000","last_undegraded":"2024-09-15T22:38:25.240840+0000","last_fullsized":"2024-09-15T22:38:25.240840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:48.044608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010739+0000","last_change":"2024-09-15T22:36:08.261918+0000","last_active":"2024-09-15T22:38:25.010739+0000","last_peered":"2024-09-15T22:38:25.010739+0000","last_clean":"2024-09-15T22:38:25.010739+0000","last_became_active":"2024-09-15T22:36:08.261572+0000","last_became_peered":"2024-09-15T22:36:08.261572+0000","last_unstale":"2024-09-15T22:38:25.010739+0000","last_undegraded":"2024-09-15T22:38:25.010739+0000","last_fullsized":"2024-09-15T22:38:25.010739+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:52:52.327647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241387+0000","last_change":"2024-09-15T22:37:12.529149+0000","last_active":"2024-09-15T22:38:25.241387+0000","last_peered":"2024-09-15T22:38:25.241387+0000","last_clean":"2024-09-15T22:38:25.241387+0000","last_became_active":"2024-09-15T22:37:12.524758+0000","last_became_peered":"2024-09-15T22:37:12.524758+0000","last_unstale":"2024-09-15T22:38:25.241387+0000","last_undegraded":"2024-09-15T22:38:25.241387+0000","last_fullsized":"2024-09-15T22:38:25.241387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:32.091988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241362+0000","last_change":"2024-09-15T22:36:15.303490+0000","last_active":"2024-09-15T22:38:25.241362+0000","last_peered":"2024-09-15T22:38:25.241362+0000","last_clean":"2024-09-15T22:38:25.241362+0000","last_became_active":"2024-09-15T22:36:15.302956+0000","last_became_peered":"2024-09-15T22:36:15.302956+0000","last_unstale":"2024-09-15T22:38:25.241362+0000","last_undegraded":"2024-09-15T22:38:25.241362+0000","last_fullsized":"2024-09-15T22:38:25.241362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:44.487359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113745+0000","last_change":"2024-09-15T22:36:08.256690+0000","last_active":"2024-09-15T22:38:25.113745+0000","last_peered":"2024-09-15T22:38:25.113745+0000","last_clean":"2024-09-15T22:38:25.113745+0000","last_became_active":"2024-09-15T22:36:08.256360+0000","last_became_peered":"2024-09-15T22:36:08.256360+0000","last_unstale":"2024-09-15T22:38:25.113745+0000","last_undegraded":"2024-09-15T22:38:25.113745+0000","last_fullsized":"2024-09-15T22:38:25.113745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:24.554106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241575+0000","last_change":"2024-09-15T22:36:06.220673+0000","last_active":"2024-09-15T22:38:25.241575+0000","last_peered":"2024-09-15T22:38:25.241575+0000","last_clean":"2024-09-15T22:38:25.241575+0000","last_became_active":"2024-09-15T22:36:06.220335+0000","last_became_peered":"2024-09-15T22:36:06.220335+0000","last_unstale":"2024-09-15T22:38:25.241575+0000","last_undegraded":"2024-09-15T22:38:25.241575+0000","last_fullsized":"2024-09-15T22:38:25.241575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:04.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241265+0000","last_change":"2024-09-15T22:36:15.297180+0000","last_active":"2024-09-15T22:38:25.241265+0000","last_peered":"2024-09-15T22:38:25.241265+0000","last_clean":"2024-09-15T22:38:25.241265+0000","last_became_active":"2024-09-15T22:36:15.296719+0000","last_became_peered":"2024-09-15T22:36:15.296719+0000","last_unstale":"2024-09-15T22:38:25.241265+0000","last_undegraded":"2024-09-15T22:38:25.241265+0000","last_fullsized":"2024-09-15T22:38:25.241265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:47.387290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241234+0000","last_change":"2024-09-15T22:36:08.250798+0000","last_active":"2024-09-15T22:38:25.241234+0000","last_peered":"2024-09-15T22:38:25.241234+0000","last_clean":"2024-09-15T22:38:25.241234+0000","last_became_active":"2024-09-15T22:36:08.250517+0000","last_became_peered":"2024-09-15T22:36:08.250517+0000","last_unstale":"2024-09-15T22:38:25.241234+0000","last_undegraded":"2024-09-15T22:38:25.241234+0000","last_fullsized":"2024-09-15T22:38:25.241234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:58.714072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.114246+0000","last_change":"2024-09-15T22:36:06.213456+0000","last_active":"2024-09-15T22:38:25.114246+0000","last_peered":"2024-09-15T22:38:25.114246+0000","last_clean":"2024-09-15T22:38:25.114246+0000","last_became_active":"2024-09-15T22:36:06.213062+0000","last_became_peered":"2024-09-15T22:36:06.213062+0000","last_unstale":"2024-09-15T22:38:25.114246+0000","last_undegraded":"2024-09-15T22:38:25.114246+0000","last_fullsized":"2024-09-15T22:38:25.114246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:06.960329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108657+0000","last_change":"2024-09-15T22:36:15.301258+0000","last_active":"2024-09-15T22:38:25.108657+0000","last_peered":"2024-09-15T22:38:25.108657+0000","last_clean":"2024-09-15T22:38:25.108657+0000","last_became_active":"2024-09-15T22:36:15.300849+0000","last_became_peered":"2024-09-15T22:36:15.300849+0000","last_unstale":"2024-09-15T22:38:25.108657+0000","last_undegraded":"2024-09-15T22:38:25.108657+0000","last_fullsized":"2024-09-15T22:38:25.108657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:58.436857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112056+0000","last_change":"2024-09-15T22:36:08.256411+0000","last_active":"2024-09-15T22:38:25.112056+0000","last_peered":"2024-09-15T22:38:25.112056+0000","last_clean":"2024-09-15T22:38:25.112056+0000","last_became_active":"2024-09-15T22:36:08.256061+0000","last_became_peered":"2024-09-15T22:36:08.256061+0000","last_unstale":"2024-09-15T22:38:25.112056+0000","last_undegraded":"2024-09-15T22:38:25.112056+0000","last_fullsized":"2024-09-15T22:38:25.112056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:31.457482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241262+0000","last_change":"2024-09-15T22:36:06.337887+0000","last_active":"2024-09-15T22:38:25.241262+0000","last_peered":"2024-09-15T22:38:25.241262+0000","last_clean":"2024-09-15T22:38:25.241262+0000","last_became_active":"2024-09-15T22:36:06.337476+0000","last_became_peered":"2024-09-15T22:36:06.337476+0000","last_unstale":"2024-09-15T22:38:25.241262+0000","last_undegraded":"2024-09-15T22:38:25.241262+0000","last_fullsized":"2024-09-15T22:38:25.241262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:06.205915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241293+0000","last_change":"2024-09-15T22:37:12.602394+0000","last_active":"2024-09-15T22:38:25.241293+0000","last_peered":"2024-09-15T22:38:25.241293+0000","last_clean":"2024-09-15T22:38:25.241293+0000","last_became_active":"2024-09-15T22:37:12.601896+0000","last_became_peered":"2024-09-15T22:37:12.601896+0000","last_unstale":"2024-09-15T22:38:25.241293+0000","last_undegraded":"2024-09-15T22:38:25.241293+0000","last_fullsized":"2024-09-15T22:38:25.241293+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:19:59.991455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249966+0000","last_change":"2024-09-15T22:36:08.257069+0000","last_active":"2024-09-15T22:38:25.249966+0000","last_peered":"2024-09-15T22:38:25.249966+0000","last_clean":"2024-09-15T22:38:25.249966+0000","last_became_active":"2024-09-15T22:36:08.256590+0000","last_became_peered":"2024-09-15T22:36:08.256590+0000","last_unstale":"2024-09-15T22:38:25.249966+0000","last_undegraded":"2024-09-15T22:38:25.249966+0000","last_fullsized":"2024-09-15T22:38:25.249966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:12:38.034503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241375+0000","last_change":"2024-09-15T22:36:06.225076+0000","last_active":"2024-09-15T22:38:25.241375+0000","last_peered":"2024-09-15T22:38:25.241375+0000","last_clean":"2024-09-15T22:38:25.241375+0000","last_became_active":"2024-09-15T22:36:06.224480+0000","last_became_peered":"2024-09-15T22:36:06.224480+0000","last_unstale":"2024-09-15T22:38:25.241375+0000","last_undegraded":"2024-09-15T22:38:25.241375+0000","last_fullsized":"2024-09-15T22:38:25.241375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:06.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241418+0000","last_change":"2024-09-15T22:36:15.290678+0000","last_active":"2024-09-15T22:38:25.241418+0000","last_peered":"2024-09-15T22:38:25.241418+0000","last_clean":"2024-09-15T22:38:25.241418+0000","last_became_active":"2024-09-15T22:36:15.290244+0000","last_became_peered":"2024-09-15T22:36:15.290244+0000","last_unstale":"2024-09-15T22:38:25.241418+0000","last_undegraded":"2024-09-15T22:38:25.241418+0000","last_fullsized":"2024-09-15T22:38:25.241418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.652841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115826+0000","last_change":"2024-09-15T22:36:08.251538+0000","last_active":"2024-09-15T22:38:25.115826+0000","last_peered":"2024-09-15T22:38:25.115826+0000","last_clean":"2024-09-15T22:38:25.115826+0000","last_became_active":"2024-09-15T22:36:08.251284+0000","last_became_peered":"2024-09-15T22:36:08.251284+0000","last_unstale":"2024-09-15T22:38:25.115826+0000","last_undegraded":"2024-09-15T22:38:25.115826+0000","last_fullsized":"2024-09-15T22:38:25.115826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:12.597849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'101","reported_seq":234,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.578517+0000","last_change":"2024-09-15T22:36:06.220770+0000","last_active":"2024-09-15T22:38:34.578517+0000","last_peered":"2024-09-15T22:38:34.578517+0000","last_clean":"2024-09-15T22:38:34.578517+0000","last_became_active":"2024-09-15T22:36:06.220421+0000","last_became_peered":"2024-09-15T22:36:06.220421+0000","last_unstale":"2024-09-15T22:38:34.578517+0000","last_undegraded":"2024-09-15T22:38:34.578517+0000","last_fullsized":"2024-09-15T22:38:34.578517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:30.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.111983+0000","last_change":"2024-09-15T22:36:15.300666+0000","last_active":"2024-09-15T22:38:25.111983+0000","last_peered":"2024-09-15T22:38:25.111983+0000","last_clean":"2024-09-15T22:38:25.111983+0000","last_became_active":"2024-09-15T22:36:15.300313+0000","last_became_peered":"2024-09-15T22:36:15.300313+0000","last_unstale":"2024-09-15T22:38:25.111983+0000","last_undegraded":"2024-09-15T22:38:25.111983+0000","last_fullsized":"2024-09-15T22:38:25.111983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:04.752769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006514+0000","last_change":"2024-09-15T22:36:08.252685+0000","last_active":"2024-09-15T22:38:25.006514+0000","last_peered":"2024-09-15T22:38:25.006514+0000","last_clean":"2024-09-15T22:38:25.006514+0000","last_became_active":"2024-09-15T22:36:08.252380+0000","last_became_peered":"2024-09-15T22:36:08.252380+0000","last_unstale":"2024-09-15T22:38:25.006514+0000","last_undegraded":"2024-09-15T22:38:25.006514+0000","last_fullsized":"2024-09-15T22:38:25.006514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:39.822606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112492+0000","last_change":"2024-09-15T22:36:06.337919+0000","last_active":"2024-09-15T22:38:25.112492+0000","last_peered":"2024-09-15T22:38:25.112492+0000","last_clean":"2024-09-15T22:38:25.112492+0000","last_became_active":"2024-09-15T22:36:06.337514+0000","last_became_peered":"2024-09-15T22:36:06.337514+0000","last_unstale":"2024-09-15T22:38:25.112492+0000","last_undegraded":"2024-09-15T22:38:25.112492+0000","last_fullsized":"2024-09-15T22:38:25.112492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:49:46.716512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241541+0000","last_change":"2024-09-15T22:36:15.301341+0000","last_active":"2024-09-15T22:38:25.241541+0000","last_peered":"2024-09-15T22:38:25.241541+0000","last_clean":"2024-09-15T22:38:25.241541+0000","last_became_active":"2024-09-15T22:36:15.300855+0000","last_became_peered":"2024-09-15T22:36:15.300855+0000","last_unstale":"2024-09-15T22:38:25.241541+0000","last_undegraded":"2024-09-15T22:38:25.241541+0000","last_fullsized":"2024-09-15T22:38:25.241541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:02.734555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124170+0000","last_change":"2024-09-15T22:36:08.252352+0000","last_active":"2024-09-15T22:38:25.124170+0000","last_peered":"2024-09-15T22:38:25.124170+0000","last_clean":"2024-09-15T22:38:25.124170+0000","last_became_active":"2024-09-15T22:36:08.252039+0000","last_became_peered":"2024-09-15T22:36:08.252039+0000","last_unstale":"2024-09-15T22:38:25.124170+0000","last_undegraded":"2024-09-15T22:38:25.124170+0000","last_fullsized":"2024-09-15T22:38:25.124170+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:10:52.358045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.110438+0000","last_change":"2024-09-15T22:38:14.242090+0000","last_active":"2024-09-15T22:38:25.110438+0000","last_peered":"2024-09-15T22:38:25.110438+0000","last_clean":"2024-09-15T22:38:25.110438+0000","last_became_active":"2024-09-15T22:38:14.241634+0000","last_became_peered":"2024-09-15T22:38:14.241634+0000","last_unstale":"2024-09-15T22:38:25.110438+0000","last_undegraded":"2024-09-15T22:38:25.110438+0000","last_fullsized":"2024-09-15T22:38:25.110438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:13.485506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112062+0000","last_change":"2024-09-15T22:36:15.302306+0000","last_active":"2024-09-15T22:38:25.112062+0000","last_peered":"2024-09-15T22:38:25.112062+0000","last_clean":"2024-09-15T22:38:25.112062+0000","last_became_active":"2024-09-15T22:36:15.301875+0000","last_became_peered":"2024-09-15T22:36:15.301875+0000","last_unstale":"2024-09-15T22:38:25.112062+0000","last_undegraded":"2024-09-15T22:38:25.112062+0000","last_fullsized":"2024-09-15T22:38:25.112062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:02.983439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124297+0000","last_change":"2024-09-15T22:36:08.258778+0000","last_active":"2024-09-15T22:38:25.124297+0000","last_peered":"2024-09-15T22:38:25.124297+0000","last_clean":"2024-09-15T22:38:25.124297+0000","last_became_active":"2024-09-15T22:36:08.258441+0000","last_became_peered":"2024-09-15T22:36:08.258441+0000","last_unstale":"2024-09-15T22:38:25.124297+0000","last_undegraded":"2024-09-15T22:38:25.124297+0000","last_fullsized":"2024-09-15T22:38:25.124297+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:45.152782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010822+0000","last_change":"2024-09-15T22:36:06.218408+0000","last_active":"2024-09-15T22:38:25.010822+0000","last_peered":"2024-09-15T22:38:25.010822+0000","last_clean":"2024-09-15T22:38:25.010822+0000","last_became_active":"2024-09-15T22:36:06.218044+0000","last_became_peered":"2024-09-15T22:36:06.218044+0000","last_unstale":"2024-09-15T22:38:25.010822+0000","last_undegraded":"2024-09-15T22:38:25.010822+0000","last_fullsized":"2024-09-15T22:38:25.010822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:18:57.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006453+0000","last_change":"2024-09-15T22:36:15.299492+0000","last_active":"2024-09-15T22:38:25.006453+0000","last_peered":"2024-09-15T22:38:25.006453+0000","last_clean":"2024-09-15T22:38:25.006453+0000","last_became_active":"2024-09-15T22:36:15.298832+0000","last_became_peered":"2024-09-15T22:36:15.298832+0000","last_unstale":"2024-09-15T22:38:25.006453+0000","last_undegraded":"2024-09-15T22:38:25.006453+0000","last_fullsized":"2024-09-15T22:38:25.006453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:41.866742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009249+0000","last_change":"2024-09-15T22:36:08.253251+0000","last_active":"2024-09-15T22:38:25.009249+0000","last_peered":"2024-09-15T22:38:25.009249+0000","last_clean":"2024-09-15T22:38:25.009249+0000","last_became_active":"2024-09-15T22:36:08.252950+0000","last_became_peered":"2024-09-15T22:36:08.252950+0000","last_unstale":"2024-09-15T22:38:25.009249+0000","last_undegraded":"2024-09-15T22:38:25.009249+0000","last_fullsized":"2024-09-15T22:38:25.009249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:11:38.920449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.549838+0000","last_change":"2024-09-15T22:36:06.335577+0000","last_active":"2024-09-15T22:38:34.549838+0000","last_peered":"2024-09-15T22:38:34.549838+0000","last_clean":"2024-09-15T22:38:34.549838+0000","last_became_active":"2024-09-15T22:36:06.335151+0000","last_became_peered":"2024-09-15T22:36:06.335151+0000","last_unstale":"2024-09-15T22:38:34.549838+0000","last_undegraded":"2024-09-15T22:38:34.549838+0000","last_fullsized":"2024-09-15T22:38:34.549838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:19:44.275225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006955+0000","last_change":"2024-09-15T22:36:15.301364+0000","last_active":"2024-09-15T22:38:25.006955+0000","last_peered":"2024-09-15T22:38:25.006955+0000","last_clean":"2024-09-15T22:38:25.006955+0000","last_became_active":"2024-09-15T22:36:15.300802+0000","last_became_peered":"2024-09-15T22:36:15.300802+0000","last_unstale":"2024-09-15T22:38:25.006955+0000","last_undegraded":"2024-09-15T22:38:25.006955+0000","last_fullsized":"2024-09-15T22:38:25.006955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:48:02.348689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.005813+0000","last_change":"2024-09-15T22:36:08.253527+0000","last_active":"2024-09-15T22:38:25.005813+0000","last_peered":"2024-09-15T22:38:25.005813+0000","last_clean":"2024-09-15T22:38:25.005813+0000","last_became_active":"2024-09-15T22:36:08.253220+0000","last_became_peered":"2024-09-15T22:36:08.253220+0000","last_unstale":"2024-09-15T22:38:25.005813+0000","last_undegraded":"2024-09-15T22:38:25.005813+0000","last_fullsized":"2024-09-15T22:38:25.005813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:56:49.318251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010164+0000","last_change":"2024-09-15T22:36:06.223332+0000","last_active":"2024-09-15T22:38:25.010164+0000","last_peered":"2024-09-15T22:38:25.010164+0000","last_clean":"2024-09-15T22:38:25.010164+0000","last_became_active":"2024-09-15T22:36:06.222914+0000","last_became_peered":"2024-09-15T22:36:06.222914+0000","last_unstale":"2024-09-15T22:38:25.010164+0000","last_undegraded":"2024-09-15T22:38:25.010164+0000","last_fullsized":"2024-09-15T22:38:25.010164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:06:08.338666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009598+0000","last_change":"2024-09-15T22:36:15.305187+0000","last_active":"2024-09-15T22:38:25.009598+0000","last_peered":"2024-09-15T22:38:25.009598+0000","last_clean":"2024-09-15T22:38:25.009598+0000","last_became_active":"2024-09-15T22:36:15.304782+0000","last_became_peered":"2024-09-15T22:36:15.304782+0000","last_unstale":"2024-09-15T22:38:25.009598+0000","last_undegraded":"2024-09-15T22:38:25.009598+0000","last_fullsized":"2024-09-15T22:38:25.009598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.879003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.113256+0000","last_change":"2024-09-15T22:36:08.259173+0000","last_active":"2024-09-15T22:38:25.113256+0000","last_peered":"2024-09-15T22:38:25.113256+0000","last_clean":"2024-09-15T22:38:25.113256+0000","last_became_active":"2024-09-15T22:36:08.258872+0000","last_became_peered":"2024-09-15T22:36:08.258872+0000","last_unstale":"2024-09-15T22:38:25.113256+0000","last_undegraded":"2024-09-15T22:38:25.113256+0000","last_fullsized":"2024-09-15T22:38:25.113256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:21:04.491920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241238+0000","last_change":"2024-09-15T22:36:06.221679+0000","last_active":"2024-09-15T22:38:25.241238+0000","last_peered":"2024-09-15T22:38:25.241238+0000","last_clean":"2024-09-15T22:38:25.241238+0000","last_became_active":"2024-09-15T22:36:06.220553+0000","last_became_peered":"2024-09-15T22:36:06.220553+0000","last_unstale":"2024-09-15T22:38:25.241238+0000","last_undegraded":"2024-09-15T22:38:25.241238+0000","last_fullsized":"2024-09-15T22:38:25.241238+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:31.009758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":404,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:27.116484+0000","last_change":"2024-09-15T22:34:56.428149+0000","last_active":"2024-09-15T22:38:27.116484+0000","last_peered":"2024-09-15T22:38:27.116484+0000","last_clean":"2024-09-15T22:38:27.116484+0000","last_became_active":"2024-09-15T22:34:56.427599+0000","last_became_peered":"2024-09-15T22:34:56.427599+0000","last_unstale":"2024-09-15T22:38:27.116484+0000","last_undegraded":"2024-09-15T22:38:27.116484+0000","last_fullsized":"2024-09-15T22:38:27.116484+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:31:31.198930+0000","last_clean_scrub_stamp":"2024-09-15T22:31:31.198930+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:07.314568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"85'1","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010316+0000","last_change":"2024-09-15T22:36:15.305018+0000","last_active":"2024-09-15T22:38:25.010316+0000","last_peered":"2024-09-15T22:38:25.010316+0000","last_clean":"2024-09-15T22:38:25.010316+0000","last_became_active":"2024-09-15T22:36:15.304566+0000","last_became_peered":"2024-09-15T22:36:15.304566+0000","last_unstale":"2024-09-15T22:38:25.010316+0000","last_undegraded":"2024-09-15T22:38:25.010316+0000","last_fullsized":"2024-09-15T22:38:25.010316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:42.174755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010530+0000","last_change":"2024-09-15T22:36:08.263227+0000","last_active":"2024-09-15T22:38:25.010530+0000","last_peered":"2024-09-15T22:38:25.010530+0000","last_clean":"2024-09-15T22:38:25.010530+0000","last_became_active":"2024-09-15T22:36:08.262755+0000","last_became_peered":"2024-09-15T22:36:08.262755+0000","last_unstale":"2024-09-15T22:38:25.010530+0000","last_undegraded":"2024-09-15T22:38:25.010530+0000","last_fullsized":"2024-09-15T22:38:25.010530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:31.732776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241899+0000","last_change":"2024-09-15T22:36:06.335941+0000","last_active":"2024-09-15T22:38:25.241899+0000","last_peered":"2024-09-15T22:38:25.241899+0000","last_clean":"2024-09-15T22:38:25.241899+0000","last_became_active":"2024-09-15T22:36:06.335557+0000","last_became_peered":"2024-09-15T22:36:06.335557+0000","last_unstale":"2024-09-15T22:38:25.241899+0000","last_undegraded":"2024-09-15T22:38:25.241899+0000","last_fullsized":"2024-09-15T22:38:25.241899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:52.699407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241629+0000","last_change":"2024-09-15T22:37:12.603280+0000","last_active":"2024-09-15T22:38:25.241629+0000","last_peered":"2024-09-15T22:38:25.241629+0000","last_clean":"2024-09-15T22:38:25.241629+0000","last_became_active":"2024-09-15T22:37:12.602561+0000","last_became_peered":"2024-09-15T22:37:12.602561+0000","last_unstale":"2024-09-15T22:38:25.241629+0000","last_undegraded":"2024-09-15T22:38:25.241629+0000","last_fullsized":"2024-09-15T22:38:25.241629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:52.826104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010356+0000","last_change":"2024-09-15T22:36:08.253991+0000","last_active":"2024-09-15T22:38:25.010356+0000","last_peered":"2024-09-15T22:38:25.010356+0000","last_clean":"2024-09-15T22:38:25.010356+0000","last_became_active":"2024-09-15T22:36:08.253622+0000","last_became_peered":"2024-09-15T22:36:08.253622+0000","last_unstale":"2024-09-15T22:38:25.010356+0000","last_undegraded":"2024-09-15T22:38:25.010356+0000","last_fullsized":"2024-09-15T22:38:25.010356+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:11.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'65","reported_seq":137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.186113+0000","last_change":"2024-09-15T22:36:06.219611+0000","last_active":"2024-09-15T22:38:34.186113+0000","last_peered":"2024-09-15T22:38:34.186113+0000","last_clean":"2024-09-15T22:38:34.186113+0000","last_became_active":"2024-09-15T22:36:06.219264+0000","last_became_peered":"2024-09-15T22:36:06.219264+0000","last_unstale":"2024-09-15T22:38:34.186113+0000","last_undegraded":"2024-09-15T22:38:34.186113+0000","last_fullsized":"2024-09-15T22:38:34.186113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:47.575008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248973+0000","last_change":"2024-09-15T22:36:15.292780+0000","last_active":"2024-09-15T22:38:25.248973+0000","last_peered":"2024-09-15T22:38:25.248973+0000","last_clean":"2024-09-15T22:38:25.248973+0000","last_became_active":"2024-09-15T22:36:15.292416+0000","last_became_peered":"2024-09-15T22:36:15.292416+0000","last_unstale":"2024-09-15T22:38:25.248973+0000","last_undegraded":"2024-09-15T22:38:25.248973+0000","last_fullsized":"2024-09-15T22:38:25.248973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:30:25.176918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010556+0000","last_change":"2024-09-15T22:36:08.256131+0000","last_active":"2024-09-15T22:38:25.010556+0000","last_peered":"2024-09-15T22:38:25.010556+0000","last_clean":"2024-09-15T22:38:25.010556+0000","last_became_active":"2024-09-15T22:36:08.255816+0000","last_became_peered":"2024-09-15T22:36:08.255816+0000","last_unstale":"2024-09-15T22:38:25.010556+0000","last_undegraded":"2024-09-15T22:38:25.010556+0000","last_fullsized":"2024-09-15T22:38:25.010556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:56.514386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006902+0000","last_change":"2024-09-15T22:36:06.218187+0000","last_active":"2024-09-15T22:38:25.006902+0000","last_peered":"2024-09-15T22:38:25.006902+0000","last_clean":"2024-09-15T22:38:25.006902+0000","last_became_active":"2024-09-15T22:36:06.217797+0000","last_became_peered":"2024-09-15T22:36:06.217797+0000","last_unstale":"2024-09-15T22:38:25.006902+0000","last_undegraded":"2024-09-15T22:38:25.006902+0000","last_fullsized":"2024-09-15T22:38:25.006902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:21:21.624670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009099+0000","last_change":"2024-09-15T22:36:15.300749+0000","last_active":"2024-09-15T22:38:25.009099+0000","last_peered":"2024-09-15T22:38:25.009099+0000","last_clean":"2024-09-15T22:38:25.009099+0000","last_became_active":"2024-09-15T22:36:15.297233+0000","last_became_peered":"2024-09-15T22:36:15.297233+0000","last_unstale":"2024-09-15T22:38:25.009099+0000","last_undegraded":"2024-09-15T22:38:25.009099+0000","last_fullsized":"2024-09-15T22:38:25.009099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:30.333714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242337+0000","last_change":"2024-09-15T22:37:12.522385+0000","last_active":"2024-09-15T22:38:25.242337+0000","last_peered":"2024-09-15T22:38:25.242337+0000","last_clean":"2024-09-15T22:38:25.242337+0000","last_became_active":"2024-09-15T22:37:12.521982+0000","last_became_peered":"2024-09-15T22:37:12.521982+0000","last_unstale":"2024-09-15T22:38:25.242337+0000","last_undegraded":"2024-09-15T22:38:25.242337+0000","last_fullsized":"2024-09-15T22:38:25.242337+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:30.654646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.395057+0000","last_change":"2024-09-15T22:36:06.224408+0000","last_active":"2024-09-15T22:38:25.395057+0000","last_peered":"2024-09-15T22:38:25.395057+0000","last_clean":"2024-09-15T22:38:25.395057+0000","last_became_active":"2024-09-15T22:36:06.223859+0000","last_became_peered":"2024-09-15T22:36:06.223859+0000","last_unstale":"2024-09-15T22:38:25.395057+0000","last_undegraded":"2024-09-15T22:38:25.395057+0000","last_fullsized":"2024-09-15T22:38:25.395057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:45.924896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009234+0000","last_change":"2024-09-15T22:36:15.300173+0000","last_active":"2024-09-15T22:38:25.009234+0000","last_peered":"2024-09-15T22:38:25.009234+0000","last_clean":"2024-09-15T22:38:25.009234+0000","last_became_active":"2024-09-15T22:36:15.299613+0000","last_became_peered":"2024-09-15T22:36:15.299613+0000","last_unstale":"2024-09-15T22:38:25.009234+0000","last_undegraded":"2024-09-15T22:38:25.009234+0000","last_fullsized":"2024-09-15T22:38:25.009234+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:09.254397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009429+0000","last_change":"2024-09-15T22:36:08.249843+0000","last_active":"2024-09-15T22:38:25.009429+0000","last_peered":"2024-09-15T22:38:25.009429+0000","last_clean":"2024-09-15T22:38:25.009429+0000","last_became_active":"2024-09-15T22:36:08.249557+0000","last_became_peered":"2024-09-15T22:36:08.249557+0000","last_unstale":"2024-09-15T22:38:25.009429+0000","last_undegraded":"2024-09-15T22:38:25.009429+0000","last_fullsized":"2024-09-15T22:38:25.009429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:46.731099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112094+0000","last_change":"2024-09-15T22:37:12.535528+0000","last_active":"2024-09-15T22:38:25.112094+0000","last_peered":"2024-09-15T22:38:25.112094+0000","last_clean":"2024-09-15T22:38:25.112094+0000","last_became_active":"2024-09-15T22:37:12.518367+0000","last_became_peered":"2024-09-15T22:37:12.518367+0000","last_unstale":"2024-09-15T22:38:25.112094+0000","last_undegraded":"2024-09-15T22:38:25.112094+0000","last_fullsized":"2024-09-15T22:38:25.112094+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:35:02.029955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006770+0000","last_change":"2024-09-15T22:37:12.600316+0000","last_active":"2024-09-15T22:38:25.006770+0000","last_peered":"2024-09-15T22:38:25.006770+0000","last_clean":"2024-09-15T22:38:25.006770+0000","last_became_active":"2024-09-15T22:37:12.599840+0000","last_became_peered":"2024-09-15T22:37:12.599840+0000","last_unstale":"2024-09-15T22:38:25.006770+0000","last_undegraded":"2024-09-15T22:38:25.006770+0000","last_fullsized":"2024-09-15T22:38:25.006770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:31.607659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010143+0000","last_change":"2024-09-15T22:36:08.255210+0000","last_active":"2024-09-15T22:38:25.010143+0000","last_peered":"2024-09-15T22:38:25.010143+0000","last_clean":"2024-09-15T22:38:25.010143+0000","last_became_active":"2024-09-15T22:36:08.254898+0000","last_became_peered":"2024-09-15T22:36:08.254898+0000","last_unstale":"2024-09-15T22:38:25.010143+0000","last_undegraded":"2024-09-15T22:38:25.010143+0000","last_fullsized":"2024-09-15T22:38:25.010143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:35.626884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.550674+0000","last_change":"2024-09-15T22:36:06.220368+0000","last_active":"2024-09-15T22:38:34.550674+0000","last_peered":"2024-09-15T22:38:34.550674+0000","last_clean":"2024-09-15T22:38:34.550674+0000","last_became_active":"2024-09-15T22:36:06.219928+0000","last_became_peered":"2024-09-15T22:36:06.219928+0000","last_unstale":"2024-09-15T22:38:34.550674+0000","last_undegraded":"2024-09-15T22:38:34.550674+0000","last_fullsized":"2024-09-15T22:38:34.550674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:17.943380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1682,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108636+0000","last_change":"2024-09-15T22:36:15.291171+0000","last_active":"2024-09-15T22:38:25.108636+0000","last_peered":"2024-09-15T22:38:25.108636+0000","last_clean":"2024-09-15T22:38:25.108636+0000","last_became_active":"2024-09-15T22:36:15.290805+0000","last_became_peered":"2024-09-15T22:36:15.290805+0000","last_unstale":"2024-09-15T22:38:25.108636+0000","last_undegraded":"2024-09-15T22:38:25.108636+0000","last_fullsized":"2024-09-15T22:38:25.108636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:51.994072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010315+0000","last_change":"2024-09-15T22:36:08.257274+0000","last_active":"2024-09-15T22:38:25.010315+0000","last_peered":"2024-09-15T22:38:25.010315+0000","last_clean":"2024-09-15T22:38:25.010315+0000","last_became_active":"2024-09-15T22:36:08.256859+0000","last_became_peered":"2024-09-15T22:36:08.256859+0000","last_unstale":"2024-09-15T22:38:25.010315+0000","last_undegraded":"2024-09-15T22:38:25.010315+0000","last_fullsized":"2024-09-15T22:38:25.010315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:24.402890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009653+0000","last_change":"2024-09-15T22:36:06.220081+0000","last_active":"2024-09-15T22:38:25.009653+0000","last_peered":"2024-09-15T22:38:25.009653+0000","last_clean":"2024-09-15T22:38:25.009653+0000","last_became_active":"2024-09-15T22:36:06.219743+0000","last_became_peered":"2024-09-15T22:36:06.219743+0000","last_unstale":"2024-09-15T22:38:25.009653+0000","last_undegraded":"2024-09-15T22:38:25.009653+0000","last_fullsized":"2024-09-15T22:38:25.009653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:42.692491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241984+0000","last_change":"2024-09-15T22:36:15.302491+0000","last_active":"2024-09-15T22:38:25.241984+0000","last_peered":"2024-09-15T22:38:25.241984+0000","last_clean":"2024-09-15T22:38:25.241984+0000","last_became_active":"2024-09-15T22:36:15.302080+0000","last_became_peered":"2024-09-15T22:36:15.302080+0000","last_unstale":"2024-09-15T22:38:25.241984+0000","last_undegraded":"2024-09-15T22:38:25.241984+0000","last_fullsized":"2024-09-15T22:38:25.241984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:06:13.555191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006052+0000","last_change":"2024-09-15T22:36:08.250149+0000","last_active":"2024-09-15T22:38:25.006052+0000","last_peered":"2024-09-15T22:38:25.006052+0000","last_clean":"2024-09-15T22:38:25.006052+0000","last_became_active":"2024-09-15T22:36:08.249799+0000","last_became_peered":"2024-09-15T22:36:08.249799+0000","last_unstale":"2024-09-15T22:38:25.006052+0000","last_undegraded":"2024-09-15T22:38:25.006052+0000","last_fullsized":"2024-09-15T22:38:25.006052+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:57:06.058888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241792+0000","last_change":"2024-09-15T22:36:06.219028+0000","last_active":"2024-09-15T22:38:25.241792+0000","last_peered":"2024-09-15T22:38:25.241792+0000","last_clean":"2024-09-15T22:38:25.241792+0000","last_became_active":"2024-09-15T22:36:06.218646+0000","last_became_peered":"2024-09-15T22:36:06.218646+0000","last_unstale":"2024-09-15T22:38:25.241792+0000","last_undegraded":"2024-09-15T22:38:25.241792+0000","last_fullsized":"2024-09-15T22:38:25.241792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:34:27.257843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009484+0000","last_change":"2024-09-15T22:36:15.301358+0000","last_active":"2024-09-15T22:38:25.009484+0000","last_peered":"2024-09-15T22:38:25.009484+0000","last_clean":"2024-09-15T22:38:25.009484+0000","last_became_active":"2024-09-15T22:36:15.300966+0000","last_became_peered":"2024-09-15T22:36:15.300966+0000","last_unstale":"2024-09-15T22:38:25.009484+0000","last_undegraded":"2024-09-15T22:38:25.009484+0000","last_fullsized":"2024-09-15T22:38:25.009484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:45.251553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112424+0000","last_change":"2024-09-15T22:36:08.257608+0000","last_active":"2024-09-15T22:38:25.112424+0000","last_peered":"2024-09-15T22:38:25.112424+0000","last_clean":"2024-09-15T22:38:25.112424+0000","last_became_active":"2024-09-15T22:36:08.256886+0000","last_became_peered":"2024-09-15T22:36:08.256886+0000","last_unstale":"2024-09-15T22:38:25.112424+0000","last_undegraded":"2024-09-15T22:38:25.112424+0000","last_fullsized":"2024-09-15T22:38:25.112424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:19:40.143967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":316,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:35.549922+0000","last_change":"2024-09-15T22:36:06.218663+0000","last_active":"2024-09-15T22:38:35.549922+0000","last_peered":"2024-09-15T22:38:35.549922+0000","last_clean":"2024-09-15T22:38:35.549922+0000","last_became_active":"2024-09-15T22:36:06.218242+0000","last_became_peered":"2024-09-15T22:36:06.218242+0000","last_unstale":"2024-09-15T22:38:35.549922+0000","last_undegraded":"2024-09-15T22:38:35.549922+0000","last_fullsized":"2024-09-15T22:38:35.549922+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:30.527162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69721,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":186,"num_write":41,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107742+0000","last_change":"2024-09-15T22:36:15.301592+0000","last_active":"2024-09-15T22:38:25.107742+0000","last_peered":"2024-09-15T22:38:25.107742+0000","last_clean":"2024-09-15T22:38:25.107742+0000","last_became_active":"2024-09-15T22:36:15.301161+0000","last_became_peered":"2024-09-15T22:36:15.301161+0000","last_unstale":"2024-09-15T22:38:25.107742+0000","last_undegraded":"2024-09-15T22:38:25.107742+0000","last_fullsized":"2024-09-15T22:38:25.107742+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:45.047002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241767+0000","last_change":"2024-09-15T22:36:08.262431+0000","last_active":"2024-09-15T22:38:25.241767+0000","last_peered":"2024-09-15T22:38:25.241767+0000","last_clean":"2024-09-15T22:38:25.241767+0000","last_became_active":"2024-09-15T22:36:08.261860+0000","last_became_peered":"2024-09-15T22:36:08.261860+0000","last_unstale":"2024-09-15T22:38:25.241767+0000","last_undegraded":"2024-09-15T22:38:25.241767+0000","last_fullsized":"2024-09-15T22:38:25.241767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:09.459747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009858+0000","last_change":"2024-09-15T22:36:06.337710+0000","last_active":"2024-09-15T22:38:25.009858+0000","last_peered":"2024-09-15T22:38:25.009858+0000","last_clean":"2024-09-15T22:38:25.009858+0000","last_became_active":"2024-09-15T22:36:06.337339+0000","last_became_peered":"2024-09-15T22:36:06.337339+0000","last_unstale":"2024-09-15T22:38:25.009858+0000","last_undegraded":"2024-09-15T22:38:25.009858+0000","last_fullsized":"2024-09-15T22:38:25.009858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:45.344595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241176+0000","last_change":"2024-09-15T22:36:15.289885+0000","last_active":"2024-09-15T22:38:25.241176+0000","last_peered":"2024-09-15T22:38:25.241176+0000","last_clean":"2024-09-15T22:38:25.241176+0000","last_became_active":"2024-09-15T22:36:15.289400+0000","last_became_peered":"2024-09-15T22:36:15.289400+0000","last_unstale":"2024-09-15T22:38:25.241176+0000","last_undegraded":"2024-09-15T22:38:25.241176+0000","last_fullsized":"2024-09-15T22:38:25.241176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:01.755107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.250669+0000","last_change":"2024-09-15T22:36:08.254795+0000","last_active":"2024-09-15T22:38:25.250669+0000","last_peered":"2024-09-15T22:38:25.250669+0000","last_clean":"2024-09-15T22:38:25.250669+0000","last_became_active":"2024-09-15T22:36:08.254492+0000","last_became_peered":"2024-09-15T22:36:08.254492+0000","last_unstale":"2024-09-15T22:38:25.250669+0000","last_undegraded":"2024-09-15T22:38:25.250669+0000","last_fullsized":"2024-09-15T22:38:25.250669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:55.576443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010725+0000","last_change":"2024-09-15T22:36:06.219862+0000","last_active":"2024-09-15T22:38:25.010725+0000","last_peered":"2024-09-15T22:38:25.010725+0000","last_clean":"2024-09-15T22:38:25.010725+0000","last_became_active":"2024-09-15T22:36:06.219424+0000","last_became_peered":"2024-09-15T22:36:06.219424+0000","last_unstale":"2024-09-15T22:38:25.010725+0000","last_undegraded":"2024-09-15T22:38:25.010725+0000","last_fullsized":"2024-09-15T22:38:25.010725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:10.097520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107560+0000","last_change":"2024-09-15T22:36:15.292523+0000","last_active":"2024-09-15T22:38:25.107560+0000","last_peered":"2024-09-15T22:38:25.107560+0000","last_clean":"2024-09-15T22:38:25.107560+0000","last_became_active":"2024-09-15T22:36:15.292153+0000","last_became_peered":"2024-09-15T22:36:15.292153+0000","last_unstale":"2024-09-15T22:38:25.107560+0000","last_undegraded":"2024-09-15T22:38:25.107560+0000","last_fullsized":"2024-09-15T22:38:25.107560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:50.079999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242031+0000","last_change":"2024-09-15T22:36:08.260347+0000","last_active":"2024-09-15T22:38:25.242031+0000","last_peered":"2024-09-15T22:38:25.242031+0000","last_clean":"2024-09-15T22:38:25.242031+0000","last_became_active":"2024-09-15T22:36:08.259882+0000","last_became_peered":"2024-09-15T22:36:08.259882+0000","last_unstale":"2024-09-15T22:38:25.242031+0000","last_undegraded":"2024-09-15T22:38:25.242031+0000","last_fullsized":"2024-09-15T22:38:25.242031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:49:50.869463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.548331+0000","last_change":"2024-09-15T22:36:06.219957+0000","last_active":"2024-09-15T22:38:34.548331+0000","last_peered":"2024-09-15T22:38:34.548331+0000","last_clean":"2024-09-15T22:38:34.548331+0000","last_became_active":"2024-09-15T22:36:06.218868+0000","last_became_peered":"2024-09-15T22:36:06.218868+0000","last_unstale":"2024-09-15T22:38:34.548331+0000","last_undegraded":"2024-09-15T22:38:34.548331+0000","last_fullsized":"2024-09-15T22:38:34.548331+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:49.219482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112480+0000","last_change":"2024-09-15T22:36:15.298114+0000","last_active":"2024-09-15T22:38:25.112480+0000","last_peered":"2024-09-15T22:38:25.112480+0000","last_clean":"2024-09-15T22:38:25.112480+0000","last_became_active":"2024-09-15T22:36:15.297408+0000","last_became_peered":"2024-09-15T22:36:15.297408+0000","last_unstale":"2024-09-15T22:38:25.112480+0000","last_undegraded":"2024-09-15T22:38:25.112480+0000","last_fullsized":"2024-09-15T22:38:25.112480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:32.791945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249732+0000","last_change":"2024-09-15T22:36:15.294847+0000","last_active":"2024-09-15T22:38:25.249732+0000","last_peered":"2024-09-15T22:38:25.249732+0000","last_clean":"2024-09-15T22:38:25.249732+0000","last_became_active":"2024-09-15T22:36:15.294491+0000","last_became_peered":"2024-09-15T22:36:15.294491+0000","last_unstale":"2024-09-15T22:38:25.249732+0000","last_undegraded":"2024-09-15T22:38:25.249732+0000","last_fullsized":"2024-09-15T22:38:25.249732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:59.205803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.107954+0000","last_change":"2024-09-15T22:36:08.253106+0000","last_active":"2024-09-15T22:38:25.107954+0000","last_peered":"2024-09-15T22:38:25.107954+0000","last_clean":"2024-09-15T22:38:25.107954+0000","last_became_active":"2024-09-15T22:36:08.252787+0000","last_became_peered":"2024-09-15T22:36:08.252787+0000","last_unstale":"2024-09-15T22:38:25.107954+0000","last_undegraded":"2024-09-15T22:38:25.107954+0000","last_fullsized":"2024-09-15T22:38:25.107954+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:02:33.537619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009527+0000","last_change":"2024-09-15T22:36:06.219839+0000","last_active":"2024-09-15T22:38:25.009527+0000","last_peered":"2024-09-15T22:38:25.009527+0000","last_clean":"2024-09-15T22:38:25.009527+0000","last_became_active":"2024-09-15T22:36:06.219498+0000","last_became_peered":"2024-09-15T22:36:06.219498+0000","last_unstale":"2024-09-15T22:38:25.009527+0000","last_undegraded":"2024-09-15T22:38:25.009527+0000","last_fullsized":"2024-09-15T22:38:25.009527+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:35.605198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242387+0000","last_change":"2024-09-15T22:36:15.298527+0000","last_active":"2024-09-15T22:38:25.242387+0000","last_peered":"2024-09-15T22:38:25.242387+0000","last_clean":"2024-09-15T22:38:25.242387+0000","last_became_active":"2024-09-15T22:36:15.298106+0000","last_became_peered":"2024-09-15T22:36:15.298106+0000","last_unstale":"2024-09-15T22:38:25.242387+0000","last_undegraded":"2024-09-15T22:38:25.242387+0000","last_fullsized":"2024-09-15T22:38:25.242387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:23.226509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112180+0000","last_change":"2024-09-15T22:36:08.251514+0000","last_active":"2024-09-15T22:38:25.112180+0000","last_peered":"2024-09-15T22:38:25.112180+0000","last_clean":"2024-09-15T22:38:25.112180+0000","last_became_active":"2024-09-15T22:36:08.251209+0000","last_became_peered":"2024-09-15T22:36:08.251209+0000","last_unstale":"2024-09-15T22:38:25.112180+0000","last_undegraded":"2024-09-15T22:38:25.112180+0000","last_fullsized":"2024-09-15T22:38:25.112180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:09:36.025903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006027+0000","last_change":"2024-09-15T22:36:06.223284+0000","last_active":"2024-09-15T22:38:25.006027+0000","last_peered":"2024-09-15T22:38:25.006027+0000","last_clean":"2024-09-15T22:38:25.006027+0000","last_became_active":"2024-09-15T22:36:06.222989+0000","last_became_peered":"2024-09-15T22:36:06.222989+0000","last_unstale":"2024-09-15T22:38:25.006027+0000","last_undegraded":"2024-09-15T22:38:25.006027+0000","last_fullsized":"2024-09-15T22:38:25.006027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:22.201899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124268+0000","last_change":"2024-09-15T22:36:06.220957+0000","last_active":"2024-09-15T22:38:25.124268+0000","last_peered":"2024-09-15T22:38:25.124268+0000","last_clean":"2024-09-15T22:38:25.124268+0000","last_became_active":"2024-09-15T22:36:06.220617+0000","last_became_peered":"2024-09-15T22:36:06.220617+0000","last_unstale":"2024-09-15T22:38:25.124268+0000","last_undegraded":"2024-09-15T22:38:25.124268+0000","last_fullsized":"2024-09-15T22:38:25.124268+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:07.432954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009565+0000","last_change":"2024-09-15T22:36:08.252583+0000","last_active":"2024-09-15T22:38:25.009565+0000","last_peered":"2024-09-15T22:38:25.009565+0000","last_clean":"2024-09-15T22:38:25.009565+0000","last_became_active":"2024-09-15T22:36:08.252282+0000","last_became_peered":"2024-09-15T22:36:08.252282+0000","last_unstale":"2024-09-15T22:38:25.009565+0000","last_undegraded":"2024-09-15T22:38:25.009565+0000","last_fullsized":"2024-09-15T22:38:25.009565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:08:13.766403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241674+0000","last_change":"2024-09-15T22:36:15.297873+0000","last_active":"2024-09-15T22:38:25.241674+0000","last_peered":"2024-09-15T22:38:25.241674+0000","last_clean":"2024-09-15T22:38:25.241674+0000","last_became_active":"2024-09-15T22:36:15.297440+0000","last_became_peered":"2024-09-15T22:36:15.297440+0000","last_unstale":"2024-09-15T22:38:25.241674+0000","last_undegraded":"2024-09-15T22:38:25.241674+0000","last_fullsized":"2024-09-15T22:38:25.241674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:09:15.257940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":162,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:34.549003+0000","last_change":"2024-09-15T22:36:06.221478+0000","last_active":"2024-09-15T22:38:34.549003+0000","last_peered":"2024-09-15T22:38:34.549003+0000","last_clean":"2024-09-15T22:38:34.549003+0000","last_became_active":"2024-09-15T22:36:06.220665+0000","last_became_peered":"2024-09-15T22:36:06.220665+0000","last_unstale":"2024-09-15T22:38:34.549003+0000","last_undegraded":"2024-09-15T22:38:34.549003+0000","last_fullsized":"2024-09-15T22:38:34.549003+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:05.191804+0000","last_clean_scrub_stamp":"2024-09-15T22:36:05.191804+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:41.248325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112093+0000","last_change":"2024-09-15T22:36:08.258085+0000","last_active":"2024-09-15T22:38:25.112093+0000","last_peered":"2024-09-15T22:38:25.112093+0000","last_clean":"2024-09-15T22:38:25.112093+0000","last_became_active":"2024-09-15T22:36:08.257747+0000","last_became_peered":"2024-09-15T22:36:08.257747+0000","last_unstale":"2024-09-15T22:38:25.112093+0000","last_undegraded":"2024-09-15T22:38:25.112093+0000","last_fullsized":"2024-09-15T22:38:25.112093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:45:55.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241736+0000","last_change":"2024-09-15T22:36:15.299056+0000","last_active":"2024-09-15T22:38:25.241736+0000","last_peered":"2024-09-15T22:38:25.241736+0000","last_clean":"2024-09-15T22:38:25.241736+0000","last_became_active":"2024-09-15T22:36:15.298665+0000","last_became_peered":"2024-09-15T22:36:15.298665+0000","last_unstale":"2024-09-15T22:38:25.241736+0000","last_undegraded":"2024-09-15T22:38:25.241736+0000","last_fullsized":"2024-09-15T22:38:25.241736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:14.266021+0000","last_clean_scrub_stamp":"2024-09-15T22:36:14.266021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:55.589201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241163+0000","last_change":"2024-09-15T22:36:08.256140+0000","last_active":"2024-09-15T22:38:25.241163+0000","last_peered":"2024-09-15T22:38:25.241163+0000","last_clean":"2024-09-15T22:38:25.241163+0000","last_became_active":"2024-09-15T22:36:08.255788+0000","last_became_peered":"2024-09-15T22:36:08.255788+0000","last_unstale":"2024-09-15T22:38:25.241163+0000","last_undegraded":"2024-09-15T22:38:25.241163+0000","last_fullsized":"2024-09-15T22:38:25.241163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:00.390090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241675+0000","last_change":"2024-09-15T22:36:08.260415+0000","last_active":"2024-09-15T22:38:25.241675+0000","last_peered":"2024-09-15T22:38:25.241675+0000","last_clean":"2024-09-15T22:38:25.241675+0000","last_became_active":"2024-09-15T22:36:08.259665+0000","last_became_peered":"2024-09-15T22:36:08.259665+0000","last_unstale":"2024-09-15T22:38:25.241675+0000","last_undegraded":"2024-09-15T22:38:25.241675+0000","last_fullsized":"2024-09-15T22:38:25.241675+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:54.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241517+0000","last_change":"2024-09-15T22:36:08.256741+0000","last_active":"2024-09-15T22:38:25.241517+0000","last_peered":"2024-09-15T22:38:25.241517+0000","last_clean":"2024-09-15T22:38:25.241517+0000","last_became_active":"2024-09-15T22:36:08.256439+0000","last_became_peered":"2024-09-15T22:36:08.256439+0000","last_unstale":"2024-09-15T22:38:25.241517+0000","last_undegraded":"2024-09-15T22:38:25.241517+0000","last_fullsized":"2024-09-15T22:38:25.241517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:32.044189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249023+0000","last_change":"2024-09-15T22:37:12.522337+0000","last_active":"2024-09-15T22:38:25.249023+0000","last_peered":"2024-09-15T22:38:25.249023+0000","last_clean":"2024-09-15T22:38:25.249023+0000","last_became_active":"2024-09-15T22:37:12.521990+0000","last_became_peered":"2024-09-15T22:37:12.521990+0000","last_unstale":"2024-09-15T22:38:25.249023+0000","last_undegraded":"2024-09-15T22:38:25.249023+0000","last_fullsized":"2024-09-15T22:38:25.249023+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:28.243284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006558+0000","last_change":"2024-09-15T22:36:08.241460+0000","last_active":"2024-09-15T22:38:25.006558+0000","last_peered":"2024-09-15T22:38:25.006558+0000","last_clean":"2024-09-15T22:38:25.006558+0000","last_became_active":"2024-09-15T22:36:08.241088+0000","last_became_peered":"2024-09-15T22:36:08.241088+0000","last_unstale":"2024-09-15T22:38:25.006558+0000","last_undegraded":"2024-09-15T22:38:25.006558+0000","last_fullsized":"2024-09-15T22:38:25.006558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:03.789198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242143+0000","last_change":"2024-09-15T22:36:08.256372+0000","last_active":"2024-09-15T22:38:25.242143+0000","last_peered":"2024-09-15T22:38:25.242143+0000","last_clean":"2024-09-15T22:38:25.242143+0000","last_became_active":"2024-09-15T22:36:08.255630+0000","last_became_peered":"2024-09-15T22:36:08.255630+0000","last_unstale":"2024-09-15T22:38:25.242143+0000","last_undegraded":"2024-09-15T22:38:25.242143+0000","last_fullsized":"2024-09-15T22:38:25.242143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:00:28.601462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241544+0000","last_change":"2024-09-15T22:36:08.255012+0000","last_active":"2024-09-15T22:38:25.241544+0000","last_peered":"2024-09-15T22:38:25.241544+0000","last_clean":"2024-09-15T22:38:25.241544+0000","last_became_active":"2024-09-15T22:36:08.254694+0000","last_became_peered":"2024-09-15T22:36:08.254694+0000","last_unstale":"2024-09-15T22:38:25.241544+0000","last_undegraded":"2024-09-15T22:38:25.241544+0000","last_fullsized":"2024-09-15T22:38:25.241544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:32.217887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.115695+0000","last_change":"2024-09-15T22:36:08.259092+0000","last_active":"2024-09-15T22:38:25.115695+0000","last_peered":"2024-09-15T22:38:25.115695+0000","last_clean":"2024-09-15T22:38:25.115695+0000","last_became_active":"2024-09-15T22:36:08.258785+0000","last_became_peered":"2024-09-15T22:36:08.258785+0000","last_unstale":"2024-09-15T22:38:25.115695+0000","last_undegraded":"2024-09-15T22:38:25.115695+0000","last_fullsized":"2024-09-15T22:38:25.115695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:25.631099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.009267+0000","last_change":"2024-09-15T22:36:08.248711+0000","last_active":"2024-09-15T22:38:25.009267+0000","last_peered":"2024-09-15T22:38:25.009267+0000","last_clean":"2024-09-15T22:38:25.009267+0000","last_became_active":"2024-09-15T22:36:08.248415+0000","last_became_peered":"2024-09-15T22:36:08.248415+0000","last_unstale":"2024-09-15T22:38:25.009267+0000","last_undegraded":"2024-09-15T22:38:25.009267+0000","last_fullsized":"2024-09-15T22:38:25.009267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:33:22.365963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112660+0000","last_change":"2024-09-15T22:36:08.257774+0000","last_active":"2024-09-15T22:38:25.112660+0000","last_peered":"2024-09-15T22:38:25.112660+0000","last_clean":"2024-09-15T22:38:25.112660+0000","last_became_active":"2024-09-15T22:36:08.256416+0000","last_became_peered":"2024-09-15T22:36:08.256416+0000","last_unstale":"2024-09-15T22:38:25.112660+0000","last_undegraded":"2024-09-15T22:38:25.112660+0000","last_fullsized":"2024-09-15T22:38:25.112660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:24.729275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241756+0000","last_change":"2024-09-15T22:36:08.257215+0000","last_active":"2024-09-15T22:38:25.241756+0000","last_peered":"2024-09-15T22:38:25.241756+0000","last_clean":"2024-09-15T22:38:25.241756+0000","last_became_active":"2024-09-15T22:36:08.256825+0000","last_became_peered":"2024-09-15T22:36:08.256825+0000","last_unstale":"2024-09-15T22:38:25.241756+0000","last_undegraded":"2024-09-15T22:38:25.241756+0000","last_fullsized":"2024-09-15T22:38:25.241756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:12.749572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006219+0000","last_change":"2024-09-15T22:36:08.247622+0000","last_active":"2024-09-15T22:38:25.006219+0000","last_peered":"2024-09-15T22:38:25.006219+0000","last_clean":"2024-09-15T22:38:25.006219+0000","last_became_active":"2024-09-15T22:36:08.246913+0000","last_became_peered":"2024-09-15T22:36:08.246913+0000","last_unstale":"2024-09-15T22:38:25.006219+0000","last_undegraded":"2024-09-15T22:38:25.006219+0000","last_fullsized":"2024-09-15T22:38:25.006219+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:54.050367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242055+0000","last_change":"2024-09-15T22:37:12.522651+0000","last_active":"2024-09-15T22:38:25.242055+0000","last_peered":"2024-09-15T22:38:25.242055+0000","last_clean":"2024-09-15T22:38:25.242055+0000","last_became_active":"2024-09-15T22:37:12.522126+0000","last_became_peered":"2024-09-15T22:37:12.522126+0000","last_unstale":"2024-09-15T22:38:25.242055+0000","last_undegraded":"2024-09-15T22:38:25.242055+0000","last_fullsized":"2024-09-15T22:38:25.242055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:41.560132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241921+0000","last_change":"2024-09-15T22:36:08.254663+0000","last_active":"2024-09-15T22:38:25.241921+0000","last_peered":"2024-09-15T22:38:25.241921+0000","last_clean":"2024-09-15T22:38:25.241921+0000","last_became_active":"2024-09-15T22:36:08.254360+0000","last_became_peered":"2024-09-15T22:36:08.254360+0000","last_unstale":"2024-09-15T22:38:25.241921+0000","last_undegraded":"2024-09-15T22:38:25.241921+0000","last_fullsized":"2024-09-15T22:38:25.241921+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:31:04.447724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.109165+0000","last_change":"2024-09-15T22:36:08.254565+0000","last_active":"2024-09-15T22:38:25.109165+0000","last_peered":"2024-09-15T22:38:25.109165+0000","last_clean":"2024-09-15T22:38:25.109165+0000","last_became_active":"2024-09-15T22:36:08.254237+0000","last_became_peered":"2024-09-15T22:36:08.254237+0000","last_unstale":"2024-09-15T22:38:25.109165+0000","last_undegraded":"2024-09-15T22:38:25.109165+0000","last_fullsized":"2024-09-15T22:38:25.109165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:16.369934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242550+0000","last_change":"2024-09-15T22:36:08.248688+0000","last_active":"2024-09-15T22:38:25.242550+0000","last_peered":"2024-09-15T22:38:25.242550+0000","last_clean":"2024-09-15T22:38:25.242550+0000","last_became_active":"2024-09-15T22:36:08.248396+0000","last_became_peered":"2024-09-15T22:36:08.248396+0000","last_unstale":"2024-09-15T22:38:25.242550+0000","last_undegraded":"2024-09-15T22:38:25.242550+0000","last_fullsized":"2024-09-15T22:38:25.242550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:29.950468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241084+0000","last_change":"2024-09-15T22:36:08.251403+0000","last_active":"2024-09-15T22:38:25.241084+0000","last_peered":"2024-09-15T22:38:25.241084+0000","last_clean":"2024-09-15T22:38:25.241084+0000","last_became_active":"2024-09-15T22:36:08.250670+0000","last_became_peered":"2024-09-15T22:36:08.250670+0000","last_unstale":"2024-09-15T22:38:25.241084+0000","last_undegraded":"2024-09-15T22:38:25.241084+0000","last_fullsized":"2024-09-15T22:38:25.241084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:04.469951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248330+0000","last_change":"2024-09-15T22:36:08.257257+0000","last_active":"2024-09-15T22:38:25.248330+0000","last_peered":"2024-09-15T22:38:25.248330+0000","last_clean":"2024-09-15T22:38:25.248330+0000","last_became_active":"2024-09-15T22:36:08.256963+0000","last_became_peered":"2024-09-15T22:36:08.256963+0000","last_unstale":"2024-09-15T22:38:25.248330+0000","last_undegraded":"2024-09-15T22:38:25.248330+0000","last_fullsized":"2024-09-15T22:38:25.248330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:31:40.460310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.248630+0000","last_change":"2024-09-15T22:36:08.251916+0000","last_active":"2024-09-15T22:38:25.248630+0000","last_peered":"2024-09-15T22:38:25.248630+0000","last_clean":"2024-09-15T22:38:25.248630+0000","last_became_active":"2024-09-15T22:36:08.251516+0000","last_became_peered":"2024-09-15T22:36:08.251516+0000","last_unstale":"2024-09-15T22:38:25.248630+0000","last_undegraded":"2024-09-15T22:38:25.248630+0000","last_fullsized":"2024-09-15T22:38:25.248630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:29.710754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.006222+0000","last_change":"2024-09-15T22:36:08.248408+0000","last_active":"2024-09-15T22:38:25.006222+0000","last_peered":"2024-09-15T22:38:25.006222+0000","last_clean":"2024-09-15T22:38:25.006222+0000","last_became_active":"2024-09-15T22:36:08.248097+0000","last_became_peered":"2024-09-15T22:36:08.248097+0000","last_unstale":"2024-09-15T22:38:25.006222+0000","last_undegraded":"2024-09-15T22:38:25.006222+0000","last_fullsized":"2024-09-15T22:38:25.006222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:11.795514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.241501+0000","last_change":"2024-09-15T22:36:08.252210+0000","last_active":"2024-09-15T22:38:25.241501+0000","last_peered":"2024-09-15T22:38:25.241501+0000","last_clean":"2024-09-15T22:38:25.241501+0000","last_became_active":"2024-09-15T22:36:08.251719+0000","last_became_peered":"2024-09-15T22:36:08.251719+0000","last_unstale":"2024-09-15T22:38:25.241501+0000","last_undegraded":"2024-09-15T22:38:25.241501+0000","last_fullsized":"2024-09-15T22:38:25.241501+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:26.968977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010748+0000","last_change":"2024-09-15T22:36:08.254685+0000","last_active":"2024-09-15T22:38:25.010748+0000","last_peered":"2024-09-15T22:38:25.010748+0000","last_clean":"2024-09-15T22:38:25.010748+0000","last_became_active":"2024-09-15T22:36:08.254250+0000","last_became_peered":"2024-09-15T22:36:08.254250+0000","last_unstale":"2024-09-15T22:38:25.010748+0000","last_undegraded":"2024-09-15T22:38:25.010748+0000","last_fullsized":"2024-09-15T22:38:25.010748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:23.934549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124521+0000","last_change":"2024-09-15T22:36:08.258908+0000","last_active":"2024-09-15T22:38:25.124521+0000","last_peered":"2024-09-15T22:38:25.124521+0000","last_clean":"2024-09-15T22:38:25.124521+0000","last_became_active":"2024-09-15T22:36:08.258386+0000","last_became_peered":"2024-09-15T22:36:08.258386+0000","last_unstale":"2024-09-15T22:38:25.124521+0000","last_undegraded":"2024-09-15T22:38:25.124521+0000","last_fullsized":"2024-09-15T22:38:25.124521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:22.164074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242023+0000","last_change":"2024-09-15T22:36:08.253839+0000","last_active":"2024-09-15T22:38:25.242023+0000","last_peered":"2024-09-15T22:38:25.242023+0000","last_clean":"2024-09-15T22:38:25.242023+0000","last_became_active":"2024-09-15T22:36:08.253448+0000","last_became_peered":"2024-09-15T22:36:08.253448+0000","last_unstale":"2024-09-15T22:38:25.242023+0000","last_undegraded":"2024-09-15T22:38:25.242023+0000","last_fullsized":"2024-09-15T22:38:25.242023+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:56:45.138409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112144+0000","last_change":"2024-09-15T22:36:08.258385+0000","last_active":"2024-09-15T22:38:25.112144+0000","last_peered":"2024-09-15T22:38:25.112144+0000","last_clean":"2024-09-15T22:38:25.112144+0000","last_became_active":"2024-09-15T22:36:08.258095+0000","last_became_peered":"2024-09-15T22:36:08.258095+0000","last_unstale":"2024-09-15T22:38:25.112144+0000","last_undegraded":"2024-09-15T22:38:25.112144+0000","last_fullsized":"2024-09-15T22:38:25.112144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:03:02.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.010114+0000","last_change":"2024-09-15T22:36:08.246711+0000","last_active":"2024-09-15T22:38:25.010114+0000","last_peered":"2024-09-15T22:38:25.010114+0000","last_clean":"2024-09-15T22:38:25.010114+0000","last_became_active":"2024-09-15T22:36:08.246340+0000","last_became_peered":"2024-09-15T22:36:08.246340+0000","last_unstale":"2024-09-15T22:38:25.010114+0000","last_undegraded":"2024-09-15T22:38:25.010114+0000","last_fullsized":"2024-09-15T22:38:25.010114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:54.386049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.124205+0000","last_change":"2024-09-15T22:36:08.255139+0000","last_active":"2024-09-15T22:38:25.124205+0000","last_peered":"2024-09-15T22:38:25.124205+0000","last_clean":"2024-09-15T22:38:25.124205+0000","last_became_active":"2024-09-15T22:36:08.254716+0000","last_became_peered":"2024-09-15T22:36:08.254716+0000","last_unstale":"2024-09-15T22:38:25.124205+0000","last_undegraded":"2024-09-15T22:38:25.124205+0000","last_fullsized":"2024-09-15T22:38:25.124205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:59.718955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.112085+0000","last_change":"2024-09-15T22:36:08.261107+0000","last_active":"2024-09-15T22:38:25.112085+0000","last_peered":"2024-09-15T22:38:25.112085+0000","last_clean":"2024-09-15T22:38:25.112085+0000","last_became_active":"2024-09-15T22:36:08.260755+0000","last_became_peered":"2024-09-15T22:36:08.260755+0000","last_unstale":"2024-09-15T22:38:25.112085+0000","last_undegraded":"2024-09-15T22:38:25.112085+0000","last_fullsized":"2024-09-15T22:38:25.112085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:18.455407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.108245+0000","last_change":"2024-09-15T22:37:12.524074+0000","last_active":"2024-09-15T22:38:25.108245+0000","last_peered":"2024-09-15T22:38:25.108245+0000","last_clean":"2024-09-15T22:38:25.108245+0000","last_became_active":"2024-09-15T22:37:12.523702+0000","last_became_peered":"2024-09-15T22:37:12.523702+0000","last_unstale":"2024-09-15T22:38:25.108245+0000","last_undegraded":"2024-09-15T22:38:25.108245+0000","last_fullsized":"2024-09-15T22:38:25.108245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:00.575032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249437+0000","last_change":"2024-09-15T22:36:08.253709+0000","last_active":"2024-09-15T22:38:25.249437+0000","last_peered":"2024-09-15T22:38:25.249437+0000","last_clean":"2024-09-15T22:38:25.249437+0000","last_became_active":"2024-09-15T22:36:08.253428+0000","last_became_peered":"2024-09-15T22:36:08.253428+0000","last_unstale":"2024-09-15T22:38:25.249437+0000","last_undegraded":"2024-09-15T22:38:25.249437+0000","last_fullsized":"2024-09-15T22:38:25.249437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:57:35.293096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.249830+0000","last_change":"2024-09-15T22:36:08.253359+0000","last_active":"2024-09-15T22:38:25.249830+0000","last_peered":"2024-09-15T22:38:25.249830+0000","last_clean":"2024-09-15T22:38:25.249830+0000","last_became_active":"2024-09-15T22:36:08.251678+0000","last_became_peered":"2024-09-15T22:36:08.251678+0000","last_unstale":"2024-09-15T22:38:25.249830+0000","last_undegraded":"2024-09-15T22:38:25.249830+0000","last_fullsized":"2024-09-15T22:38:25.249830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:50:21.848954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"87'8","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T22:38:25.242437+0000","last_change":"2024-09-15T22:36:08.248405+0000","last_active":"2024-09-15T22:38:25.242437+0000","last_peered":"2024-09-15T22:38:25.242437+0000","last_clean":"2024-09-15T22:38:25.242437+0000","last_became_active":"2024-09-15T22:36:08.247982+0000","last_became_peered":"2024-09-15T22:36:08.247982+0000","last_unstale":"2024-09-15T22:38:25.242437+0000","last_undegraded":"2024-09-15T22:38:25.242437+0000","last_fullsized":"2024-09-15T22:38:25.242437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:36:07.225521+0000","last_clean_scrub_stamp":"2024-09-15T22:36:07.225521+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:57:10.059686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":796407,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1045,"num_read_kb":738,"num_write":424,"num_write_kb":2358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2389221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612650,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1040384,"data_stored":519982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841583,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29164,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93715988,"statfs":{"total":95995035648,"available":95965171712,"internally_reserved":0,"allocated":2068480,"data_stored":1506297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005109,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1454080,"data_stored":917530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1449984,"data_stored":913354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462975,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1650688,"data_stored":1117284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626500,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1036288,"data_stored":515394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855433,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29088,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716064,"statfs":{"total":95995035648,"available":95965249536,"internally_reserved":0,"allocated":1990656,"data_stored":1440010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1056768,"data_stored":521250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215188,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37196,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707956,"statfs":{"total":95995035648,"available":95956946944,"internally_reserved":0,"allocated":1773568,"data_stored":1237036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1126400,"data_stored":586797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1110016,"data_stored":585025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":516782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":71493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":400625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:38:40.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:40.271 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:40.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:40 smithi016 ceph-mon[31492]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-15T22:38:40.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:40 smithi016 ceph-mon[31492]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:40.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:40 smithi059 ceph-mon[34872]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-15T22:38:40.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:40 smithi059 ceph-mon[34872]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:40.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:40 smithi130 ceph-mon[35742]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-09-15T22:38:40.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:40 smithi130 ceph-mon[35742]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:38:40.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 -- 172.21.15.16:0/3000174217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe038102da0 msgr2=0x7fe038103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3000174217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe038102da0 0x7fe038103180 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe028009900 tx=0x7fe02802a480 comp rx=0 tx=0).stop 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 -- 172.21.15.16:0/3000174217 shutdown_connections 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3000174217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe03810c180 0x7fe03810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3000174217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe0381036c0 0x7fe03810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3000174217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe038102da0 0x7fe038103180 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 -- 172.21.15.16:0/3000174217 >> 172.21.15.16:0/3000174217 conn(0x7fe0380fc7a0 msgr2=0x7fe0380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 -- 172.21.15.16:0/3000174217 shutdown_connections 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.772+0000 7fe03d56f640 1 -- 172.21.15.16:0/3000174217 wait complete. 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 Processor -- start 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 -- start start 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe038102da0 0x7fe0381983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:40.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0381036c0 0x7fe038198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe03810c180 0x7fe03819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe038111f60 con 0x7fe0381036c0 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe038111de0 con 0x7fe03810c180 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe03d56f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe0381120e0 con 0x7fe038102da0 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe0367fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0381036c0 0x7fe038198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe0367fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0381036c0 0x7fe038198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54426/0 (socket says 172.21.15.16:54426) 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe0367fc640 1 -- 172.21.15.16:0/3314505079 learned_addr learned my addr 172.21.15.16:0/3314505079 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe0377fe640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe03810c180 0x7fe03819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.773+0000 7fe036ffd640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe038102da0 0x7fe0381983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe0367fc640 1 -- 172.21.15.16:0/3314505079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe038102da0 msgr2=0x7fe0381983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe0367fc640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe038102da0 0x7fe0381983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe0367fc640 1 -- 172.21.15.16:0/3314505079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe03810c180 msgr2=0x7fe03819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:40.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe0367fc640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe03810c180 0x7fe03819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe0367fc640 1 -- 172.21.15.16:0/3314505079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03819d370 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe0367fc640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0381036c0 0x7fe038198900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00d770 tx=0x7fe02c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe01ffff640 1 -- 172.21.15.16:0/3314505079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c017070 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03819d660 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe01ffff640 1 -- 172.21.15.16:0/3314505079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02c005500 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe01ffff640 1 -- 172.21.15.16:0/3314505079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c004e00 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.774+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0381a4f40 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.775+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe038103180 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.776+0000 7fe01ffff640 1 -- 172.21.15.16:0/3314505079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7fe02c004fa0 con 0x7fe0381036c0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.777+0000 7fe01ffff640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe0200a3440 0x7fe0200a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.777+0000 7fe036ffd640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe0200a3440 0x7fe0200a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:40.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.777+0000 7fe01ffff640 1 -- 172.21.15.16:0/3314505079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe02c0d0130 con 0x7fe0381036c0 2024-09-15T22:38:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.778+0000 7fe036ffd640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe0200a3440 0x7fe0200a5900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe028005ab0 tx=0x7fe028005a20 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:40.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.780+0000 7fe01ffff640 1 -- 172.21.15.16:0/3314505079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe02c09c5e0 con 0x7fe0381036c0 2024-09-15T22:38:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.994+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe038199720 con 0x7fe0381036c0 2024-09-15T22:38:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.994+0000 7fe01ffff640 1 -- 172.21.15.16:0/3314505079 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fe038199720 con 0x7fe0381036c0 2024-09-15T22:38:40.996 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:40.996 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:38:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:38:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe0200a3440 msgr2=0x7fe0200a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe0200a3440 0x7fe0200a5900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe028005ab0 tx=0x7fe028005a20 comp rx=0 tx=0).stop 2024-09-15T22:38:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0381036c0 msgr2=0x7fe038198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0381036c0 0x7fe038198900 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00d770 tx=0x7fe02c00dc30 comp rx=0 tx=0).stop 2024-09-15T22:38:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 shutdown_connections 2024-09-15T22:38:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe03810c180 0x7fe03819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0381036c0 0x7fe038198900 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe0200a3440 0x7fe0200a5900 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 --2- 172.21.15.16:0/3314505079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe038102da0 0x7fe0381983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 >> 172.21.15.16:0/3314505079 conn(0x7fe0380fc7a0 msgr2=0x7fe0380fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 shutdown_connections 2024-09-15T22:38:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:40.997+0000 7fe03d56f640 1 -- 172.21.15.16:0/3314505079 wait complete. 2024-09-15T22:38:41.681 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:41 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3314505079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:41.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:41 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3314505079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:41.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:41.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:41 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3314505079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 -- 172.21.15.16:0/462404598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 msgr2=0x7fc66c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 --2- 172.21.15.16:0/462404598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c1029b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc660009900 tx=0x7fc66002a480 comp rx=0 tx=0).stop 2024-09-15T22:38:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 -- 172.21.15.16:0/462404598 shutdown_connections 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 --2- 172.21.15.16:0/462404598 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc66c102ef0 0x7fc66c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 --2- 172.21.15.16:0/462404598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c1029b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 --2- 172.21.15.16:0/462404598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc66c100e20 0x7fc66c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 -- 172.21.15.16:0/462404598 >> 172.21.15.16:0/462404598 conn(0x7fc66c0fc7d0 msgr2=0x7fc66c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 -- 172.21.15.16:0/462404598 shutdown_connections 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 -- 172.21.15.16:0/462404598 wait complete. 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.498+0000 7fc674707640 1 Processor -- start 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc674707640 1 -- start start 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc674707640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc66c100e20 0x7fc66c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc674707640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc674707640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc66c102ef0 0x7fc66c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:42.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc674707640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c1120a0 con 0x7fc66c102570 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc674707640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c111f20 con 0x7fc66c100e20 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc674707640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c112220 con 0x7fc66c102ef0 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47352/0 (socket says 172.21.15.16:47352) 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 -- 172.21.15.16:0/1455522133 learned_addr learned my addr 172.21.15.16:0/1455522133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc672ca4640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc66c102ef0 0x7fc66c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc6724a3640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc66c100e20 0x7fc66c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 -- 172.21.15.16:0/1455522133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc66c102ef0 msgr2=0x7fc66c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc66c102ef0 0x7fc66c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 -- 172.21.15.16:0/1455522133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc66c100e20 msgr2=0x7fc66c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc66c100e20 0x7fc66c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 -- 172.21.15.16:0/1455522133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c19d590 con 0x7fc66c102570 2024-09-15T22:38:42.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc6724a3640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc66c100e20 0x7fc66c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.499+0000 7fc671ca2640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c198b20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc660006010 tx=0x7fc660003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.500+0000 7fc65f7fe640 1 -- 172.21.15.16:0/1455522133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66003f070 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.500+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c19d820 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.500+0000 7fc65f7fe640 1 -- 172.21.15.16:0/1455522133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc660031c30 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.500+0000 7fc65f7fe640 1 -- 172.21.15.16:0/1455522133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660035030 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.500+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c1a5160 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.501+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c101200 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.502+0000 7fc65f7fe640 1 -- 172.21.15.16:0/1455522133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7fc660042070 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.502+0000 7fc65f7fe640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6580a3440 0x7fc6580a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.503+0000 7fc6724a3640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6580a3440 0x7fc6580a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.503+0000 7fc65f7fe640 1 -- 172.21.15.16:0/1455522133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fc6600f0f90 con 0x7fc66c102570 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.503+0000 7fc6724a3640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6580a3440 0x7fc6580a5900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc6680058c0 tx=0x7fc668005d30 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.506+0000 7fc65f7fe640 1 -- 172.21.15.16:0/1455522133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc660031de0 con 0x7fc66c102570 2024-09-15T22:38:42.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:42 smithi059 ceph-mon[34872]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-09-15T22:38:42.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.715+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc66c109090 con 0x7fc66c102570 2024-09-15T22:38:42.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:42 smithi016 ceph-mon[31492]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-09-15T22:38:42.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.716+0000 7fc65f7fe640 1 -- 172.21.15.16:0/1455522133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fc6600bd560 con 0x7fc66c102570 2024-09-15T22:38:42.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:42.718 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:38:42.718 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:38:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.718+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6580a3440 msgr2=0x7fc6580a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.718+0000 7fc674707640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6580a3440 0x7fc6580a5900 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc6680058c0 tx=0x7fc668005d30 comp rx=0 tx=0).stop 2024-09-15T22:38:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.718+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 msgr2=0x7fc66c198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.718+0000 7fc674707640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c198b20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc660006010 tx=0x7fc660003c10 comp rx=0 tx=0).stop 2024-09-15T22:38:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 shutdown_connections 2024-09-15T22:38:42.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6580a3440 0x7fc6580a5900 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc66c102ef0 0x7fc66c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc66c102570 0x7fc66c198b20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 --2- 172.21.15.16:0/1455522133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc66c100e20 0x7fc66c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:42.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 >> 172.21.15.16:0/1455522133 conn(0x7fc66c0fc7d0 msgr2=0x7fc66c106940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:42.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 shutdown_connections 2024-09-15T22:38:42.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:42.719+0000 7fc674707640 1 -- 172.21.15.16:0/1455522133 wait complete. 2024-09-15T22:38:42.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:42 smithi130 ceph-mon[35742]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-09-15T22:38:43.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:38:43.617 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:43.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:43 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1455522133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:43.738 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:43 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1455522133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:43.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:43 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1455522133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/4094184900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f181008d0 msgr2=0x7f1f18100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/4094184900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f181008d0 0x7f1f18100cb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009900 tx=0x7f1f0c02a410 comp rx=0 tx=0).stop 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/4094184900 shutdown_connections 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/4094184900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f1810e480 0x7f1f18110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/4094184900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f18101280 0x7f1f1810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/4094184900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f181008d0 0x7f1f18100cb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/4094184900 >> 172.21.15.16:0/4094184900 conn(0x7f1f180fc7a0 msgr2=0x7f1f180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/4094184900 shutdown_connections 2024-09-15T22:38:44.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.099+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/4094184900 wait complete. 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 Processor -- start 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 -- start start 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f181008d0 0x7f1f1819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f18101280 0x7f1f1819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f1810e480 0x7f1f1819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f181141b0 con 0x7f1f18101280 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18114030 con 0x7f1f181008d0 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1fa7b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1f18114330 con 0x7f1f1810e480 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1d016640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f18101280 0x7f1f1819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1d016640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f18101280 0x7f1f1819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47370/0 (socket says 172.21.15.16:47370) 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1d016640 1 -- 172.21.15.16:0/503394757 learned_addr learned my addr 172.21.15.16:0/503394757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1d817640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f181008d0 0x7f1f1819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:44.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.100+0000 7f1f1e018640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f1810e480 0x7f1f1819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1d016640 1 -- 172.21.15.16:0/503394757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f1810e480 msgr2=0x7f1f1819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1d016640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f1810e480 0x7f1f1819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1d016640 1 -- 172.21.15.16:0/503394757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f181008d0 msgr2=0x7f1f1819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1d016640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f181008d0 0x7f1f1819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1d016640 1 -- 172.21.15.16:0/503394757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1819f640 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1d016640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f18101280 0x7f1f1819abd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400e9f0 tx=0x7f1f1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f0affd640 1 -- 172.21.15.16:0/503394757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1400cd80 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1819f930 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f0affd640 1 -- 172.21.15.16:0/503394757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f14010910 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f0affd640 1 -- 172.21.15.16:0/503394757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f140188f0 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.101+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181a7210 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.102+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f18100cb0 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.104+0000 7f1f0affd640 1 -- 172.21.15.16:0/503394757 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f1f14010ac0 con 0x7f1f18101280 2024-09-15T22:38:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.104+0000 7f1f0affd640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1f040a3440 0x7f1f040a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.104+0000 7f1f1d817640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1f040a3440 0x7f1f040a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.104+0000 7f1f0affd640 1 -- 172.21.15.16:0/503394757 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1f140d0a10 con 0x7f1f18101280 2024-09-15T22:38:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.105+0000 7f1f1d817640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1f040a3440 0x7f1f040a5900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009230 tx=0x7f1f0c0058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:44.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.107+0000 7f1f0affd640 1 -- 172.21.15.16:0/503394757 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f1409c870 con 0x7f1f18101280 2024-09-15T22:38:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.287+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f1819b9f0 con 0x7f1f18101280 2024-09-15T22:38:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.288+0000 7f1f0affd640 1 -- 172.21.15.16:0/503394757 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25138 (secure 0 0 0) 0x7f1f1819b9f0 con 0x7f1f18101280 2024-09-15T22:38:44.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:44.290 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":90,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:38:25.000286+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3275020767":"2024-09-16T22:38:25.000235+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3377264779":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3984482177":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/1746300554":"2024-09-16T22:38:25.000235+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1f040a3440 msgr2=0x7f1f040a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1f040a3440 0x7f1f040a5900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009230 tx=0x7f1f0c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f18101280 msgr2=0x7f1f1819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f18101280 0x7f1f1819abd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400e9f0 tx=0x7f1f1400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 shutdown_connections 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1f040a3440 0x7f1f040a5900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1f1810e480 0x7f1f1819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f18101280 0x7f1f1819abd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 --2- 172.21.15.16:0/503394757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f181008d0 0x7f1f1819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 >> 172.21.15.16:0/503394757 conn(0x7f1f180fc7a0 msgr2=0x7f1f180feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 shutdown_connections 2024-09-15T22:38:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:44.291+0000 7f1f1fa7b640 1 -- 172.21.15.16:0/503394757 wait complete. 2024-09-15T22:38:44.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:44 smithi016 ceph-mon[31492]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 48 op/s 2024-09-15T22:38:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:44 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:38:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:44 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/503394757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:44.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:44 smithi059 ceph-mon[34872]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 48 op/s 2024-09-15T22:38:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:44 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:38:44.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:44 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/503394757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:44.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:44 smithi130 ceph-mon[35742]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 48 op/s 2024-09-15T22:38:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:44 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:38:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:44 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/503394757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:38:44.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-15T22:38:45.260 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:46.448 INFO:teuthology.orchestra.run.smithi016.stdout:{"run_dir":"/var/run/ceph"} 2024-09-15T22:38:46.449 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-15T22:38:46.449 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-15T22:38:46.449 DEBUG:tasks.quiescer:Yielding 2024-09-15T22:38:46.449 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T22:38:46.464 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T22:38:46.464 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T22:38:46.464 INFO:tasks.workunit:timeout=3h 2024-09-15T22:38:46.464 INFO:tasks.workunit:cleanup=True 2024-09-15T22:38:46.465 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:46.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:38:46.482 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-15 22:36:54.607483970 +0000 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-15 22:36:54.607483970 +0000 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-15 22:38:22.959356857 +0000 2024-09-15T22:38:46.483 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-15 22:36:54.607483970 +0000 2024-09-15T22:38:46.483 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-15T22:38:46.483 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-15T22:38:46.558 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T22:38:46.615 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T22:38:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:46 smithi059 ceph-mon[34872]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-15T22:38:46.725 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:46 smithi130 ceph-mon[35742]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-15T22:38:46.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:46 smithi016 ceph-mon[31492]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 -- 172.21.15.16:0/1119310639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541036e0 msgr2=0x7f7c54107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 --2- 172.21.15.16:0/1119310639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541036e0 0x7f7c54107b90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009930 tx=0x7f7c4002a420 comp rx=0 tx=0).stop 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 -- 172.21.15.16:0/1119310639 shutdown_connections 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 --2- 172.21.15.16:0/1119310639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c541082a0 0x7f7c5410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 --2- 172.21.15.16:0/1119310639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541036e0 0x7f7c54107b90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 --2- 172.21.15.16:0/1119310639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c54102d30 0x7f7c54103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 -- 172.21.15.16:0/1119310639 >> 172.21.15.16:0/1119310639 conn(0x7f7c540fc7d0 msgr2=0x7f7c540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 -- 172.21.15.16:0/1119310639 shutdown_connections 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.193+0000 7f7c5af18640 1 -- 172.21.15.16:0/1119310639 wait complete. 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.194+0000 7f7c5af18640 1 Processor -- start 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.194+0000 7f7c5af18640 1 -- start start 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c5af18640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c54102d30 0x7f7c54198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c5af18640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c541036e0 0x7f7c54198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c5af18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541082a0 0x7f7c5419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c5af18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c541120a0 con 0x7f7c541082a0 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c5af18640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c54111f20 con 0x7f7c541036e0 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c5af18640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7c54112220 con 0x7f7c54102d30 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541082a0 0x7f7c5419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541082a0 0x7f7c5419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47384/0 (socket says 172.21.15.16:47384) 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 -- 172.21.15.16:0/991858931 learned_addr learned my addr 172.21.15.16:0/991858931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:47.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c58cb4640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c54102d30 0x7f7c54198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c53fff640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c541036e0 0x7f7c54198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 -- 172.21.15.16:0/991858931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c54102d30 msgr2=0x7f7c54198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c54102d30 0x7f7c54198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 -- 172.21.15.16:0/991858931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c541036e0 msgr2=0x7f7c54198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c541036e0 0x7f7c54198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 -- 172.21.15.16:0/991858931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5419d530 con 0x7f7c541082a0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.195+0000 7f7c594b5640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541082a0 0x7f7c5419ce50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00cce0 tx=0x7f7c4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.196+0000 7f7c51ffb640 1 -- 172.21.15.16:0/991858931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00f450 con 0x7f7c541082a0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.196+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5419d820 con 0x7f7c541082a0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.196+0000 7f7c51ffb640 1 -- 172.21.15.16:0/991858931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c4c00fad0 con 0x7f7c541082a0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.196+0000 7f7c51ffb640 1 -- 172.21.15.16:0/991858931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c018c10 con 0x7f7c541082a0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.196+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c541a5100 con 0x7f7c541082a0 2024-09-15T22:38:47.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.197+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c54103110 con 0x7f7c541082a0 2024-09-15T22:38:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.199+0000 7f7c51ffb640 1 -- 172.21.15.16:0/991858931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f7c4c00f5f0 con 0x7f7c541082a0 2024-09-15T22:38:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.199+0000 7f7c51ffb640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c440a3440 0x7f7c440a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.200+0000 7f7c51ffb640 1 -- 172.21.15.16:0/991858931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7c4c0d07a0 con 0x7f7c541082a0 2024-09-15T22:38:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.200+0000 7f7c58cb4640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c440a3440 0x7f7c440a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.200+0000 7f7c58cb4640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c440a3440 0x7f7c440a5900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c48005fd0 tx=0x7f7c4800b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:47.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.204+0000 7f7c51ffb640 1 -- 172.21.15.16:0/991858931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c4c014030 con 0x7f7c541082a0 2024-09-15T22:38:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.418+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c5410a2b0 con 0x7f7c541082a0 2024-09-15T22:38:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.419+0000 7f7c51ffb640 1 -- 172.21.15.16:0/991858931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f7c4c09ccb0 con 0x7f7c541082a0 2024-09-15T22:38:47.422 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:38:47.422 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:38:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:38:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c440a3440 msgr2=0x7f7c440a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c440a3440 0x7f7c440a5900 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c48005fd0 tx=0x7f7c4800b390 comp rx=0 tx=0).stop 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541082a0 msgr2=0x7f7c5419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541082a0 0x7f7c5419ce50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00cce0 tx=0x7f7c4c00a620 comp rx=0 tx=0).stop 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 shutdown_connections 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c541082a0 0x7f7c5419ce50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c541036e0 0x7f7c54198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c440a3440 0x7f7c440a5900 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 --2- 172.21.15.16:0/991858931 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c54102d30 0x7f7c54198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 >> 172.21.15.16:0/991858931 conn(0x7f7c540fc7d0 msgr2=0x7f7c54105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 shutdown_connections 2024-09-15T22:38:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:47.422+0000 7f7c5af18640 1 -- 172.21.15.16:0/991858931 wait complete. 2024-09-15T22:38:47.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:47 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/991858931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:47.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:47 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/991858931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:47.798 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:47 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/991858931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:38:48.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:36:08.871593+0000', 'modified': '2024-09-15T22:38:23.657387+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24500, 'mds_1': 24479, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24445': {'gid': 24445, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6835/2738921095', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6834', 'nonce': 2738921095}, {'type': 'v1', 'addr': '172.21.15.130:6835', 'nonce': 2738921095}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6835/2202482741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 2202482741}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2202482741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.130:6837/1773539140', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6836', 'nonce': 1773539140}, {'type': 'v1', 'addr': '172.21.15.130:6837', 'nonce': 1773539140}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.130:6839/1236682624', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.130:6838', 'nonce': 1236682624}, {'type': 'v1', 'addr': '172.21.15.130:6839', 'nonce': 1236682624}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.59:6837/2084675816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6836', 'nonce': 2084675816}, {'type': 'v1', 'addr': '172.21.15.59:6837', 'nonce': 2084675816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6839/384228279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 384228279}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 384228279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24500, 24479, 24469]} 2024-09-15T22:38:48.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:38:48.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:38:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:48.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:48 smithi059 ceph-mon[34872]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-15T22:38:48.719 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:48 smithi016 ceph-mon[31492]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-15T22:38:48.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:48 smithi130 ceph-mon[35742]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-15T22:38:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.934+0000 7feddcf0c640 1 -- 172.21.15.16:0/81568145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8101de0 msgr2=0x7fedd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.934+0000 7feddcf0c640 1 --2- 172.21.15.16:0/81568145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8101de0 0x7fedd81021c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009ca0 tx=0x7fedc802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 -- 172.21.15.16:0/81568145 shutdown_connections 2024-09-15T22:38:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 --2- 172.21.15.16:0/81568145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd8106f10 0x7fedd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 --2- 172.21.15.16:0/81568145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedd8102790 0x7fedd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 --2- 172.21.15.16:0/81568145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8101de0 0x7fedd81021c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 -- 172.21.15.16:0/81568145 >> 172.21.15.16:0/81568145 conn(0x7fedd80fd8c0 msgr2=0x7fedd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 -- 172.21.15.16:0/81568145 shutdown_connections 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 -- 172.21.15.16:0/81568145 wait complete. 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.935+0000 7feddcf0c640 1 Processor -- start 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7feddcf0c640 1 -- start start 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7feddcf0c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd8101de0 0x7fedd80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7feddcf0c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedd8102790 0x7fedd8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7feddcf0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8106f10 0x7fedd8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7feddcf0c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8111f30 con 0x7fedd8106f10 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7feddcf0c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8111db0 con 0x7fedd8101de0 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7feddcf0c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81120b0 con 0x7fedd8102790 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8106f10 0x7fedd8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8106f10 0x7fedd8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47398/0 (socket says 172.21.15.16:47398) 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 -- 172.21.15.16:0/4091174707 learned_addr learned my addr 172.21.15.16:0/4091174707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd659c640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd8101de0 0x7fedd80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedd8102790 0x7fedd8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 -- 172.21.15.16:0/4091174707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedd8102790 msgr2=0x7fedd8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedd8102790 0x7fedd8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 -- 172.21.15.16:0/4091174707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd8101de0 msgr2=0x7fedd80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd8101de0 0x7fedd80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.936+0000 7fedd6d9d640 1 -- 172.21.15.16:0/4091174707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd81a6920 con 0x7fedd8106f10 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.937+0000 7fedd6d9d640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8106f10 0x7fedd8074f70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d750 tx=0x7fedd000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.937+0000 7fedcf7fe640 1 -- 172.21.15.16:0/4091174707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0017070 con 0x7fedd8106f10 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.937+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd81a6c10 con 0x7fedd8106f10 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.937+0000 7fedcf7fe640 1 -- 172.21.15.16:0/4091174707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd0005500 con 0x7fedd8106f10 2024-09-15T22:38:48.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.937+0000 7fedcf7fe640 1 -- 172.21.15.16:0/4091174707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0004d90 con 0x7fedd8106f10 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.937+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a7100 con 0x7fedd8106f10 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.939+0000 7fedcf7fe640 1 -- 172.21.15.16:0/4091174707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7fedd0004f30 con 0x7fedd8106f10 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.939+0000 7feddf2e5740 1 Processor -- start 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7feddf2e5740 1 -- start start 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f8cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7feddf2e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b0f828090 con 0x563b0f8c22f0 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7feddf2e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b0f7cfa70 con 0x563b0f8c19d0 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7feddf2e5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563b0f824050 con 0x563b0f8ccbb0 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedcf7fe640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fedc40a3440 0x7fedc40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47412/0 (socket says 172.21.15.16:47412) 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 -- 172.21.15.16:0/2017186758 learned_addr learned my addr 172.21.15.16:0/2017186758 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd6d9d640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fedc40a3440 0x7fedc40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f8cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedcf7fe640 1 -- 172.21.15.16:0/4091174707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fedd00cfae0 con 0x7fedd8106f10 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 -- 172.21.15.16:0/2017186758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 msgr2=0x563b0f8cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f8cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 -- 172.21.15.16:0/2017186758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 msgr2=0x563b0f8c1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8c1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.940+0000 7fedd659c640 1 -- 172.21.15.16:0/2017186758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0f7dc130 con 0x563b0f8c22f0 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.941+0000 7fedd6d9d640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fedc40a3440 0x7fedc40a5900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedd00cce70 tx=0x7fedd0009bd0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.941+0000 7fedd659c640 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8cc670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009bf0 tx=0x7fedc8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e00d8b03b5da8c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.941+0000 7fedccff9640 1 -- 172.21.15.16:0/2017186758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8049070 con 0x563b0f8c22f0 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.941+0000 7fedccff9640 1 -- 172.21.15.16:0/2017186758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8036040 con 0x563b0f8c22f0 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.941+0000 7fedccff9640 1 -- 172.21.15.16:0/2017186758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc802fb70 con 0x563b0f8c22f0 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- 172.21.15.16:0/2017186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 msgr2=0x563b0f8cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8cc670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009bf0 tx=0x7fedc8005fa0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- 172.21.15.16:0/2017186758 shutdown_connections 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f8cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8cc670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 --2- 172.21.15.16:0/2017186758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8c1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- 172.21.15.16:0/2017186758 >> 172.21.15.16:0/2017186758 conn(0x563b0f8bdf60 msgr2=0x563b0f8be340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- 172.21.15.16:0/2017186758 shutdown_connections 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- 172.21.15.16:0/2017186758 wait complete. 2024-09-15T22:38:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- start start 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 Processor -- start 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c22f0 0x563b0f8c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8cf600 con 0x563b0f8ccbb0 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b0f7e2f20 con 0x563b0f8c22f0 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.942+0000 7feddf2e5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8cf780 con 0x563b0f8c19d0 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7feddf2e5740 10 client.0 _ll_register_callbacks cb 0x563b0f7ac030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47426/0 (socket says 172.21.15.16:47426) 2024-09-15T22:38:48.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 -- 172.21.15.16:0/3495394730 learned_addr learned my addr 172.21.15.16:0/3495394730 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd659c640 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c22f0 0x563b0f8c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 -- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 msgr2=0x563b0f8cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 -- 172.21.15.16:0/3495394730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c22f0 msgr2=0x563b0f8c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c22f0 0x563b0f8c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 -- 172.21.15.16:0/3495394730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0f5992b0 con 0x563b0f8ccbb0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c9d00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fedc0012ba0 tx=0x7fedc0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc4f3f86fc8661f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedbdffb640 1 -- 172.21.15.16:0/3495394730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0012110 con 0x563b0f8ccbb0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedbdffb640 1 -- 172.21.15.16:0/3495394730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fedc0010c00 con 0x563b0f8ccbb0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc8036040 con 0x563b0f8ccbb0 2024-09-15T22:38:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7fedbdffb640 1 -- 172.21.15.16:0/3495394730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fedc00122b0 con 0x563b0f8ccbb0 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7feddf2e5740 10 client.15489 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.943+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b0f8dafe0 con 0x563b0f8ccbb0 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7fedbdffb640 1 -- 172.21.15.16:0/3495394730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8036040 con 0x563b0f8ccbb0 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7fedbdffb640 1 -- 172.21.15.16:0/3495394730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fedc0004920 con 0x563b0f8ccbb0 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7feddf2e5740 10 client.15489 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7feddf2e5740 10 client.15489 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7feddf2e5740 20 client.15489 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7feddf2e5740 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f8dd5b0 0x563b0f8dea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7feddf2e5740 4 client.15489 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b0f835600 con 0x563b0f8dd5b0 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.944+0000 7fedd659c640 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f8dd5b0 0x563b0f8dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7fedd659c640 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f8dd5b0 0x563b0f8dea20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e1538ddf1e53 server_cookie=bc8a70bd57b85ddd in_seq=0 out_seq=0 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7fedbdffb640 10 client.15489 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7fedbdffb640 1 -- 172.21.15.16:0/3495394730 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b0f835600 con 0x563b0f8dd5b0 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7fedbdffb640 10 client.15489 handle_command_reply: tid=0 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 client.15489 shutdown 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8ccbb0 msgr2=0x563b0f8c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c9d00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fedc0012ba0 tx=0x7fedc0014040 comp rx=0 tx=0).stop 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 shutdown_connections 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f8dd5b0 0x563b0f8dea20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c9d00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c22f0 0x563b0f8c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7fedbdffb640 0 client.15489 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 --2- 172.21.15.16:0/3495394730 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.945+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 >> 172.21.15.16:0/3495394730 conn(0x563b0f8bdf60 msgr2=0x563b0f8c3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.946+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 shutdown_connections 2024-09-15T22:38:48.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.946+0000 7feddf2e5740 1 -- 172.21.15.16:0/3495394730 wait complete. 2024-09-15T22:38:48.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.946+0000 7feddf2e5740 20 client.15489 trim_cache size 0 max 16384 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 Processor -- start 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 -- start start 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8dea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8dcd60 con 0x563b0f8c22f0 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8dd430 con 0x563b0f8ccbb0 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7feddf2e5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8ddb00 con 0x563b0f8c19d0 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7fedd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7fedd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47438/0 (socket says 172.21.15.16:47438) 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.964+0000 7fedd659c640 1 -- 172.21.15.16:0/296459873 learned_addr learned my addr 172.21.15.16:0/296459873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:48.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd659c640 1 -- 172.21.15.16:0/296459873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 msgr2=0x563b0f8dea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd659c640 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8dea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd659c640 1 -- 172.21.15.16:0/296459873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8ccbb0 msgr2=0x563b0f8c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd659c640 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd659c640 1 -- 172.21.15.16:0/296459873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0f8c1800 con 0x563b0f8c22f0 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedd659c640 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8c8090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fedc802f680 tx=0x7fedc802f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecb83616b10361b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedbffff640 1 -- 172.21.15.16:0/296459873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8049040 con 0x563b0f8c22f0 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedbffff640 1 -- 172.21.15.16:0/296459873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8036040 con 0x563b0f8c22f0 2024-09-15T22:38:48.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.965+0000 7fedbffff640 1 -- 172.21.15.16:0/296459873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc80305c0 con 0x563b0f8c22f0 2024-09-15T22:38:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 -- 172.21.15.16:0/296459873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 msgr2=0x563b0f8c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8c8090 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fedc802f680 tx=0x7fedc802f6b0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 -- 172.21.15.16:0/296459873 shutdown_connections 2024-09-15T22:38:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8ccbb0 0x563b0f8c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8c8090 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 --2- 172.21.15.16:0/296459873 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8c19d0 0x563b0f8dea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 -- 172.21.15.16:0/296459873 >> 172.21.15.16:0/296459873 conn(0x563b0f8bdf60 msgr2=0x563b0f8c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 -- 172.21.15.16:0/296459873 shutdown_connections 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 -- 172.21.15.16:0/296459873 wait complete. 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 -- start start 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.966+0000 7feddf2e5740 1 Processor -- start 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7feddf2e5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f945cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7feddf2e5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8de8a0 con 0x563b0f8c22f0 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7feddf2e5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8de1d0 con 0x563b0f8c19d0 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7feddf2e5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563b0f8def70 con 0x563b0f8ccbb0 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7feddf2e5740 10 client.0 _ll_register_callbacks cb 0x7fedc000be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8e2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47440/0 (socket says 172.21.15.16:47440) 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 -- 172.21.15.16:0/215732590 learned_addr learned my addr 172.21.15.16:0/215732590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f945cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 -- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 msgr2=0x563b0f945cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f945cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 -- 172.21.15.16:0/215732590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 msgr2=0x563b0f8e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8e1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd659c640 1 -- 172.21.15.16:0/215732590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0f8e2b40 con 0x563b0f8c22f0 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.967+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7fedd659c640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8e2290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fedc802ade0 tx=0x7fedc802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16bfac8ed38c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7fedbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7fedbdffb640 1 -- 172.21.15.16:0/215732590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8049040 con 0x563b0f8c22f0 2024-09-15T22:38:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7fedbdffb640 1 -- 172.21.15.16:0/215732590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fedc802fd60 con 0x563b0f8c22f0 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc8036040 con 0x563b0f8c22f0 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7fedbdffb640 1 -- 172.21.15.16:0/215732590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fedc802f7c0 con 0x563b0f8c22f0 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7feddf2e5740 10 client.15495 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b0f947240 con 0x563b0f8c22f0 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7fedbdffb640 1 -- 172.21.15.16:0/215732590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8036040 con 0x563b0f8c22f0 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7fedbdffb640 1 -- 172.21.15.16:0/215732590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fedc8004040 con 0x563b0f8c22f0 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7feddf2e5740 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7feddf2e5740 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7feddf2e5740 20 client.15495 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.968+0000 7feddf2e5740 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f94b980 0x563b0f94a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7feddf2e5740 4 client.15495 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:38:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "status"}) -- 0x563b0f82fb10 con 0x563b0f94b980 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f94b980 0x563b0f94a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7fedd5d9b640 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f94b980 0x563b0f94a690 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad1d34e0fc832acd server_cookie=4de4771becff1897 in_seq=0 out_seq=0 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7fedbdffb640 10 client.15495 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7fedbdffb640 1 -- 172.21.15.16:0/215732590 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563b0f82fb10 con 0x563b0f94b980 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7fedbdffb640 10 client.15495 handle_command_reply: tid=0 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.969+0000 7feddf2e5740 1 client.15495 shutdown 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 msgr2=0x563b0f8e2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8e2290 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fedc802ade0 tx=0x7fedc802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 shutdown_connections 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563b0f94b980 0x563b0f94a690 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b0f8ccbb0 0x563b0f945cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b0f8c22f0 0x563b0f8e2290 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 --2- 172.21.15.16:0/215732590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b0f8c19d0 0x563b0f8e1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7fedbdffb640 0 client.15495 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 >> 172.21.15.16:0/215732590 conn(0x563b0f8bdf60 msgr2=0x563b0f8c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 shutdown_connections 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 1 -- 172.21.15.16:0/215732590 wait complete. 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddf2e5740 20 client.15495 trim_cache size 0 max 16384 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:38:48.978 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 2, 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24469, 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 124.594645371, 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 26, 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 89, 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 179.77921434699999 2024-09-15T22:38:48.979 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fedc40a3440 msgr2=0x7fedc40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fedc40a3440 0x7fedc40a5900 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedd00cce70 tx=0x7fedd0009bd0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8106f10 msgr2=0x7fedd8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8106f10 0x7fedd8074f70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fedd000d750 tx=0x7fedd000dc10 comp rx=0 tx=0).stop 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 shutdown_connections 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedd8106f10 0x7fedd8074f70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fedc40a3440 0x7fedc40a5900 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fedd8102790 0x7fedd8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.970+0000 7feddcf0c640 1 --2- 172.21.15.16:0/4091174707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fedd8101de0 0x7fedd80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.971+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 >> 172.21.15.16:0/4091174707 conn(0x7fedd80fd8c0 msgr2=0x7fedd80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:48.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.971+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 shutdown_connections 2024-09-15T22:38:48.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:48.971+0000 7feddcf0c640 1 -- 172.21.15.16:0/4091174707 wait complete. 2024-09-15T22:38:49.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:38:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 -- 172.21.15.16:0/2697843221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a4102d30 msgr2=0x7fe6a4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/2697843221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a4102d30 0x7fe6a4103110 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe694009930 tx=0x7fe69402a420 comp rx=0 tx=0).stop 2024-09-15T22:38:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 -- 172.21.15.16:0/2697843221 shutdown_connections 2024-09-15T22:38:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/2697843221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a41082a0 0x7fe6a410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/2697843221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6a41036e0 0x7fe6a4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/2697843221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a4102d30 0x7fe6a4103110 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 -- 172.21.15.16:0/2697843221 >> 172.21.15.16:0/2697843221 conn(0x7fe6a40fc7d0 msgr2=0x7fe6a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 -- 172.21.15.16:0/2697843221 shutdown_connections 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 -- 172.21.15.16:0/2697843221 wait complete. 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 Processor -- start 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.377+0000 7fe6ab06a640 1 -- start start 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6ab06a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a4102d30 0x7fe6a4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6ab06a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6a41036e0 0x7fe6a4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6ab06a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41082a0 0x7fe6a4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6ab06a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41120a0 con 0x7fe6a41082a0 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6ab06a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4111f20 con 0x7fe6a4102d30 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6ab06a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4112220 con 0x7fe6a41036e0 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41082a0 0x7fe6a4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41082a0 0x7fe6a4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47460/0 (socket says 172.21.15.16:47460) 2024-09-15T22:38:50.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 -- 172.21.15.16:0/1507016620 learned_addr learned my addr 172.21.15.16:0/1507016620 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6a41036e0 0x7fe6a4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a4102d30 0x7fe6a4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 -- 172.21.15.16:0/1507016620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6a41036e0 msgr2=0x7fe6a4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6a41036e0 0x7fe6a4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 -- 172.21.15.16:0/1507016620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a4102d30 msgr2=0x7fe6a4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a4102d30 0x7fe6a4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.378+0000 7fe6a9607640 1 -- 172.21.15.16:0/1507016620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a41a6a50 con 0x7fe6a41082a0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.379+0000 7fe6a9607640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41082a0 0x7fe6a4101410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00cad0 tx=0x7fe69c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.379+0000 7fe6a1ffb640 1 -- 172.21.15.16:0/1507016620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00ae20 con 0x7fe6a41082a0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.379+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a41a6d40 con 0x7fe6a41082a0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.379+0000 7fe6a1ffb640 1 -- 172.21.15.16:0/1507016620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe69c005020 con 0x7fe6a41082a0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.379+0000 7fe6a1ffb640 1 -- 172.21.15.16:0/1507016620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00f6f0 con 0x7fe6a41082a0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.379+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a41a7230 con 0x7fe6a41082a0 2024-09-15T22:38:50.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.381+0000 7fe6a1ffb640 1 -- 172.21.15.16:0/1507016620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7fe69c0051d0 con 0x7fe6a41082a0 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.382+0000 7fe6a1ffb640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6900a3440 0x7fe6900a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.382+0000 7fe6a1ffb640 1 -- 172.21.15.16:0/1507016620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe69c0d01e0 con 0x7fe6a41082a0 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.382+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6900a3440 0x7fe6900a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6900a3440 0x7fe6900a5900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe694000c00 tx=0x7fe694005a20 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 Processor -- start 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 -- start start 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60de670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 0x562ce60e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ce603a090 con 0x562ce60d42f0 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ce5fe1a70 con 0x562ce60d39d0 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6ad443740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562ce6036050 con 0x562ce60debb0 2024-09-15T22:38:50.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60de670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60de670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47472/0 (socket says 172.21.15.16:47472) 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6a3fff640 1 -- 172.21.15.16:0/779346363 learned_addr learned my addr 172.21.15.16:0/779346363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.383+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe6a9607640 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 0x562ce60e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe6a3fff640 1 -- 172.21.15.16:0/779346363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 msgr2=0x562ce60e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 0x562ce60e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe6a3fff640 1 -- 172.21.15.16:0/779346363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 msgr2=0x562ce60d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe6a3fff640 1 -- 172.21.15.16:0/779346363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce5fee130 con 0x562ce60d42f0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60de670 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00bc60 tx=0x7fe68c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb40da95509d8393 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe69affd640 1 -- 172.21.15.16:0/779346363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c01c480 con 0x562ce60d42f0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe69affd640 1 -- 172.21.15.16:0/779346363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c01cb00 con 0x562ce60d42f0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.384+0000 7fe69affd640 1 -- 172.21.15.16:0/779346363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c011cb0 con 0x562ce60d42f0 2024-09-15T22:38:50.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 -- 172.21.15.16:0/779346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 msgr2=0x562ce60de670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60de670 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00bc60 tx=0x7fe68c0107a0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 -- 172.21.15.16:0/779346363 shutdown_connections 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 0x562ce60e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60de670 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 --2- 172.21.15.16:0/779346363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 -- 172.21.15.16:0/779346363 >> 172.21.15.16:0/779346363 conn(0x562ce60cff60 msgr2=0x562ce60d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 -- 172.21.15.16:0/779346363 shutdown_connections 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.385+0000 7fe6ad443740 1 -- 172.21.15.16:0/779346363 wait complete. 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 -- start start 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 Processor -- start 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60de470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d42f0 0x562ce60db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60debb0 0x562ce60dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ce60e1780 con 0x562ce60debb0 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ce5ff4f20 con 0x562ce60d39d0 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562ce60e1900 con 0x562ce60d42f0 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6ad443740 10 client.0 _ll_register_callbacks cb 0x562ce5fbe030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6a9607640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60debb0 0x562ce60dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6a9607640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60debb0 0x562ce60dbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47482/0 (socket says 172.21.15.16:47482) 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6a9607640 1 -- 172.21.15.16:0/343788740 learned_addr learned my addr 172.21.15.16:0/343788740 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.386+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d42f0 0x562ce60db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6a9607640 1 -- 172.21.15.16:0/343788740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d42f0 msgr2=0x562ce60db7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6a9607640 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d42f0 0x562ce60db7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6a9607640 1 -- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 msgr2=0x562ce60de470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6a9607640 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60de470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6a9607640 1 -- 172.21.15.16:0/343788740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce5dab2b0 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6a9607640 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60debb0 0x562ce60dbd00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe69c09bcf0 tx=0x7fe69c09c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e3cff29f46496 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe683fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe683fff640 1 -- 172.21.15.16:0/343788740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c01d040 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe683fff640 1 -- 172.21.15.16:0/343788740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe69c0cd7c0 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe68c01cb00 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe683fff640 1 -- 172.21.15.16:0/343788740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe69c0cd9c0 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6ad443740 10 client.15507 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ce60ed130 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.387+0000 7fe683fff640 1 -- 172.21.15.16:0/343788740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c01cb00 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe683fff640 1 -- 172.21.15.16:0/343788740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe69c086040 con 0x562ce60debb0 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe6ad443740 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe6ad443740 10 client.15507 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe6ad443740 20 client.15507 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe6ad443740 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce60f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe6ad443740 4 client.15507 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce6047600 con 0x562ce60ef6e0 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.388+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce60f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.389+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce60f0b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=227dab20c6e5bf92 server_cookie=efb1f19bd8167319 in_seq=0 out_seq=0 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.389+0000 7fe683fff640 10 client.15507 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.389+0000 7fe683fff640 1 -- 172.21.15.16:0/343788740 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ce6047600 con 0x562ce60ef6e0 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.389+0000 7fe683fff640 10 client.15507 handle_command_reply: tid=0 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.389+0000 7fe6ad443740 1 client.15507 shutdown 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.389+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60debb0 msgr2=0x562ce60dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.389+0000 7fe6ad443740 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60debb0 0x562ce60dbd00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe69c09bcf0 tx=0x7fe69c09c7b0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 shutdown_connections 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce60f0b50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe683fff640 0 client.15507 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:50.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60debb0 0x562ce60dbd00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d42f0 0x562ce60db7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 --2- 172.21.15.16:0/343788740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60de470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 >> 172.21.15.16:0/343788740 conn(0x562ce60cff60 msgr2=0x562ce60d5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 shutdown_connections 2024-09-15T22:38:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 1 -- 172.21.15.16:0/343788740 wait complete. 2024-09-15T22:38:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.390+0000 7fe6ad443740 20 client.15507 trim_cache size 0 max 16384 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 Processor -- start 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 -- start start 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d39d0 0x562ce60f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60debb0 0x562ce60da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ce60eee90 con 0x562ce60d42f0 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ce60ef560 con 0x562ce60debb0 2024-09-15T22:38:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6ad443740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562ce60efc30 con 0x562ce60d39d0 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47488/0 (socket says 172.21.15.16:47488) 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6a3fff640 1 -- 172.21.15.16:0/688579209 learned_addr learned my addr 172.21.15.16:0/688579209 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.422+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d39d0 0x562ce60f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a3fff640 1 -- 172.21.15.16:0/688579209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d39d0 msgr2=0x562ce60f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d39d0 0x562ce60f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a9607640 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60debb0 0x562ce60da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a3fff640 1 -- 172.21.15.16:0/688579209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60debb0 msgr2=0x562ce60da0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60debb0 0x562ce60da0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a3fff640 1 -- 172.21.15.16:0/688579209 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce60e1890 con 0x562ce60d42f0 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a9607640 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60debb0 0x562ce60da0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60d0340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00bbb0 tx=0x7fe68c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fb0262e2c86553 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe69a7fc640 1 -- 172.21.15.16:0/688579209 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c00be10 con 0x562ce60d42f0 2024-09-15T22:38:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe69a7fc640 1 -- 172.21.15.16:0/688579209 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c011910 con 0x562ce60d42f0 2024-09-15T22:38:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.423+0000 7fe69a7fc640 1 -- 172.21.15.16:0/688579209 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c005500 con 0x562ce60d42f0 2024-09-15T22:38:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 -- 172.21.15.16:0/688579209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 msgr2=0x562ce60d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60d0340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00bbb0 tx=0x7fe68c011810 comp rx=0 tx=0).stop 2024-09-15T22:38:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 -- 172.21.15.16:0/688579209 shutdown_connections 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60debb0 0x562ce60da0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60d0340 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 --2- 172.21.15.16:0/688579209 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60d39d0 0x562ce60f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 -- 172.21.15.16:0/688579209 >> 172.21.15.16:0/688579209 conn(0x562ce60cff60 msgr2=0x562ce60db780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 -- 172.21.15.16:0/688579209 shutdown_connections 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 -- 172.21.15.16:0/688579209 wait complete. 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 -- start start 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.424+0000 7fe6ad443740 1 Processor -- start 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6ad443740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 0x562ce60f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6ad443740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562ce60f09d0 con 0x562ce60d42f0 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6ad443740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562ce60f0300 con 0x562ce60d39d0 2024-09-15T22:38:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6ad443740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562ce60f10a0 con 0x562ce60debb0 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6ad443740 10 client.0 _ll_register_callbacks cb 0x7fe69c0cd3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47492/0 (socket says 172.21.15.16:47492) 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 -- 172.21.15.16:0/1062513961 learned_addr learned my addr 172.21.15.16:0/1062513961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 -- 172.21.15.16:0/1062513961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 msgr2=0x562ce60f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 0x562ce60f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 -- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 msgr2=0x562ce60f9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60f9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.425+0000 7fe6a3fff640 1 -- 172.21.15.16:0/1062513961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce60f5550 con 0x562ce60d42f0 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60f4830 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005d00 tx=0x7fe68c0116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f990db3b441d691 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe683fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe683fff640 1 -- 172.21.15.16:0/1062513961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c034060 con 0x562ce60d42f0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe683fff640 1 -- 172.21.15.16:0/1062513961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe68c004020 con 0x562ce60d42f0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe68c011910 con 0x562ce60d42f0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe683fff640 1 -- 172.21.15.16:0/1062513961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe68c03b050 con 0x562ce60d42f0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe6ad443740 10 client.15513 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ce6159d10 con 0x562ce60d42f0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.426+0000 7fe683fff640 1 -- 172.21.15.16:0/1062513961 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c011910 con 0x562ce60d42f0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe683fff640 1 -- 172.21.15.16:0/1062513961 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe68c01c640 con 0x562ce60d42f0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6ad443740 10 client.15513 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6ad443740 10 client.15513 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6ad443740 20 client.15513 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6ad443740 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce615ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6ad443740 4 client.15513 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "status"}) -- 0x562ce6041b10 con 0x562ce60ef6e0 2024-09-15T22:38:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce615ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6a8e06640 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce615ce90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a3e8f70397d7b99 server_cookie=fbbe3d2d1e4fcc04 in_seq=0 out_seq=0 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe683fff640 10 client.15513 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe683fff640 1 -- 172.21.15.16:0/1062513961 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562ce6041b10 con 0x562ce60ef6e0 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe683fff640 10 client.15513 handle_command_reply: tid=0 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.427+0000 7fe6ad443740 1 client.15513 shutdown 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 msgr2=0x562ce60f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60f4830 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005d00 tx=0x7fe68c0116e0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 shutdown_connections 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562ce60ef6e0 0x562ce615ce90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe683fff640 0 client.15513 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562ce60debb0 0x562ce60f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562ce60d42f0 0x562ce60f4830 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 --2- 172.21.15.16:0/1062513961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562ce60d39d0 0x562ce60f9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 >> 172.21.15.16:0/1062513961 conn(0x562ce60cff60 msgr2=0x562ce60d9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 shutdown_connections 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 1 -- 172.21.15.16:0/1062513961 wait complete. 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.428+0000 7fe6ad443740 20 client.15513 trim_cache size 0 max 16384 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 1, 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24479, 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:38:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 127.05758810499999, 2024-09-15T22:38:50.436 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 26, 2024-09-15T22:38:50.436 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-09-15T22:38:50.436 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 89, 2024-09-15T22:38:50.436 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 183.77591555199999 2024-09-15T22:38:50.436 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6900a3440 msgr2=0x7fe6900a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6900a3440 0x7fe6900a5900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe694000c00 tx=0x7fe694005a20 comp rx=0 tx=0).stop 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41082a0 msgr2=0x7fe6a4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41082a0 0x7fe6a4101410 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00cad0 tx=0x7fe69c00cf90 comp rx=0 tx=0).stop 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 shutdown_connections 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41082a0 0x7fe6a4101410 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6900a3440 0x7fe6900a5900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6a41036e0 0x7fe6a4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 --2- 172.21.15.16:0/1507016620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6a4102d30 0x7fe6a4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 >> 172.21.15.16:0/1507016620 conn(0x7fe6a40fc7d0 msgr2=0x7fe6a4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 shutdown_connections 2024-09-15T22:38:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:50.429+0000 7fe6ab06a640 1 -- 172.21.15.16:0/1507016620 wait complete. 2024-09-15T22:38:50.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:50 smithi016 ceph-mon[31492]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:38:50.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:50 smithi059 ceph-mon[34872]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:38:50.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:50 smithi130 ceph-mon[35742]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:38:51.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:38:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:38:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 -- 172.21.15.16:0/2344770617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 msgr2=0x7f64b81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/2344770617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b81021c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009900 tx=0x7f64a802a410 comp rx=0 tx=0).stop 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 -- 172.21.15.16:0/2344770617 shutdown_connections 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/2344770617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b8106f10 0x7f64b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/2344770617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64b8102790 0x7f64b81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/2344770617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b81021c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 -- 172.21.15.16:0/2344770617 >> 172.21.15.16:0/2344770617 conn(0x7f64b80fd8c0 msgr2=0x7f64b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 -- 172.21.15.16:0/2344770617 shutdown_connections 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.994+0000 7f64bd2aa640 1 -- 172.21.15.16:0/2344770617 wait complete. 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 Processor -- start 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 -- start start 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b8102790 0x7f64b81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64b8106f10 0x7f64b819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111f70 con 0x7f64b8101de0 2024-09-15T22:38:52.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f64b8111df0 con 0x7f64b8106f10 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64bd2aa640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f64b81120f0 con 0x7f64b8102790 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43054/0 (socket says 172.21.15.16:43054) 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64b6ffd640 1 -- 172.21.15.16:0/828962616 learned_addr learned my addr 172.21.15.16:0/828962616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b8102790 0x7f64b81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64b77fe640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64b8106f10 0x7f64b819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64b6ffd640 1 -- 172.21.15.16:0/828962616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b8102790 msgr2=0x7f64b81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b8102790 0x7f64b81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.995+0000 7f64b6ffd640 1 -- 172.21.15.16:0/828962616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64b8106f10 msgr2=0x7f64b819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64b8106f10 0x7f64b819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64b6ffd640 1 -- 172.21.15.16:0/828962616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b819d420 con 0x7f64b8101de0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64b77fe640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64b8106f10 0x7f64b819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b8198470 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f64a8006010 tx=0x7f64a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64b4ff9640 1 -- 172.21.15.16:0/828962616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a803f030 con 0x7f64b8101de0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b819d6b0 con 0x7f64b8101de0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64b4ff9640 1 -- 172.21.15.16:0/828962616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a8003ce0 con 0x7f64b8101de0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64b4ff9640 1 -- 172.21.15.16:0/828962616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8035070 con 0x7f64b8101de0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.996+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b81a4ff0 con 0x7f64b8101de0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.998+0000 7f64b4ff9640 1 -- 172.21.15.16:0/828962616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 138531+0+0 (secure 0 0 0) 0x7f64a8042030 con 0x7f64b8101de0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 Processor -- start 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 -- start start 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64b4ff9640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f64a40a3440 0x7f64a40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64b77fe640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f64a40a3440 0x7f64a40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64b4ff9640 1 -- 172.21.15.16:0/828962616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f64a80f0790 con 0x7f64b8101de0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170a8d0 0x55b0c170acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1719a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c1719fc0 0x55b0c171c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b0c1671090 con 0x55b0c170b1f0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0c1618a70 con 0x55b0c170a8d0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64bf683740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b0c166d050 con 0x55b0c1719fc0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64ae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1719a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64ae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1719a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43066/0 (socket says 172.21.15.16:43066) 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64ae7fc640 1 -- 172.21.15.16:0/590657709 learned_addr learned my addr 172.21.15.16:0/590657709 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:51.999+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170a8d0 0x55b0c170acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64b77fe640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f64a40a3440 0x7f64a40a5900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64b00092f0 tx=0x7f64b0009280 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64b77fe640 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c1719fc0 0x55b0c171c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ae7fc640 1 -- 172.21.15.16:0/590657709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c1719fc0 msgr2=0x55b0c171c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c1719fc0 0x55b0c171c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ae7fc640 1 -- 172.21.15.16:0/590657709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170a8d0 msgr2=0x55b0c170acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170a8d0 0x55b0c170acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ae7fc640 1 -- 172.21.15.16:0/590657709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c1625130 con 0x55b0c170b1f0 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64b77fe640 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c1719fc0 0x55b0c171c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170a8d0 0x55b0c170acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1719a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f64a000abf0 tx=0x7f64a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6277d3a6c7f30ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ad7fa640 1 -- 172.21.15.16:0/590657709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a001d070 con 0x55b0c170b1f0 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ad7fa640 1 -- 172.21.15.16:0/590657709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a0018b20 con 0x55b0c170b1f0 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.000+0000 7f64ad7fa640 1 -- 172.21.15.16:0/590657709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a0010ac0 con 0x55b0c170b1f0 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 -- 172.21.15.16:0/590657709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 msgr2=0x55b0c1719a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1719a80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f64a000abf0 tx=0x7f64a000f790 comp rx=0 tx=0).stop 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 -- 172.21.15.16:0/590657709 shutdown_connections 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c1719fc0 0x55b0c171c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1719a80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 --2- 172.21.15.16:0/590657709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170a8d0 0x55b0c170acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 -- 172.21.15.16:0/590657709 >> 172.21.15.16:0/590657709 conn(0x55b0c1706f60 msgr2=0x55b0c1707340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 -- 172.21.15.16:0/590657709 shutdown_connections 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 -- 172.21.15.16:0/590657709 wait complete. 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 -- start start 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.001+0000 7f64bf683740 1 Processor -- start 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64bf683740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c17198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64bf683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1716c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64bf683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1717180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64bf683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b0c171cbf0 con 0x55b0c170b1f0 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64bf683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0c162bf20 con 0x55b0c1719fc0 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64bf683740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b0c171cd70 con 0x55b0c170a8d0 2024-09-15T22:38:52.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64bf683740 10 client.0 _ll_register_callbacks cb 0x55b0c15f5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1716c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1716c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43078/0 (socket says 172.21.15.16:43078) 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 -- 172.21.15.16:0/1536475763 learned_addr learned my addr 172.21.15.16:0/1536475763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c17198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64b77fe640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1717180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 -- 172.21.15.16:0/1536475763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 msgr2=0x55b0c17198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c17198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 -- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 msgr2=0x55b0c1717180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1717180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 -- 172.21.15.16:0/1536475763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c13e22b0 con 0x55b0c170b1f0 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c17198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64b77fe640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1717180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:52.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1716c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f64a000a7c0 tx=0x7f64a00186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f60c23f555f692a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f649e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.002+0000 7f649e7fc640 1 -- 172.21.15.16:0/1536475763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a001d050 con 0x55b0c170b1f0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a0018b20 con 0x55b0c170b1f0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f649e7fc640 1 -- 172.21.15.16:0/1536475763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f64a0010840 con 0x55b0c170b1f0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f649e7fc640 1 -- 172.21.15.16:0/1536475763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64a0031050 con 0x55b0c170b1f0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 10 client.15525 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0c1728550 con 0x55b0c170b1f0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f649e7fc640 1 -- 172.21.15.16:0/1536475763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a0018b20 con 0x55b0c170b1f0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f649e7fc640 1 -- 172.21.15.16:0/1536475763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64a00046c0 con 0x55b0c170b1f0 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 10 client.15525 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 10 client.15525 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 20 client.15525 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c172a7b0 0x55b0c172bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 4 client.15525 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0c167e600 con 0x55b0c172a7b0 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.003+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c172a7b0 0x55b0c172bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.004+0000 7f649e7fc640 10 client.15525 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.004+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c172a7b0 0x55b0c172bc20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f4d1b8aa180cbe server_cookie=b8dfff3e905f8e64 in_seq=0 out_seq=0 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f649e7fc640 1 -- 172.21.15.16:0/1536475763 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0c167e600 con 0x55b0c172a7b0 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f649e7fc640 10 client.15525 handle_command_reply: tid=0 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 client.15525 shutdown 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 msgr2=0x55b0c1716c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1716c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f64a000a7c0 tx=0x7f64a00186f0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 shutdown_connections 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c172a7b0 0x55b0c172bc20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1717180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f649e7fc640 0 client.15525 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:52.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170b1f0 0x55b0c1716c40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 --2- 172.21.15.16:0/1536475763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c17198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 >> 172.21.15.16:0/1536475763 conn(0x55b0c1706f60 msgr2=0x55b0c170ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 shutdown_connections 2024-09-15T22:38:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 1 -- 172.21.15.16:0/1536475763 wait complete. 2024-09-15T22:38:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.005+0000 7f64bf683740 20 client.15525 trim_cache size 0 max 16384 2024-09-15T22:38:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 Processor -- start 2024-09-15T22:38:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 -- start start 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170a8d0 0x55b0c172a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170b1f0 0x55b0c172ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1712240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b0c1729f60 con 0x55b0c170a8d0 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0c172a630 con 0x55b0c1719fc0 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64bf683740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b0c172ad00 con 0x55b0c170b1f0 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170a8d0 0x55b0c172a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64b6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170a8d0 0x55b0c172a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43090/0 (socket says 172.21.15.16:43090) 2024-09-15T22:38:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.023+0000 7f64b6ffd640 1 -- 172.21.15.16:0/838423344 learned_addr learned my addr 172.21.15.16:0/838423344 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170b1f0 0x55b0c172ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b77fe640 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1712240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b6ffd640 1 -- 172.21.15.16:0/838423344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170b1f0 msgr2=0x55b0c172ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170b1f0 0x55b0c172ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b6ffd640 1 -- 172.21.15.16:0/838423344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 msgr2=0x55b0c1712240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1712240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b6ffd640 1 -- 172.21.15.16:0/838423344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c17080f0 con 0x55b0c170a8d0 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b77fe640 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1712240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170a8d0 0x55b0c172a0e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f64a80bc590 tx=0x7f64a8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04e72249b186e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64acff9640 1 -- 172.21.15.16:0/838423344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80d9070 con 0x55b0c170a8d0 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64acff9640 1 -- 172.21.15.16:0/838423344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a80dc070 con 0x55b0c170a8d0 2024-09-15T22:38:52.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.024+0000 7f64acff9640 1 -- 172.21.15.16:0/838423344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8049040 con 0x55b0c170a8d0 2024-09-15T22:38:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 -- 172.21.15.16:0/838423344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170a8d0 msgr2=0x55b0c172a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170a8d0 0x55b0c172a0e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f64a80bc590 tx=0x7f64a8003f10 comp rx=0 tx=0).stop 2024-09-15T22:38:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 -- 172.21.15.16:0/838423344 shutdown_connections 2024-09-15T22:38:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c1719fc0 0x55b0c1712240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170b1f0 0x55b0c172ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 --2- 172.21.15.16:0/838423344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c170a8d0 0x55b0c172a0e0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 -- 172.21.15.16:0/838423344 >> 172.21.15.16:0/838423344 conn(0x55b0c1706f60 msgr2=0x55b0c1707340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 -- 172.21.15.16:0/838423344 shutdown_connections 2024-09-15T22:38:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 -- 172.21.15.16:0/838423344 wait complete. 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 -- start start 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.025+0000 7f64bf683740 1 Processor -- start 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c172d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170b1f0 0x55b0c172d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c1719fc0 0x55b0c1792eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b0c172baa0 con 0x55b0c1719fc0 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b0c172b3d0 con 0x55b0c170b1f0 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b0c172c170 con 0x55b0c170a8d0 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 10 client.0 _ll_register_callbacks cb 0x7f64a001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c1719fc0 0x55b0c1792eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c1719fc0 0x55b0c1792eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43096/0 (socket says 172.21.15.16:43096) 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 -- 172.21.15.16:0/2223466277 learned_addr learned my addr 172.21.15.16:0/2223466277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:38:52.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c172d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64ae7fc640 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170b1f0 0x55b0c172d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 -- 172.21.15.16:0/2223466277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 msgr2=0x55b0c172d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c172d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 -- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170b1f0 msgr2=0x55b0c172d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170b1f0 0x55b0c172d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 -- 172.21.15.16:0/2223466277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c17933f0 con 0x55b0c1719fc0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64b77fe640 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c1719fc0 0x55b0c1792eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64b00128a0 tx=0x7f64b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3054806836230aef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f649e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f649e7fc640 1 -- 172.21.15.16:0/2223466277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0011d70 con 0x55b0c1719fc0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f649e7fc640 1 -- 172.21.15.16:0/2223466277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f64b002a470 con 0x55b0c1719fc0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.026+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a80dc070 con 0x55b0c1719fc0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f649e7fc640 1 -- 172.21.15.16:0/2223466277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64b0033050 con 0x55b0c1719fc0 2024-09-15T22:38:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 10 client.15531 fetch_fsmap learned FSMap version 26 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b0c1794670 con 0x55b0c1719fc0 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f649e7fc640 1 -- 172.21.15.16:0/2223466277 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64a80dc070 con 0x55b0c1719fc0 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f649e7fc640 1 -- 172.21.15.16:0/2223466277 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64b000f040 con 0x55b0c1719fc0 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 10 client.15531 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 10 client.15531 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 20 client.15531 populate_metadata read hostname 'smithi016' 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c1799290 0x55b0c1797fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 4 client.15531 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "status"}) -- 0x55b0c1678b10 con 0x55b0c1799290 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.027+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c1799290 0x55b0c1797fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64b6ffd640 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c1799290 0x55b0c1797fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c0d46a146fc2225 server_cookie=e99ad6afc2a19efc in_seq=0 out_seq=0 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f649e7fc640 10 client.15531 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f649e7fc640 1 -- 172.21.15.16:0/2223466277 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b0c1678b10 con 0x55b0c1799290 2024-09-15T22:38:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f649e7fc640 10 client.15531 handle_command_reply: tid=0 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 client.15531 shutdown 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c1719fc0 msgr2=0x55b0c1792eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c1719fc0 0x55b0c1792eb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64b00128a0 tx=0x7f64b0004a40 comp rx=0 tx=0).stop 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 shutdown_connections 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b0c1799290 0x55b0c1797fa0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b0c1719fc0 0x55b0c1792eb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b0c170b1f0 0x55b0c172d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f649e7fc640 0 client.15531 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 --2- 172.21.15.16:0/2223466277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b0c170a8d0 0x55b0c172d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 >> 172.21.15.16:0/2223466277 conn(0x55b0c1706f60 msgr2=0x55b0c1711a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 shutdown_connections 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 1 -- 172.21.15.16:0/2223466277 wait complete. 2024-09-15T22:38:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.028+0000 7f64bf683740 20 client.15531 trim_cache size 0 max 16384 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "af43fc90-73b1-11ef-bceb-c7b262605968", 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24500, 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 162.77866130999999, 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 26, 2024-09-15T22:38:52.037 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 90, 2024-09-15T22:38:52.038 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 89, 2024-09-15T22:38:52.038 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 173.62981596399999 2024-09-15T22:38:52.038 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:38:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f64a40a3440 msgr2=0x7f64a40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f64a40a3440 0x7f64a40a5900 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f64b00092f0 tx=0x7f64b0009280 comp rx=0 tx=0).stop 2024-09-15T22:38:52.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 msgr2=0x7f64b8198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b8198470 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f64a8006010 tx=0x7f64a80042f0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 shutdown_connections 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64b8106f10 0x7f64b819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f64a40a3440 0x7f64a40a5900 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f64b8102790 0x7f64b81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 --2- 172.21.15.16:0/828962616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64b8101de0 0x7f64b8198470 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 >> 172.21.15.16:0/828962616 conn(0x7f64b80fd8c0 msgr2=0x7f64b80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 shutdown_connections 2024-09-15T22:38:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:38:52.029+0000 7f64bd2aa640 1 -- 172.21.15.16:0/828962616 wait complete. 2024-09-15T22:38:52.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:52 smithi059 ceph-mon[34872]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:38:52.750 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:52 smithi016 ceph-mon[31492]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:38:52.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:52 smithi130 ceph-mon[35742]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:38:52.781 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-15T22:38:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:54 smithi130 ceph-mon[35742]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-15T22:38:54.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:54 smithi016 ceph-mon[31492]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-15T22:38:54.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:54 smithi059 ceph-mon[34872]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-15T22:38:56.746 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:56 smithi059 ceph-mon[34872]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:38:56.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:56 smithi130 ceph-mon[35742]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:38:56.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:56 smithi016 ceph-mon[31492]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:38:58.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:38:58 smithi130 ceph-mon[35742]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-15T22:38:58.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:38:58 smithi016 ceph-mon[31492]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-15T22:38:58.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:38:58 smithi059 ceph-mon[34872]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-15T22:39:00.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:00 smithi130 ceph-mon[35742]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:39:00.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:00 smithi016 ceph-mon[31492]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:39:00.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:00 smithi059 ceph-mon[34872]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:39:01.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:39:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:39:00.856+0000 7f4af5358640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:39:01.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:39:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:39:00.856+0000 7f4af5358640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:39:01.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:39:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:39:00.856+0000 7f4af5358640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:39:01.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:39:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:39:00.856+0000 7f4af5358640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:39:01.307 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:39:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:39:00.856+0000 7f4af5358640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:39:02.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:02 smithi130 ceph-mon[35742]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-15T22:39:02.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:02 smithi130 ceph-mon[35742]: mgrmap e25: x(active, since 36s), standbys: z, y 2024-09-15T22:39:02.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:02 smithi016 ceph-mon[31492]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-15T22:39:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:02 smithi016 ceph-mon[31492]: mgrmap e25: x(active, since 36s), standbys: z, y 2024-09-15T22:39:02.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:02 smithi059 ceph-mon[34872]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-15T22:39:02.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:02 smithi059 ceph-mon[34872]: mgrmap e25: x(active, since 36s), standbys: z, y 2024-09-15T22:39:04.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:04 smithi130 ceph-mon[35742]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:39:04.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:04 smithi016 ceph-mon[31492]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:39:04.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:04 smithi059 ceph-mon[34872]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:39:05.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:39:05.420 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 -- 172.21.15.16:0/3500165811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c101de0 msgr2=0x7ff08c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 --2- 172.21.15.16:0/3500165811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c101de0 0x7ff08c1021c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff088009900 tx=0x7ff08802a410 comp rx=0 tx=0).stop 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 -- 172.21.15.16:0/3500165811 shutdown_connections 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 --2- 172.21.15.16:0/3500165811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff08c106f10 0x7ff08c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 --2- 172.21.15.16:0/3500165811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08c102790 0x7ff08c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 --2- 172.21.15.16:0/3500165811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c101de0 0x7ff08c1021c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 -- 172.21.15.16:0/3500165811 >> 172.21.15.16:0/3500165811 conn(0x7ff08c0fd8c0 msgr2=0x7ff08c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 -- 172.21.15.16:0/3500165811 shutdown_connections 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 -- 172.21.15.16:0/3500165811 wait complete. 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 Processor -- start 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.941+0000 7ff093f06640 1 -- start start 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff093f06640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff08c101de0 0x7ff08c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff093f06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c102790 0x7ff08c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff093f06640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08c106f10 0x7ff08c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff093f06640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c111f70 con 0x7ff08c102790 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff093f06640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c111df0 con 0x7ff08c106f10 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff093f06640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1120f0 con 0x7ff08c101de0 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c102790 0x7ff08c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c102790 0x7ff08c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55290/0 (socket says 172.21.15.16:55290) 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 -- 172.21.15.16:0/96714581 learned_addr learned my addr 172.21.15.16:0/96714581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff091ca2640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff08c101de0 0x7ff08c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0924a3640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08c106f10 0x7ff08c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 -- 172.21.15.16:0/96714581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff08c101de0 msgr2=0x7ff08c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff08c101de0 0x7ff08c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 -- 172.21.15.16:0/96714581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08c106f10 msgr2=0x7ff08c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08c106f10 0x7ff08c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0914a1640 1 -- 172.21.15.16:0/96714581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c1a6870 con 0x7ff08c102790 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.942+0000 7ff0924a3640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08c106f10 0x7ff08c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.943+0000 7ff0914a1640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c102790 0x7ff08c074a30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c920 tx=0x7ff07c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.943+0000 7ff082ffd640 1 -- 172.21.15.16:0/96714581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c007c70 con 0x7ff08c102790 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.943+0000 7ff082ffd640 1 -- 172.21.15.16:0/96714581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff07c007e10 con 0x7ff08c102790 2024-09-15T22:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.943+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c1a6b60 con 0x7ff08c102790 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.943+0000 7ff082ffd640 1 -- 172.21.15.16:0/96714581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c018890 con 0x7ff08c102790 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.943+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a70f0 con 0x7ff08c102790 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.945+0000 7ff0962df740 1 Processor -- start 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0962df740 1 -- start start 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0962df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0962df740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d07c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0962df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 0x55c20d07ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0962df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c20cfd72e0 con 0x55c20d0714a0 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0962df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c20cf7ece0 con 0x55c20d07c680 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0962df740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c20cfd32a0 con 0x55c20d071dc0 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55294/0 (socket says 172.21.15.16:55294) 2024-09-15T22:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 -- 172.21.15.16:0/2659327698 learned_addr learned my addr 172.21.15.16:0/2659327698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff091ca2640 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d07c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff082ffd640 1 -- 172.21.15.16:0/96714581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7ff07c018a70 con 0x7ff08c102790 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 -- 172.21.15.16:0/2659327698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 msgr2=0x55c20d07c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d07c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 -- 172.21.15.16:0/2659327698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 msgr2=0x55c20d07ea70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 0x55c20d07ea70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff0924a3640 1 -- 172.21.15.16:0/2659327698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c20cf63440 con 0x55c20d0714a0 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.946+0000 7ff091ca2640 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d07c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.947+0000 7ff0924a3640 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d071880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff084009fd0 tx=0x7ff084010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cac74612967f829 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.947+0000 7ff077fff640 1 -- 172.21.15.16:0/2659327698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084011700 con 0x55c20d0714a0 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.947+0000 7ff077fff640 1 -- 172.21.15.16:0/2659327698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff084011d80 con 0x55c20d0714a0 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.947+0000 7ff077fff640 1 -- 172.21.15.16:0/2659327698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400e3c0 con 0x55c20d0714a0 2024-09-15T22:39:05.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.947+0000 7ff082ffd640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0780a3550 0x7ff0780a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.947+0000 7ff091ca2640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0780a3550 0x7ff0780a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff082ffd640 1 -- 172.21.15.16:0/96714581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff07c0d03f0 con 0x7ff08c102790 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 -- 172.21.15.16:0/2659327698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 msgr2=0x55c20d071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d071880 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff084009fd0 tx=0x7ff084010860 comp rx=0 tx=0).stop 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 -- 172.21.15.16:0/2659327698 shutdown_connections 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 0x55c20d07ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d07c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 --2- 172.21.15.16:0/2659327698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d071880 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 -- 172.21.15.16:0/2659327698 >> 172.21.15.16:0/2659327698 conn(0x55c20d06da30 msgr2=0x55c20d06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff091ca2640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0780a3550 0x7ff0780a5a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff088009230 tx=0x7ff0880058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 -- 172.21.15.16:0/2659327698 shutdown_connections 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 -- 172.21.15.16:0/2659327698 wait complete. 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.948+0000 7ff0962df740 1 -- start start 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 1 Processor -- start 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d07c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d079350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d079960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c20d07f250 con 0x55c20d07c680 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c20cf92050 con 0x55c20d071dc0 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c20d07f3d0 con 0x55c20d0714a0 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0962df740 10 client.0 _ll_register_callbacks cb 0x55c20cfd2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0924a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d079960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0924a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d079960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55306/0 (socket says 172.21.15.16:55306) 2024-09-15T22:39:05.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0924a3640 1 -- 172.21.15.16:0/3774862810 learned_addr learned my addr 172.21.15.16:0/3774862810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff091ca2640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d07c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.949+0000 7ff0914a1640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d079350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0924a3640 1 -- 172.21.15.16:0/3774862810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 msgr2=0x55c20d07c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0924a3640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d07c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0924a3640 1 -- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 msgr2=0x55c20d079350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0924a3640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d079350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0924a3640 1 -- 172.21.15.16:0/3774862810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c20cd483c0 con 0x55c20d07c680 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff091ca2640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d07c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0914a1640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d079350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0924a3640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d079960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff084010d60 tx=0x7ff084004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdea7e70e47f2fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0757fa640 1 -- 172.21.15.16:0/3774862810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084022070 con 0x55c20d07c680 2024-09-15T22:39:05.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0757fa640 1 -- 172.21.15.16:0/3774862810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff08401b840 con 0x55c20d07c680 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff084011d80 con 0x55c20d07c680 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0757fa640 1 -- 172.21.15.16:0/3774862810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff084031050 con 0x55c20d07c680 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0962df740 10 client.15549 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c20d08ace0 con 0x55c20d07c680 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.950+0000 7ff0757fa640 1 -- 172.21.15.16:0/3774862810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff084011d80 con 0x55c20d07c680 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff0757fa640 1 -- 172.21.15.16:0/3774862810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0840054c0 con 0x55c20d07c680 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff0962df740 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff0962df740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff0962df740 20 client.15549 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff0962df740 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d08e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff0962df740 4 client.15549 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c20cfe4850 con 0x55c20d08c1a0 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.951+0000 7ff091ca2640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d08e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.952+0000 7ff091ca2640 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d08e590 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabd6cbe296caa5 server_cookie=1e8d504924ef32bd in_seq=0 out_seq=0 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.952+0000 7ff0757fa640 10 client.15549 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.952+0000 7ff0757fa640 1 -- 172.21.15.16:0/3774862810 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c20cfe4850 con 0x55c20d08c1a0 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.952+0000 7ff0757fa640 10 client.15549 handle_command_reply: tid=0 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.952+0000 7ff0962df740 1 client.15549 shutdown 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 msgr2=0x55c20d079960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d079960 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff084010d60 tx=0x7ff084004f10 comp rx=0 tx=0).stop 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 shutdown_connections 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d08e590 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d079960 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0757fa640 0 client.15549 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d079350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 --2- 172.21.15.16:0/3774862810 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d07c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 >> 172.21.15.16:0/3774862810 conn(0x55c20d06da30 msgr2=0x55c20d073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 shutdown_connections 2024-09-15T22:39:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 1 -- 172.21.15.16:0/3774862810 wait complete. 2024-09-15T22:39:05.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.953+0000 7ff0962df740 20 client.15549 trim_cache size 0 max 16384 2024-09-15T22:39:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 Processor -- start 2024-09-15T22:39:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 -- start start 2024-09-15T22:39:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d07b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d0884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 0x55c20d093b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1b38f0 con 0x55c20d0714a0 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1b11c0 con 0x55c20d07c680 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff0962df740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1af010 con 0x55c20d071dc0 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff091ca2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d07b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff091ca2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d07b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55320/0 (socket says 172.21.15.16:55320) 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.986+0000 7ff091ca2640 1 -- 172.21.15.16:0/1128035656 learned_addr learned my addr 172.21.15.16:0/1128035656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff0914a1640 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d0884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff0924a3640 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 0x55c20d093b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff091ca2640 1 -- 172.21.15.16:0/1128035656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 msgr2=0x55c20d0884b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff091ca2640 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d0884b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff091ca2640 1 -- 172.21.15.16:0/1128035656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 msgr2=0x55c20d093b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff091ca2640 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 0x55c20d093b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff091ca2640 1 -- 172.21.15.16:0/1128035656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c20d08e900 con 0x55c20d0714a0 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff091ca2640 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d07b680 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff08802a930 tx=0x7ff08803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c42ba3410947862 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff0777fe640 1 -- 172.21.15.16:0/1128035656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088057070 con 0x55c20d0714a0 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff0777fe640 1 -- 172.21.15.16:0/1128035656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff088003ad0 con 0x55c20d0714a0 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.987+0000 7ff0777fe640 1 -- 172.21.15.16:0/1128035656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088035040 con 0x55c20d0714a0 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 -- 172.21.15.16:0/1128035656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 msgr2=0x55c20d07b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d07b680 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff08802a930 tx=0x7ff08803e000 comp rx=0 tx=0).stop 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 -- 172.21.15.16:0/1128035656 shutdown_connections 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d07c680 0x55c20d093b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d071dc0 0x55c20d0884b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 --2- 172.21.15.16:0/1128035656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d0714a0 0x55c20d07b680 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 -- 172.21.15.16:0/1128035656 >> 172.21.15.16:0/1128035656 conn(0x55c20d06da30 msgr2=0x55c20d073710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 -- 172.21.15.16:0/1128035656 shutdown_connections 2024-09-15T22:39:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 -- 172.21.15.16:0/1128035656 wait complete. 2024-09-15T22:39:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.988+0000 7ff0962df740 1 -- start start 2024-09-15T22:39:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 1 Processor -- start 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d084c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d07ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d080490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1af680 con 0x55c20d07c680 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1b0af0 con 0x55c20d071dc0 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c1afd50 con 0x55c20d0714a0 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0962df740 10 client.0 _ll_register_callbacks cb 0x7ff084009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0924a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d080490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0924a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d080490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55332/0 (socket says 172.21.15.16:55332) 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0924a3640 1 -- 172.21.15.16:0/1548101982 learned_addr learned my addr 172.21.15.16:0/1548101982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff091ca2640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d084c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.989+0000 7ff0914a1640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d07ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0924a3640 1 -- 172.21.15.16:0/1548101982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 msgr2=0x55c20d084c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0924a3640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d084c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0924a3640 1 -- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 msgr2=0x55c20d07ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0924a3640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d07ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0924a3640 1 -- 172.21.15.16:0/1548101982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c20d080b80 con 0x55c20d07c680 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0914a1640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d07ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0924a3640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d080490 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff084011010 tx=0x7ff08401be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab1f66448880a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0757fa640 1 -- 172.21.15.16:0/1548101982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084022040 con 0x55c20d07c680 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0757fa640 1 -- 172.21.15.16:0/1548101982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff08401b850 con 0x55c20d07c680 2024-09-15T22:39:05.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff088003ad0 con 0x55c20d07c680 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0757fa640 1 -- 172.21.15.16:0/1548101982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff084043050 con 0x55c20d07c680 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0962df740 10 client.15555 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.990+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c20d0f69b0 con 0x55c20d07c680 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0757fa640 1 -- 172.21.15.16:0/1548101982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff088003ad0 con 0x55c20d07c680 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0757fa640 1 -- 172.21.15.16:0/1548101982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0840118a0 con 0x55c20d07c680 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0962df740 10 client.15555 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0962df740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0962df740 20 client.15555 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0962df740 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d0f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0962df740 4 client.15555 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c20cfded60 con 0x55c20d08c1a0 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.991+0000 7ff091ca2640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d0f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff091ca2640 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d0f9040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63339135903807f7 server_cookie=13b69a3ba3b55d92 in_seq=0 out_seq=0 2024-09-15T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0757fa640 10 client.15555 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0757fa640 1 -- 172.21.15.16:0/1548101982 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c20cfded60 con 0x55c20d08c1a0 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0757fa640 10 client.15555 handle_command_reply: tid=0 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 client.15555 shutdown 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 msgr2=0x55c20d080490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d080490 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff084011010 tx=0x7ff08401be10 comp rx=0 tx=0).stop 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 shutdown_connections 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c20d08c1a0 0x55c20d0f9040 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c20d07c680 0x55c20d080490 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c20d071dc0 0x55c20d07ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 --2- 172.21.15.16:0/1548101982 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c20d0714a0 0x55c20d084c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 >> 172.21.15.16:0/1548101982 conn(0x55c20d06da30 msgr2=0x55c20d077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0757fa640 0 client.15555 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 shutdown_connections 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.992+0000 7ff0962df740 1 -- 172.21.15.16:0/1548101982 wait complete. 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff0962df740 20 client.15555 trim_cache size 0 max 16384 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:39:06.001 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:39:06.001 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:39:06.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0780a3550 msgr2=0x7ff0780a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0780a3550 0x7ff0780a5a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff088009230 tx=0x7ff0880058d0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c102790 msgr2=0x7ff08c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c102790 0x7ff08c074a30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00c920 tx=0x7ff07c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 shutdown_connections 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08c106f10 0x7ff08c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff08c102790 0x7ff08c074a30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0780a3550 0x7ff0780a5a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 --2- 172.21.15.16:0/96714581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff08c101de0 0x7ff08c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 >> 172.21.15.16:0/96714581 conn(0x7ff08c0fd8c0 msgr2=0x7ff08c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 shutdown_connections 2024-09-15T22:39:06.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:05.993+0000 7ff093f06640 1 -- 172.21.15.16:0/96714581 wait complete. 2024-09-15T22:39:06.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:06 smithi016 ceph-mon[31492]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:39:06.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:06 smithi130 ceph-mon[35742]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:39:06.906 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:39:06.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:39:06.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:06 smithi059 ceph-mon[34872]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:39:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:07.782 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-15T22:39:07.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 -- 172.21.15.16:0/205418455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 msgr2=0x7feb541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/205418455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541067e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7feb44009900 tx=0x7feb4402a410 comp rx=0 tx=0).stop 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 -- 172.21.15.16:0/205418455 shutdown_connections 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/205418455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb54106f10 0x7feb5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/205418455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541067e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/205418455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb54101de0 0x7feb541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 -- 172.21.15.16:0/205418455 >> 172.21.15.16:0/205418455 conn(0x7feb540fd8c0 msgr2=0x7feb540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 -- 172.21.15.16:0/205418455 shutdown_connections 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 -- 172.21.15.16:0/205418455 wait complete. 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 Processor -- start 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.861+0000 7feb5bb4f640 1 -- start start 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb5bb4f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb54101de0 0x7feb54198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb5bb4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb5bb4f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb54106f10 0x7feb5419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb5bb4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feb54111f70 con 0x7feb54102790 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb5bb4f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feb54111df0 con 0x7feb54106f10 2024-09-15T22:39:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb5bb4f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7feb541120f0 con 0x7feb54101de0 2024-09-15T22:39:08.644 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:39:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55344/0 (socket says 172.21.15.16:55344) 2024-09-15T22:39:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 -- 172.21.15.16:0/208422581 learned_addr learned my addr 172.21.15.16:0/208422581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb598eb640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb54101de0 0x7feb54198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb5a0ec640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb54106f10 0x7feb5419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 -- 172.21.15.16:0/208422581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb54101de0 msgr2=0x7feb54198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb54101de0 0x7feb54198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 -- 172.21.15.16:0/208422581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb54106f10 msgr2=0x7feb5419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb54106f10 0x7feb5419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.862+0000 7feb590ea640 1 -- 172.21.15.16:0/208422581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5419d440 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.863+0000 7feb590ea640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541989d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7feb44006010 tx=0x7feb440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.863+0000 7feb4affd640 1 -- 172.21.15.16:0/208422581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4403f030 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.863+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5419d6d0 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.863+0000 7feb4affd640 1 -- 172.21.15.16:0/208422581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb44003ce0 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.863+0000 7feb4affd640 1 -- 172.21.15.16:0/208422581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44035070 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.863+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb541a5010 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.865+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb541021c0 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.865+0000 7feb4affd640 1 -- 172.21.15.16:0/208422581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7feb44042030 con 0x7feb54102790 2024-09-15T22:39:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.866+0000 7feb4affd640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7feb400a34d0 0x7feb400a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.866+0000 7feb598eb640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7feb400a34d0 0x7feb400a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.867+0000 7feb4affd640 1 -- 172.21.15.16:0/208422581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7feb440f0790 con 0x7feb54102790 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.867+0000 7feb598eb640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7feb400a34d0 0x7feb400a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb50009bd0 tx=0x7feb50005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:07.872+0000 7feb4affd640 1 -- 172.21.15.16:0/208422581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb440bc700 con 0x7feb54102790 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.140+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb5410a150 con 0x7feb54102790 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.141+0000 7feb4affd640 1 -- 172.21.15.16:0/208422581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7feb440c0a00 con 0x7feb54102790 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7feb400a34d0 msgr2=0x7feb400a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7feb400a34d0 0x7feb400a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb50009bd0 tx=0x7feb50005f90 comp rx=0 tx=0).stop 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 msgr2=0x7feb541989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541989d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7feb44006010 tx=0x7feb440042f0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 shutdown_connections 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb54106f10 0x7feb5419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb54102790 0x7feb541989d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7feb400a34d0 0x7feb400a5990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 --2- 172.21.15.16:0/208422581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7feb54101de0 0x7feb54198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 >> 172.21.15.16:0/208422581 conn(0x7feb540fd8c0 msgr2=0x7feb540fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.143+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 shutdown_connections 2024-09-15T22:39:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:08.144+0000 7feb5bb4f640 1 -- 172.21.15.16:0/208422581 wait complete. 2024-09-15T22:39:08.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:08 smithi130 ceph-mon[35742]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:39:08.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:08 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/208422581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:08.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:08 smithi016 ceph-mon[31492]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:39:08.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:08 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/208422581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:08.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:08 smithi059 ceph-mon[34872]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:39:08.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:08 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/208422581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:10.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:39:10.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:10 smithi130 ceph-mon[35742]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-15T22:39:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:10.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:10 smithi016 ceph-mon[31492]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-15T22:39:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:10 smithi059 ceph-mon[34872]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.339+0000 7fa946b7a640 1 -- 172.21.15.16:0/4193592449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401082a0 msgr2=0x7fa94010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.339+0000 7fa946b7a640 1 --2- 172.21.15.16:0/4193592449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401082a0 0x7fa94010ec40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa938009cd0 tx=0x7fa93802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 -- 172.21.15.16:0/4193592449 shutdown_connections 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 --2- 172.21.15.16:0/4193592449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401082a0 0x7fa94010ec40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 --2- 172.21.15.16:0/4193592449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9401036e0 0x7fa940107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 --2- 172.21.15.16:0/4193592449 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa940102d30 0x7fa940103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 -- 172.21.15.16:0/4193592449 >> 172.21.15.16:0/4193592449 conn(0x7fa9400fc7d0 msgr2=0x7fa9400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 -- 172.21.15.16:0/4193592449 shutdown_connections 2024-09-15T22:39:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 -- 172.21.15.16:0/4193592449 wait complete. 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 Processor -- start 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 -- start start 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.340+0000 7fa946b7a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa940102d30 0x7fa9401985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa946b7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401036e0 0x7fa940198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa946b7a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9401082a0 0x7fa94019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa946b7a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa940112100 con 0x7fa9401036e0 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa946b7a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa940111f80 con 0x7fa940102d30 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa946b7a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa940112280 con 0x7fa9401082a0 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401036e0 0x7fa940198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401036e0 0x7fa940198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55360/0 (socket says 172.21.15.16:55360) 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 -- 172.21.15.16:0/1058830333 learned_addr learned my addr 172.21.15.16:0/1058830333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa945117640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9401082a0 0x7fa94019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa944916640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa940102d30 0x7fa9401985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 -- 172.21.15.16:0/1058830333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9401082a0 msgr2=0x7fa94019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9401082a0 0x7fa94019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 -- 172.21.15.16:0/1058830333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa940102d30 msgr2=0x7fa9401985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa940102d30 0x7fa9401985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 -- 172.21.15.16:0/1058830333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94019d580 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93ffff640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401036e0 0x7fa940198b10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c970 tx=0x7fa92c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93dffb640 1 -- 172.21.15.16:0/1058830333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c007bb0 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93dffb640 1 -- 172.21.15.16:0/1058830333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92c007d50 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa93dffb640 1 -- 172.21.15.16:0/1058830333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00f450 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.341+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94019d870 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.342+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9401a5150 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.343+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa940103110 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.344+0000 7fa93dffb640 1 -- 172.21.15.16:0/1058830333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7fa92c004f60 con 0x7fa9401036e0 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.345+0000 7fa93dffb640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9300a3550 0x7fa9300a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.345+0000 7fa93dffb640 1 -- 172.21.15.16:0/1058830333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa92c0d0000 con 0x7fa9401036e0 2024-09-15T22:39:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.345+0000 7fa944916640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9300a3550 0x7fa9300a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.345+0000 7fa944916640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9300a3550 0x7fa9300a5a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa934009890 tx=0x7fa934008040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.350+0000 7fa93dffb640 1 -- 172.21.15.16:0/1058830333 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa92c014030 con 0x7fa9401036e0 2024-09-15T22:39:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.583+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa94010a2b0 con 0x7fa9401036e0 2024-09-15T22:39:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.584+0000 7fa93dffb640 1 -- 172.21.15.16:0/1058830333 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fa92c09c430 con 0x7fa9401036e0 2024-09-15T22:39:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:39:11.585 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:39:11.586 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:39:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9300a3550 msgr2=0x7fa9300a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9300a3550 0x7fa9300a5a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa934009890 tx=0x7fa934008040 comp rx=0 tx=0).stop 2024-09-15T22:39:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401036e0 msgr2=0x7fa940198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401036e0 0x7fa940198b10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c970 tx=0x7fa92c00ce30 comp rx=0 tx=0).stop 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 shutdown_connections 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa9300a3550 0x7fa9300a5a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa9401082a0 0x7fa94019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9401036e0 0x7fa940198b10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 --2- 172.21.15.16:0/1058830333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa940102d30 0x7fa9401985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 >> 172.21.15.16:0/1058830333 conn(0x7fa9400fc7d0 msgr2=0x7fa940105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 shutdown_connections 2024-09-15T22:39:11.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:11.586+0000 7fa946b7a640 1 -- 172.21.15.16:0/1058830333 wait complete. 2024-09-15T22:39:12.440 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-15T22:39:12.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-15T22:39:12.687 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:12 smithi016 ceph-mon[31492]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:39:12.688 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:12 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1058830333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.685+0000 7fb42bf21640 1 -- 172.21.15.16:0/2933775863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb424102d30 msgr2=0x7fb424103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.685+0000 7fb42bf21640 1 --2- 172.21.15.16:0/2933775863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb424102d30 0x7fb424103110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb420009930 tx=0x7fb42002a420 comp rx=0 tx=0).stop 2024-09-15T22:39:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 -- 172.21.15.16:0/2933775863 shutdown_connections 2024-09-15T22:39:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 --2- 172.21.15.16:0/2933775863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4241082a0 0x7fb42410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 --2- 172.21.15.16:0/2933775863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4241036e0 0x7fb424107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 --2- 172.21.15.16:0/2933775863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb424102d30 0x7fb424103110 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 -- 172.21.15.16:0/2933775863 >> 172.21.15.16:0/2933775863 conn(0x7fb4240fc7d0 msgr2=0x7fb4240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 -- 172.21.15.16:0/2933775863 shutdown_connections 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 -- 172.21.15.16:0/2933775863 wait complete. 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 Processor -- start 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.686+0000 7fb42bf21640 1 -- start start 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb42bf21640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb424102d30 0x7fb4241a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb42bf21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4241036e0 0x7fb4241a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb42bf21640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4241082a0 0x7fb42419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb42bf21640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241120a0 con 0x7fb4241036e0 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb42bf21640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb424111f20 con 0x7fb424102d30 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb42bf21640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb424112220 con 0x7fb4241082a0 2024-09-15T22:39:12.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb429cbd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb424102d30 0x7fb4241a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb4294bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4241036e0 0x7fb4241a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb429cbd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb424102d30 0x7fb4241a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:57884/0 (socket says 172.21.15.16:57884) 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb429cbd640 1 -- 172.21.15.16:0/136819772 learned_addr learned my addr 172.21.15.16:0/136819772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.687+0000 7fb42a4be640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4241082a0 0x7fb42419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb4294bc640 1 -- 172.21.15.16:0/136819772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4241082a0 msgr2=0x7fb42419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb4294bc640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4241082a0 0x7fb42419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb4294bc640 1 -- 172.21.15.16:0/136819772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb424102d30 msgr2=0x7fb4241a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb4294bc640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb424102d30 0x7fb4241a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb4294bc640 1 -- 172.21.15.16:0/136819772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42419ad40 con 0x7fb4241036e0 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb429cbd640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb424102d30 0x7fb4241a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb4294bc640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4241036e0 0x7fb4241a0990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb42406a370 tx=0x7fb41400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb41affd640 1 -- 172.21.15.16:0/136819772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414017070 con 0x7fb4241036e0 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb41affd640 1 -- 172.21.15.16:0/136819772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4140099a0 con 0x7fb4241036e0 2024-09-15T22:39:12.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42419b030 con 0x7fb4241036e0 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb41affd640 1 -- 172.21.15.16:0/136819772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4140047a0 con 0x7fb4241036e0 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.688+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4241a7320 con 0x7fb4241036e0 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.691+0000 7fb41affd640 1 -- 172.21.15.16:0/136819772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7fb414009b50 con 0x7fb4241036e0 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.691+0000 7fb42e2fa740 1 Processor -- start 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.691+0000 7fb42e2fa740 1 -- start start 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53b9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9c5d0 0x564f53b9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53ba7370 0x564f53ba9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f53b022f0 con 0x564f53b9db40 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f53aa9cf0 con 0x564f53b9c5d0 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564f53afe2b0 con 0x564f53ba7370 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53b9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53b9c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49218/0 (socket says 172.21.15.16:49218) 2024-09-15T22:39:12.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 -- 172.21.15.16:0/369494781 learned_addr learned my addr 172.21.15.16:0/369494781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb4294bc640 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53ba7370 0x564f53ba9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb429cbd640 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9c5d0 0x564f53b9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 -- 172.21.15.16:0/369494781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53ba7370 msgr2=0x564f53ba9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53ba7370 0x564f53ba9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 -- 172.21.15.16:0/369494781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9c5d0 msgr2=0x564f53b9ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9c5d0 0x564f53b9ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 -- 172.21.15.16:0/369494781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f53a8e3a0 con 0x564f53b9db40 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb429cbd640 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9c5d0 0x564f53b9ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb4294bc640 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53ba7370 0x564f53ba9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.692+0000 7fb42a4be640 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53b9c090 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009f50 tx=0x7fb41c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b189fc6eee161f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:12.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.693+0000 7fb41affd640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4100a34d0 0x7fb4100a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.693+0000 7fb40ffff640 1 -- 172.21.15.16:0/369494781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c004d20 con 0x564f53b9db40 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.693+0000 7fb40ffff640 1 -- 172.21.15.16:0/369494781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41c004050 con 0x564f53b9db40 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.693+0000 7fb40ffff640 1 -- 172.21.15.16:0/369494781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c005280 con 0x564f53b9db40 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.693+0000 7fb429cbd640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4100a34d0 0x7fb4100a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.693+0000 7fb41affd640 1 -- 172.21.15.16:0/136819772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb4140cfef0 con 0x7fb4241036e0 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 -- 172.21.15.16:0/369494781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 msgr2=0x564f53b9c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb429cbd640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4100a34d0 0x7fb4100a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x564f53b99ac0 tx=0x7fb420009600 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53b9c090 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009f50 tx=0x7fb41c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 -- 172.21.15.16:0/369494781 shutdown_connections 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53ba7370 0x564f53ba9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9c5d0 0x564f53b9ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/369494781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53b9c090 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 -- 172.21.15.16:0/369494781 >> 172.21.15.16:0/369494781 conn(0x564f53b65310 msgr2=0x564f53b98c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 -- 172.21.15.16:0/369494781 shutdown_connections 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 -- 172.21.15.16:0/369494781 wait complete. 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 -- start start 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.694+0000 7fb42e2fa740 1 Processor -- start 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53ba3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53ba4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f53baa070 con 0x564f53ba7370 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f53abd060 con 0x564f53b9db40 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564f53baa1f0 con 0x564f53b9c5d0 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42e2fa740 10 client.0 _ll_register_callbacks cb 0x564f538108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42a4be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53ba4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42a4be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53ba4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49224/0 (socket says 172.21.15.16:49224) 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb42a4be640 1 -- 172.21.15.16:0/3249693184 learned_addr learned my addr 172.21.15.16:0/3249693184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.695+0000 7fb4294bc640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53ba3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb429cbd640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb42a4be640 1 -- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 msgr2=0x564f53ba6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb42a4be640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb42a4be640 1 -- 172.21.15.16:0/3249693184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 msgr2=0x564f53ba3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb42a4be640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53ba3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb42a4be640 1 -- 172.21.15.16:0/3249693184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f538733c0 con 0x564f53ba7370 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb4294bc640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53ba3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb42a4be640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53ba4530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009250 tx=0x7fb41c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e712f4297fa3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:12.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb40d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb40d7fa640 1 -- 172.21.15.16:0/3249693184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c029070 con 0x564f53ba7370 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb40d7fa640 1 -- 172.21.15.16:0/3249693184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb41c00f480 con 0x564f53ba7370 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.696+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb41c004050 con 0x564f53ba7370 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb40d7fa640 1 -- 172.21.15.16:0/3249693184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb41c035050 con 0x564f53ba7370 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 10 client.15579 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f53bb5a10 con 0x564f53ba7370 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb40d7fa640 1 -- 172.21.15.16:0/3249693184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41c004050 con 0x564f53ba7370 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb40d7fa640 1 -- 172.21.15.16:0/3249693184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb41c019630 con 0x564f53ba7370 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 10 client.15579 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 10 client.15579 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 20 client.15579 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53bb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 4 client.15579 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f53b0f860 con 0x564f53bb6d60 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.697+0000 7fb429cbd640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53bb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.698+0000 7fb40d7fa640 10 client.15579 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.698+0000 7fb429cbd640 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53bb9150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea9ecd3b14f0054 server_cookie=9e5581e206309f48 in_seq=0 out_seq=0 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb40d7fa640 1 -- 172.21.15.16:0/3249693184 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f53b0f860 con 0x564f53bb6d60 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb40d7fa640 10 client.15579 handle_command_reply: tid=0 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 client.15579 shutdown 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 msgr2=0x564f53ba4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53ba4530 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009250 tx=0x7fb41c009d20 comp rx=0 tx=0).stop 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 shutdown_connections 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53bb9150 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb40d7fa640 0 client.15579 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53ba4530 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53ba3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/3249693184 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 >> 172.21.15.16:0/3249693184 conn(0x564f53b65310 msgr2=0x564f53b9e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 shutdown_connections 2024-09-15T22:39:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 1 -- 172.21.15.16:0/3249693184 wait complete. 2024-09-15T22:39:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.699+0000 7fb42e2fa740 20 client.15579 trim_cache size 0 max 16384 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 Processor -- start 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 -- start start 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53b9f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53b9f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241b3b00 con 0x564f53ba7370 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241b13d0 con 0x564f53b9db40 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241af220 con 0x564f53b9c5d0 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42a4be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53b9f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42a4be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53b9f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49228/0 (socket says 172.21.15.16:49228) 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb42a4be640 1 -- 172.21.15.16:0/1749551101 learned_addr learned my addr 172.21.15.16:0/1749551101 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:12.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb429cbd640 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.715+0000 7fb4294bc640 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53b9f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb42a4be640 1 -- 172.21.15.16:0/1749551101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 msgr2=0x564f53ba5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb42a4be640 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb42a4be640 1 -- 172.21.15.16:0/1749551101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 msgr2=0x564f53b9f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb42a4be640 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53b9f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb42a4be640 1 -- 172.21.15.16:0/1749551101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f53b9fba0 con 0x564f53ba7370 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb42a4be640 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53b9f660 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb41c011910 tx=0x7fb41c004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cba9ff84dd3a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb40f7fe640 1 -- 172.21.15.16:0/1749551101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c029070 con 0x564f53ba7370 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb40f7fe640 1 -- 172.21.15.16:0/1749551101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb41c003c40 con 0x564f53ba7370 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.716+0000 7fb40f7fe640 1 -- 172.21.15.16:0/1749551101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c022ae0 con 0x564f53ba7370 2024-09-15T22:39:12.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.717+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1749551101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 msgr2=0x564f53b9f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.717+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53b9f660 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb41c011910 tx=0x7fb41c004960 comp rx=0 tx=0).stop 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.717+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1749551101 shutdown_connections 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.717+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53ba7370 0x564f53b9f660 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.717+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53b9db40 0x564f53b9f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.717+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1749551101 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53ba5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.717+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1749551101 >> 172.21.15.16:0/1749551101 conn(0x564f53b65310 msgr2=0x564f53ba5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.718+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1749551101 shutdown_connections 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.718+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1749551101 wait complete. 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.718+0000 7fb42e2fa740 1 -- start start 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.718+0000 7fb42e2fa740 1 Processor -- start 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53bab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53baba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb42e2fa740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53ba7370 0x564f53babfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241af890 con 0x564f53b9db40 2024-09-15T22:39:12.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb4294bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53baba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb42e2fa740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241b0d00 con 0x564f53ba7370 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb4294bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53baba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49232/0 (socket says 172.21.15.16:49232) 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb4294bc640 1 -- 172.21.15.16:0/1791319604 learned_addr learned my addr 172.21.15.16:0/1791319604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb4241aff60 con 0x564f53b9c5d0 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb42a4be640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53ba7370 0x564f53babfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.719+0000 7fb429cbd640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53bab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb4294bc640 1 -- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 msgr2=0x564f53bab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb4294bc640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53bab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb4294bc640 1 -- 172.21.15.16:0/1791319604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53ba7370 msgr2=0x564f53babfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb4294bc640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53ba7370 0x564f53babfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb4294bc640 1 -- 172.21.15.16:0/1791319604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f53c20bd0 con 0x564f53b9db40 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb42a4be640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53ba7370 0x564f53babfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb4294bc640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53baba30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb414098080 tx=0x7fb41409b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6cd960f4b64888 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb40d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:12.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb42e2fa740 10 client.0 _ll_register_callbacks cb 0x7fb41c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb41c003c40 con 0x564f53b9db40 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb40d7fa640 1 -- 172.21.15.16:0/1791319604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41409ffd0 con 0x564f53b9db40 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb40d7fa640 1 -- 172.21.15.16:0/1791319604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb4140cd440 con 0x564f53b9db40 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb40d7fa640 1 -- 172.21.15.16:0/1791319604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4140cd6a0 con 0x564f53b9db40 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb42e2fa740 10 client.15585 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.720+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=26}) -- 0x564f53c21dc0 con 0x564f53b9db40 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.721+0000 7fb40d7fa640 1 -- 172.21.15.16:0/1791319604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb4140860b0 con 0x564f53b9db40 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.721+0000 7fb42e2fa740 10 client.15585 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.721+0000 7fb42e2fa740 10 client.15585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.721+0000 7fb42e2fa740 20 client.15585 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.721+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53c24340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.721+0000 7fb42e2fa740 4 client.15585 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.721+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f53b09d70 con 0x564f53bb6d60 2024-09-15T22:39:12.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.722+0000 7fb429cbd640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53c24340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.722+0000 7fb429cbd640 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53c24340 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9825ee976410e5e7 server_cookie=cc7322ff0433fcea in_seq=0 out_seq=0 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.722+0000 7fb40d7fa640 10 client.15585 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.722+0000 7fb40d7fa640 1 -- 172.21.15.16:0/1791319604 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f53b09d70 con 0x564f53bb6d60 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.722+0000 7fb40d7fa640 10 client.15585 handle_command_reply: tid=0 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.722+0000 7fb42e2fa740 1 client.15585 shutdown 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 msgr2=0x564f53baba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53baba30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb414098080 tx=0x7fb41409b820 comp rx=0 tx=0).stop 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 shutdown_connections 2024-09-15T22:39:12.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564f53bb6d60 0x564f53c24340 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f53ba7370 0x564f53babfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb40d7fa640 0 client.15585 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f53b9db40 0x564f53baba30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 --2- 172.21.15.16:0/1791319604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f53b9c5d0 0x564f53bab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 >> 172.21.15.16:0/1791319604 conn(0x564f53b65310 msgr2=0x564f53ba6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 shutdown_connections 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.723+0000 7fb42e2fa740 1 -- 172.21.15.16:0/1791319604 wait complete. 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42e2fa740 20 client.15585 trim_cache size 0 max 16384 2024-09-15T22:39:12.735 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4100a34d0 msgr2=0x7fb4100a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4100a34d0 0x7fb4100a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x564f53b99ac0 tx=0x7fb420009600 comp rx=0 tx=0).stop 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4241036e0 msgr2=0x7fb4241a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4241036e0 0x7fb4241a0990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb42406a370 tx=0x7fb41400dbb0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 shutdown_connections 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4100a34d0 0x7fb4100a5990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4241082a0 0x7fb42419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.724+0000 7fb42bf21640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4241036e0 0x7fb4241a0990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.725+0000 7fb42bf21640 1 --2- 172.21.15.16:0/136819772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb424102d30 0x7fb4241a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:12.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.725+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 >> 172.21.15.16:0/136819772 conn(0x7fb4240fc7d0 msgr2=0x7fb42410cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:12.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.725+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 shutdown_connections 2024-09-15T22:39:12.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:12.725+0000 7fb42bf21640 1 -- 172.21.15.16:0/136819772 wait complete. 2024-09-15T22:39:12.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:12 smithi130 ceph-mon[35742]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:39:12.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:12 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1058830333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:12 smithi059 ceph-mon[34872]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:39:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:12 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1058830333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:13.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:39:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 -- 172.21.15.16:0/4057016514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0102ef0 msgr2=0x7fe2a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 --2- 172.21.15.16:0/4057016514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0102ef0 0x7fe2a010da40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe298009ca0 tx=0x7fe29802a950 comp rx=0 tx=0).stop 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 -- 172.21.15.16:0/4057016514 shutdown_connections 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 --2- 172.21.15.16:0/4057016514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0102ef0 0x7fe2a010da40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 --2- 172.21.15.16:0/4057016514 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2a0102570 0x7fe2a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 --2- 172.21.15.16:0/4057016514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2a0100e20 0x7fe2a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 -- 172.21.15.16:0/4057016514 >> 172.21.15.16:0/4057016514 conn(0x7fe2a00fc7d0 msgr2=0x7fe2a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 -- 172.21.15.16:0/4057016514 shutdown_connections 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.025+0000 7fe2a526c640 1 -- 172.21.15.16:0/4057016514 wait complete. 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.026+0000 7fe2a526c640 1 Processor -- start 2024-09-15T22:39:14.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.026+0000 7fe2a526c640 1 -- start start 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe2a526c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0100e20 0x7fe2a01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe2a526c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2a0102570 0x7fe2a0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe2a526c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2a0102ef0 0x7fe2a01a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe2a526c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a0110f80 con 0x7fe2a0100e20 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe2a526c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a0110e00 con 0x7fe2a0102570 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe2a526c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a0111100 con 0x7fe2a0102ef0 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0100e20 0x7fe2a01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0100e20 0x7fe2a01992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49250/0 (socket says 172.21.15.16:49250) 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 -- 172.21.15.16:0/2028780564 learned_addr learned my addr 172.21.15.16:0/2028780564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe297fff640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2a0102570 0x7fe2a0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2a0102ef0 0x7fe2a01a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 -- 172.21.15.16:0/2028780564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2a0102ef0 msgr2=0x7fe2a01a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2a0102ef0 0x7fe2a01a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 -- 172.21.15.16:0/2028780564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2a0102570 msgr2=0x7fe2a0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2a0102570 0x7fe2a0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 -- 172.21.15.16:0/2028780564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a01a4890 con 0x7fe2a0100e20 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29effd640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0100e20 0x7fe2a01992d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe29000e9f0 tx=0x7fe29000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.027+0000 7fe29cff9640 1 -- 172.21.15.16:0/2028780564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29000cd80 con 0x7fe2a0100e20 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.028+0000 7fe29cff9640 1 -- 172.21.15.16:0/2028780564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe290010910 con 0x7fe2a0100e20 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.028+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a01a4a60 con 0x7fe2a0100e20 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.028+0000 7fe29cff9640 1 -- 172.21.15.16:0/2028780564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290018990 con 0x7fe2a0100e20 2024-09-15T22:39:14.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.028+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a01a4fa0 con 0x7fe2a0100e20 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.030+0000 7fe29cff9640 1 -- 172.21.15.16:0/2028780564 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7fe290010ac0 con 0x7fe2a0100e20 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.031+0000 7fe2a7645740 1 Processor -- start 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.031+0000 7fe2a7645740 1 -- start start 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870043b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870050d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe2a7645740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56086ffa92a0 con 0x560870044080 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe2a7645740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56086ff50b10 con 0x56087004e940 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe2a7645740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56086ffa5260 con 0x560870043760 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe29effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe29f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870043b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe29f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870043b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:39944/0 (socket says 172.21.15.16:39944) 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe29f7fe640 1 -- 172.21.15.16:0/3282592179 learned_addr learned my addr 172.21.15.16:0/3282592179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe29cff9640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe28c0a3550 0x7fe28c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.032+0000 7fe297fff640 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870050d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29f7fe640 1 -- 172.21.15.16:0/3282592179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 msgr2=0x560870050d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870050d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29f7fe640 1 -- 172.21.15.16:0/3282592179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 msgr2=0x56087004e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29cff9640 1 -- 172.21.15.16:0/2028780564 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe2900d05a0 con 0x7fe2a0100e20 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29f7fe640 1 -- 172.21.15.16:0/3282592179 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086fd1ab90 con 0x560870043760 2024-09-15T22:39:14.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29effd640 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe28c0a3550 0x7fe28c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.033+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870043b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe298035000 tx=0x7fe298005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94974d22e0379914 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.034+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe28c0a3550 0x7fe28c0a5a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe29804d000 tx=0x7fe29804a000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.034+0000 7fe2957fa640 1 -- 172.21.15.16:0/3282592179 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298034040 con 0x560870043760 2024-09-15T22:39:14.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.034+0000 7fe2957fa640 1 -- 172.21.15.16:0/3282592179 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe298046950 con 0x560870043760 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.034+0000 7fe2957fa640 1 -- 172.21.15.16:0/3282592179 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298030a00 con 0x560870043760 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 -- 172.21.15.16:0/3282592179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 msgr2=0x560870043b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870043b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe298035000 tx=0x7fe298005ef0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 -- 172.21.15.16:0/3282592179 shutdown_connections 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870050d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 --2- 172.21.15.16:0/3282592179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870043b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 -- 172.21.15.16:0/3282592179 >> 172.21.15.16:0/3282592179 conn(0x56087003fcf0 msgr2=0x5608700400d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.035+0000 7fe2a7645740 1 -- 172.21.15.16:0/3282592179 shutdown_connections 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.036+0000 7fe2a7645740 1 -- 172.21.15.16:0/3282592179 wait complete. 2024-09-15T22:39:14.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.036+0000 7fe2a7645740 1 -- start start 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 1 Processor -- start 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x56087004b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560870044080 0x56087004bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56087004e940 0x56087004c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560870051510 con 0x560870043760 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56086ff316e0 con 0x560870044080 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560870051690 con 0x56087004e940 2024-09-15T22:39:14.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe2a7645740 10 client.0 _ll_register_callbacks cb 0x56086ff55a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe297fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560870044080 0x56087004bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe297fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560870044080 0x56087004bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:57952/0 (socket says 172.21.15.16:57952) 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe297fff640 1 -- 172.21.15.16:0/2202824090 learned_addr learned my addr 172.21.15.16:0/2202824090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56087004e940 0x56087004c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.037+0000 7fe29effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x56087004b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe297fff640 1 -- 172.21.15.16:0/2202824090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56087004e940 msgr2=0x56087004c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe297fff640 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56087004e940 0x56087004c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe297fff640 1 -- 172.21.15.16:0/2202824090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 msgr2=0x56087004b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe297fff640 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x56087004b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe297fff640 1 -- 172.21.15.16:0/2202824090 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087005aa00 con 0x560870044080 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe29effd640 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x56087004b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe297fff640 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560870044080 0x56087004bd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe288012700 tx=0x7fe288012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3698cf8bfa615dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.038+0000 7fe2867fc640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:39:14.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.039+0000 7fe2867fc640 1 -- 172.21.15.16:0/2202824090 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28800f8c0 con 0x560870044080 2024-09-15T22:39:14.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.039+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe298046950 con 0x560870044080 2024-09-15T22:39:14.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.039+0000 7fe2867fc640 1 -- 172.21.15.16:0/2202824090 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe28800fa60 con 0x560870044080 2024-09-15T22:39:14.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.039+0000 7fe2867fc640 1 -- 172.21.15.16:0/2202824090 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe288025050 con 0x560870044080 2024-09-15T22:39:14.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.039+0000 7fe2a7645740 10 client.25244 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:14.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.039+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56087005d420 con 0x560870044080 2024-09-15T22:39:14.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.039+0000 7fe2867fc640 1 -- 172.21.15.16:0/2202824090 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe298046950 con 0x560870044080 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe2867fc640 1 -- 172.21.15.16:0/2202824090 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe288010430 con 0x560870044080 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe2a7645740 10 client.25244 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe2a7645740 10 client.25244 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe2a7645740 20 client.25244 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56087005ec60 0x560870061100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe2a7645740 4 client.25244 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56086ffb69d0 con 0x56087005ec60 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.040+0000 7fe29effd640 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56087005ec60 0x560870061100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.041+0000 7fe29effd640 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56087005ec60 0x560870061100 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32ec12e605e9e004 server_cookie=b5324a331c8452cb in_seq=0 out_seq=0 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.041+0000 7fe2867fc640 10 client.25244 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.041+0000 7fe2867fc640 1 -- 172.21.15.16:0/2202824090 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56086ffb69d0 con 0x56087005ec60 2024-09-15T22:39:14.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2867fc640 10 client.25244 handle_command_reply: tid=0 2024-09-15T22:39:14.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 client.25244 shutdown 2024-09-15T22:39:14.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560870044080 msgr2=0x56087004bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560870044080 0x56087004bd60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe288012700 tx=0x7fe288012bc0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 shutdown_connections 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56087005ec60 0x560870061100 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56087004e940 0x56087004c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560870044080 0x56087004bd60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2202824090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x56087004b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 >> 172.21.15.16:0/2202824090 conn(0x56087003fcf0 msgr2=0x56087003ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2867fc640 0 client.25244 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 shutdown_connections 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.042+0000 7fe2a7645740 1 -- 172.21.15.16:0/2202824090 wait complete. 2024-09-15T22:39:14.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.043+0000 7fe2a7645740 20 client.25244 trim_cache size 0 max 16384 2024-09-15T22:39:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.061+0000 7fe2a7645740 1 Processor -- start 2024-09-15T22:39:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.061+0000 7fe2a7645740 1 -- start start 2024-09-15T22:39:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870046210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe2a7645740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01b1850 con 0x560870044080 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe2a7645740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01af120 con 0x56087004e940 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe2a7645740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01acf70 con 0x560870043760 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49288/0 (socket says 172.21.15.16:49288) 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 -- 172.21.15.16:0/1470932594 learned_addr learned my addr 172.21.15.16:0/1470932594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe29effd640 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870046210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 -- 172.21.15.16:0/1470932594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 msgr2=0x560870046210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870046210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 -- 172.21.15.16:0/1470932594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 msgr2=0x560870066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe297fff640 1 -- 172.21.15.16:0/1470932594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608700407b0 con 0x560870044080 2024-09-15T22:39:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.062+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.063+0000 7fe297fff640 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004a0f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe2880109c0 tx=0x7fe288001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6292096f39730a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.063+0000 7fe294ff9640 1 -- 172.21.15.16:0/1470932594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2880104f0 con 0x560870044080 2024-09-15T22:39:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.063+0000 7fe294ff9640 1 -- 172.21.15.16:0/1470932594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2880050b0 con 0x560870044080 2024-09-15T22:39:14.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.063+0000 7fe294ff9640 1 -- 172.21.15.16:0/1470932594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28800f9f0 con 0x560870044080 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 -- 172.21.15.16:0/1470932594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 msgr2=0x56087004a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004a0f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe2880109c0 tx=0x7fe288001e50 comp rx=0 tx=0).stop 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 -- 172.21.15.16:0/1470932594 shutdown_connections 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870044080 0x56087004a0f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 --2- 172.21.15.16:0/1470932594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870043760 0x560870046210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 -- 172.21.15.16:0/1470932594 >> 172.21.15.16:0/1470932594 conn(0x56087003fcf0 msgr2=0x56087004d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:14.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 -- 172.21.15.16:0/1470932594 shutdown_connections 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 -- 172.21.15.16:0/1470932594 wait complete. 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 -- start start 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.064+0000 7fe2a7645740 1 Processor -- start 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x5608700571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870044080 0x5608700524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe2a7645740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870052c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe2a7645740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01ad5e0 con 0x560870043760 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe2a7645740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01aea50 con 0x56087004e940 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe2a7645740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a01adcb0 con 0x560870044080 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe2a7645740 10 client.0 _ll_register_callbacks cb 0x7fe288009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x5608700571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x5608700571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49296/0 (socket says 172.21.15.16:49296) 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 -- 172.21.15.16:0/2802224476 learned_addr learned my addr 172.21.15.16:0/2802224476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:14.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe297fff640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870044080 0x5608700524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870052c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 -- 172.21.15.16:0/2802224476 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870044080 msgr2=0x5608700524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870044080 0x5608700524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 -- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 msgr2=0x560870052c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870052c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29effd640 1 -- 172.21.15.16:0/2802224476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087004a7d0 con 0x560870043760 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.065+0000 7fe29f7fe640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870052c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe29effd640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x5608700571b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe29009c080 tx=0x7fe290005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c5bda5d3382ba6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2867fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2867fc640 1 -- 172.21.15.16:0/2802224476 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2900a4f40 con 0x560870043760 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2867fc640 1 -- 172.21.15.16:0/2802224476 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe2900980d0 con 0x560870043760 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2880050b0 con 0x560870043760 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2867fc640 1 -- 172.21.15.16:0/2802224476 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2900a4ab0 con 0x560870043760 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2a7645740 10 client.15603 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:14.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608700c99e0 con 0x560870043760 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2867fc640 1 -- 172.21.15.16:0/2802224476 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2900a4c50 con 0x560870043760 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.066+0000 7fe2867fc640 1 -- 172.21.15.16:0/2802224476 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe2900cdb80 con 0x560870043760 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.067+0000 7fe2a7645740 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.067+0000 7fe2a7645740 10 client.15603 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.067+0000 7fe2a7645740 20 client.15603 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.067+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5608700ca120 0x5608700cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.067+0000 7fe2a7645740 4 client.15603 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.067+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x7fe2880050b0 con 0x5608700ca120 2024-09-15T22:39:14.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.067+0000 7fe297fff640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5608700ca120 0x5608700cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.068+0000 7fe2867fc640 10 client.15603 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:14.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.068+0000 7fe297fff640 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5608700ca120 0x5608700cc510 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71d40eabeacffeb9 server_cookie=17b896dd508f4e83 in_seq=0 out_seq=0 2024-09-15T22:39:14.075 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.079+0000 7fe2867fc640 1 -- 172.21.15.16:0/2802224476 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7fe2880050b0 con 0x5608700ca120 2024-09-15T22:39:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.079+0000 7fe2867fc640 10 client.15603 handle_command_reply: tid=0 2024-09-15T22:39:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.079+0000 7fe2a7645740 1 client.15603 shutdown 2024-09-15T22:39:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.079+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 msgr2=0x5608700571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.079+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x5608700571b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe29009c080 tx=0x7fe290005f10 comp rx=0 tx=0).stop 2024-09-15T22:39:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.079+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 shutdown_connections 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.079+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5608700ca120 0x5608700cc510 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56087004e940 0x560870052c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2867fc640 0 client.15603 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560870044080 0x5608700524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2a7645740 1 --2- 172.21.15.16:0/2802224476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560870043760 0x5608700571b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 >> 172.21.15.16:0/2802224476 conn(0x56087003fcf0 msgr2=0x56087004b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 shutdown_connections 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2a7645740 1 -- 172.21.15.16:0/2802224476 wait complete. 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.080+0000 7fe2a7645740 20 client.15603 trim_cache size 0 max 16384 2024-09-15T22:39:14.082 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 26, 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 2, 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "1a891f29": { 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 2, 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-15T22:39:14.083 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.2, 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "QUIESCED", 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:14.084 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:14.085 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:14.085 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:39:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe28c0a3550 msgr2=0x7fe28c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe28c0a3550 0x7fe28c0a5a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe29804d000 tx=0x7fe29804a000 comp rx=0 tx=0).stop 2024-09-15T22:39:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0100e20 msgr2=0x7fe2a01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0100e20 0x7fe2a01992d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe29000e9f0 tx=0x7fe29000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 shutdown_connections 2024-09-15T22:39:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe28c0a3550 0x7fe28c0a5a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2a0102ef0 0x7fe2a01a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2a0102570 0x7fe2a0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 --2- 172.21.15.16:0/2028780564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2a0100e20 0x7fe2a01992d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 >> 172.21.15.16:0/2028780564 conn(0x7fe2a00fc7d0 msgr2=0x7fe2a0106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 shutdown_connections 2024-09-15T22:39:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:14.081+0000 7fe2a526c640 1 -- 172.21.15.16:0/2028780564 wait complete. 2024-09-15T22:39:14.680 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:14 smithi016 ceph-mon[31492]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-15T22:39:14.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:14 smithi130 ceph-mon[35742]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-15T22:39:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:14 smithi059 ceph-mon[34872]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-15T22:39:15.122 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1a891f29', quiesce took 7.3 seconds. Will release after: 22.9 2024-09-15T22:39:15.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.285+0000 7f03500e1640 1 -- 172.21.15.16:0/2977535448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481082a0 msgr2=0x7f034810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.285+0000 7f03500e1640 1 --2- 172.21.15.16:0/2977535448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481082a0 0x7f034810ec40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0340009cd0 tx=0x7f034002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 -- 172.21.15.16:0/2977535448 shutdown_connections 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 --2- 172.21.15.16:0/2977535448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481082a0 0x7f034810ec40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 --2- 172.21.15.16:0/2977535448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f03481036e0 0x7f0348107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 --2- 172.21.15.16:0/2977535448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0348102d30 0x7f0348103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 -- 172.21.15.16:0/2977535448 >> 172.21.15.16:0/2977535448 conn(0x7f03480fc7d0 msgr2=0x7f03480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 -- 172.21.15.16:0/2977535448 shutdown_connections 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 -- 172.21.15.16:0/2977535448 wait complete. 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 Processor -- start 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.286+0000 7f03500e1640 1 -- start start 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f03500e1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0348102d30 0x7f0348198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f03500e1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481036e0 0x7f0348198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f03500e1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03481082a0 0x7f034819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f03500e1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03481120a0 con 0x7f03481036e0 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f03500e1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0348111f20 con 0x7f03481082a0 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f03500e1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0348112220 con 0x7f0348102d30 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481036e0 0x7f0348198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481036e0 0x7f0348198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49314/0 (socket says 172.21.15.16:49314) 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 -- 172.21.15.16:0/4293980906 learned_addr learned my addr 172.21.15.16:0/4293980906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034de7d640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0348102d30 0x7f0348198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034e67e640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03481082a0 0x7f034819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 -- 172.21.15.16:0/4293980906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0348102d30 msgr2=0x7f0348198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0348102d30 0x7f0348198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 -- 172.21.15.16:0/4293980906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03481082a0 msgr2=0x7f034819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03481082a0 0x7f034819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 -- 172.21.15.16:0/4293980906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034819d500 con 0x7f03481036e0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.287+0000 7f034d67c640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481036e0 0x7f0348198a90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f033800d750 tx=0x7f033800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.288+0000 7f033effd640 1 -- 172.21.15.16:0/4293980906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338017070 con 0x7f03481036e0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.288+0000 7f033effd640 1 -- 172.21.15.16:0/4293980906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0338005500 con 0x7f03481036e0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.288+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034819d7f0 con 0x7f03481036e0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.288+0000 7f033effd640 1 -- 172.21.15.16:0/4293980906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338004e00 con 0x7f03481036e0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.288+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a50d0 con 0x7f03481036e0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f033effd640 1 -- 172.21.15.16:0/4293980906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7f0338004fa0 con 0x7f03481036e0 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 Processor -- start 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 -- start start 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf23090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf23a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf30760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56147ce892f0 con 0x56147cf235d0 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56147ce30cf0 con 0x56147cf24b40 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f03524ba740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56147ce852b0 con 0x56147cf2e370 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f034de7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf23a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f034de7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf23a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49326/0 (socket says 172.21.15.16:49326) 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f034de7d640 1 -- 172.21.15.16:0/4214271933 learned_addr learned my addr 172.21.15.16:0/4214271933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.291+0000 7f034e67e640 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf23090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034d67c640 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf30760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034de7d640 1 -- 172.21.15.16:0/4214271933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 msgr2=0x56147cf30760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034de7d640 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf30760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034de7d640 1 -- 172.21.15.16:0/4214271933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 msgr2=0x56147cf23090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f033effd640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f03340a3550 0x7f03340a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034de7d640 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf23090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034de7d640 1 -- 172.21.15.16:0/4214271933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147ce153a0 con 0x56147cf235d0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034e67e640 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034e67e640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f03340a3550 0x7f03340a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f033effd640 1 -- 172.21.15.16:0/4293980906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f03380cfb50 con 0x7f03481036e0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f034de7d640 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf23a10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0344009cf0 tx=0x7f0344011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e7287ff28cf83c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f0333fff640 1 -- 172.21.15.16:0/4214271933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344004230 con 0x56147cf235d0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f0333fff640 1 -- 172.21.15.16:0/4214271933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03440043d0 con 0x56147cf235d0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.292+0000 7f0333fff640 1 -- 172.21.15.16:0/4214271933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03440198c0 con 0x56147cf235d0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 -- 172.21.15.16:0/4214271933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 msgr2=0x56147cf23a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f034e67e640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f03340a3550 0x7f03340a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0340005e30 tx=0x7f03400095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf23a10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0344009cf0 tx=0x7f0344011810 comp rx=0 tx=0).stop 2024-09-15T22:39:15.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 -- 172.21.15.16:0/4214271933 shutdown_connections 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf30760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf23a10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 --2- 172.21.15.16:0/4214271933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf23090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 -- 172.21.15.16:0/4214271933 >> 172.21.15.16:0/4214271933 conn(0x56147ceec310 msgr2=0x56147cf1fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 -- 172.21.15.16:0/4214271933 shutdown_connections 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.293+0000 7f03524ba740 1 -- 172.21.15.16:0/4214271933 wait complete. 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 -- start start 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 Processor -- start 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf235d0 0x56147cf2dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf2af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf2e370 0x56147cf2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56147cf31070 con 0x56147cf24b40 2024-09-15T22:39:15.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56147ce44060 con 0x56147cf2e370 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.294+0000 7f03524ba740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56147cf311f0 con 0x56147cf235d0 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f03524ba740 10 client.0 _ll_register_callbacks cb 0x56147cb978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf2af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf2af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49336/0 (socket says 172.21.15.16:49336) 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 -- 172.21.15.16:0/897370385 learned_addr learned my addr 172.21.15.16:0/897370385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034e67e640 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf2e370 0x56147cf2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034de7d640 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf235d0 0x56147cf2dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 -- 172.21.15.16:0/897370385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf235d0 msgr2=0x56147cf2dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf235d0 0x56147cf2dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 -- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf2e370 msgr2=0x56147cf2b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf2e370 0x56147cf2b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 -- 172.21.15.16:0/897370385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147cbfa3c0 con 0x56147cf24b40 2024-09-15T22:39:15.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f034d67c640 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf2af70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f033809b630 tx=0x7f033809b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63e7503c3f1f959 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f03317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f03317fa640 1 -- 172.21.15.16:0/897370385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338020070 con 0x56147cf24b40 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f03317fa640 1 -- 172.21.15.16:0/897370385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0338009cc0 con 0x56147cf24b40 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.295+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03440043d0 con 0x56147cf24b40 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03317fa640 1 -- 172.21.15.16:0/897370385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f03380b9050 con 0x56147cf24b40 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 10 client.15615 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56147cf3cb30 con 0x56147cf24b40 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03317fa640 1 -- 172.21.15.16:0/897370385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03440043d0 con 0x56147cf24b40 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03317fa640 1 -- 172.21.15.16:0/897370385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0338097fa0 con 0x56147cf24b40 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 10 client.15615 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 10 client.15615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 20 client.15615 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cf3df40 0x56147cf40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 4 client.15615 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56147ce96860 con 0x56147cf3df40 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.296+0000 7f034de7d640 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cf3df40 0x56147cf40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.297+0000 7f034de7d640 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cf3df40 0x56147cf40350 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1866c396a3cb2955 server_cookie=cf520055e3e38002 in_seq=0 out_seq=0 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.297+0000 7f03317fa640 10 client.15615 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03317fa640 1 -- 172.21.15.16:0/897370385 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56147ce96860 con 0x56147cf3df40 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03317fa640 10 client.15615 handle_command_reply: tid=0 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 client.15615 shutdown 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 msgr2=0x56147cf2af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf2af70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f033809b630 tx=0x7f033809b290 comp rx=0 tx=0).stop 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 shutdown_connections 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cf3df40 0x56147cf40350 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf2e370 0x56147cf2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf2af70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 --2- 172.21.15.16:0/897370385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf235d0 0x56147cf2dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 >> 172.21.15.16:0/897370385 conn(0x56147ceec310 msgr2=0x56147cf25960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.298+0000 7f03317fa640 0 client.15615 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.299+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 shutdown_connections 2024-09-15T22:39:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.299+0000 7f03524ba740 1 -- 172.21.15.16:0/897370385 wait complete. 2024-09-15T22:39:15.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.299+0000 7f03524ba740 20 client.15615 trim_cache size 0 max 16384 2024-09-15T22:39:15.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.312+0000 7f03524ba740 1 Processor -- start 2024-09-15T22:39:15.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f03524ba740 1 -- start start 2024-09-15T22:39:15.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f03524ba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf235d0 0x56147cf3a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f03524ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf24300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f03524ba740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf1f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f03524ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03481af560 con 0x56147cf24b40 2024-09-15T22:39:15.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f03524ba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03481ace30 con 0x56147cf235d0 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f03524ba740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f03481aac80 con 0x56147cf2e370 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf24300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf24300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49352/0 (socket says 172.21.15.16:49352) 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 -- 172.21.15.16:0/2427946496 learned_addr learned my addr 172.21.15.16:0/2427946496 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034de7d640 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf235d0 0x56147cf3a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034e67e640 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf1f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 -- 172.21.15.16:0/2427946496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 msgr2=0x56147cf1f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf1f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 -- 172.21.15.16:0/2427946496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf235d0 msgr2=0x56147cf3a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf235d0 0x56147cf3a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034d67c640 1 -- 172.21.15.16:0/2427946496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56147cf294e0 con 0x56147cf24b40 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.313+0000 7f034e67e640 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf1f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:15.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.314+0000 7f034d67c640 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf24300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0338097d90 tx=0x7f033809b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40db6d0a883b2da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.314+0000 7f03337fe640 1 -- 172.21.15.16:0/2427946496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033807f070 con 0x56147cf24b40 2024-09-15T22:39:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.314+0000 7f03337fe640 1 -- 172.21.15.16:0/2427946496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03380ccd00 con 0x56147cf24b40 2024-09-15T22:39:15.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.314+0000 7f03337fe640 1 -- 172.21.15.16:0/2427946496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0338020050 con 0x56147cf24b40 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 -- 172.21.15.16:0/2427946496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 msgr2=0x56147cf24300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf24300 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0338097d90 tx=0x7f033809b3c0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 -- 172.21.15.16:0/2427946496 shutdown_connections 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf1f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf24b40 0x56147cf24300 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 --2- 172.21.15.16:0/2427946496 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf235d0 0x56147cf3a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 -- 172.21.15.16:0/2427946496 >> 172.21.15.16:0/2427946496 conn(0x56147ceec310 msgr2=0x56147cf1aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 -- 172.21.15.16:0/2427946496 shutdown_connections 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 -- 172.21.15.16:0/2427946496 wait complete. 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 -- start start 2024-09-15T22:39:15.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.315+0000 7f03524ba740 1 Processor -- start 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f03524ba740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf36900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f03524ba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf27a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f03524ba740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf27f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f03524ba740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03481ab2f0 con 0x56147cf235d0 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f03524ba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03481ac760 con 0x56147cf24b40 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f03524ba740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f03481ab9c0 con 0x56147cf2e370 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf36900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf36900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49360/0 (socket says 172.21.15.16:49360) 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 -- 172.21.15.16:0/2395659727 learned_addr learned my addr 172.21.15.16:0/2395659727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f03524ba740 10 client.0 _ll_register_callbacks cb 0x7f03380d31b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034d67c640 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf27a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034e67e640 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf27f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 -- 172.21.15.16:0/2395659727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 msgr2=0x56147cf27f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf27f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 -- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 msgr2=0x56147cf27a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf27a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.316+0000 7f034de7d640 1 -- 172.21.15.16:0/2395659727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147cf284a0 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f034de7d640 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf36900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f03440099b0 tx=0x7f0344019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f4d11cb7c9d20f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03317fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03317fa640 1 -- 172.21.15.16:0/2395659727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03440054c0 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03317fa640 1 -- 172.21.15.16:0/2395659727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f03440035f0 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03380ccd00 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03317fa640 1 -- 172.21.15.16:0/2395659727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0344039050 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03524ba740 10 client.15621 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56147cfa8900 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.317+0000 7f03317fa640 1 -- 172.21.15.16:0/2395659727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03380ccd00 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03317fa640 1 -- 172.21.15.16:0/2395659727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f034402b880 con 0x56147cf235d0 2024-09-15T22:39:15.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03524ba740 10 client.15621 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03524ba740 10 client.15621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03524ba740 20 client.15621 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03524ba740 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cfa8ee0 0x56147cfab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03524ba740 4 client.15621 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56147ce90d70 con 0x56147cfa8ee0 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f034d67c640 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cfa8ee0 0x56147cfab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f03317fa640 10 client.15621 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.318+0000 7f034d67c640 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cfa8ee0 0x56147cfab300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97d519716fa0d698 server_cookie=52737097e7fdaf1c in_seq=0 out_seq=0 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03317fa640 1 -- 172.21.15.16:0/2395659727 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56147ce90d70 con 0x56147cfa8ee0 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03317fa640 10 client.15621 handle_command_reply: tid=0 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 client.15621 shutdown 2024-09-15T22:39:15.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 msgr2=0x56147cf36900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf36900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f03440099b0 tx=0x7f0344019420 comp rx=0 tx=0).stop 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 shutdown_connections 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x56147cfa8ee0 0x56147cfab300 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56147cf2e370 0x56147cf27f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03317fa640 0 client.15621 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56147cf24b40 0x56147cf27a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 --2- 172.21.15.16:0/2395659727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56147cf235d0 0x56147cf36900 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 >> 172.21.15.16:0/2395659727 conn(0x56147ceec310 msgr2=0x56147cf1f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 shutdown_connections 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 1 -- 172.21.15.16:0/2395659727 wait complete. 2024-09-15T22:39:15.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.319+0000 7f03524ba740 20 client.15621 trim_cache size 0 max 16384 2024-09-15T22:39:15.326 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:39:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f03340a3550 msgr2=0x7f03340a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f03340a3550 0x7f03340a5a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0340005e30 tx=0x7f03400095c0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481036e0 msgr2=0x7f0348198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481036e0 0x7f0348198a90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f033800d750 tx=0x7f033800dc10 comp rx=0 tx=0).stop 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 shutdown_connections 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03481082a0 0x7f034819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03481036e0 0x7f0348198a90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f03340a3550 0x7f03340a5a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 --2- 172.21.15.16:0/4293980906 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0348102d30 0x7f0348198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 >> 172.21.15.16:0/4293980906 conn(0x7f03480fc7d0 msgr2=0x7f0348106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 shutdown_connections 2024-09-15T22:39:15.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:15.320+0000 7f03500e1640 1 -- 172.21.15.16:0/4293980906 wait complete. 2024-09-15T22:39:16.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:39:16.320 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:16.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:16 smithi016 ceph-mon[31492]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-15T22:39:16.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:16 smithi130 ceph-mon[35742]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-15T22:39:16.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 -- 172.21.15.16:0/2304087361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb740106f20 msgr2=0x7fb74010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 --2- 172.21.15.16:0/2304087361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb740106f20 0x7fb74010eaa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb738009ca0 tx=0x7fb73802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 -- 172.21.15.16:0/2304087361 shutdown_connections 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 --2- 172.21.15.16:0/2304087361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb740106f20 0x7fb74010eaa0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 --2- 172.21.15.16:0/2304087361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7401027a0 0x7fb7401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 --2- 172.21.15.16:0/2304087361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb740101df0 0x7fb7401021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 -- 172.21.15.16:0/2304087361 >> 172.21.15.16:0/2304087361 conn(0x7fb7400fd8f0 msgr2=0x7fb7400ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 -- 172.21.15.16:0/2304087361 shutdown_connections 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.827+0000 7fb7470b1640 1 -- 172.21.15.16:0/2304087361 wait complete. 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 Processor -- start 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 -- start start 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb740101df0 0x7fb740198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7401027a0 0x7fb7401989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb740106f20 0x7fb74019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb740111f80 con 0x7fb7401027a0 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb740111e00 con 0x7fb740106f20 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.828+0000 7fb7470b1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb740112100 con 0x7fb740101df0 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7401027a0 0x7fb7401989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7401027a0 0x7fb7401989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49382/0 (socket says 172.21.15.16:49382) 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 -- 172.21.15.16:0/1889059818 learned_addr learned my addr 172.21.15.16:0/1889059818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb74564e640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb740106f20 0x7fb74019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 -- 172.21.15.16:0/1889059818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb740101df0 msgr2=0x7fb740198480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb740101df0 0x7fb740198480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 -- 172.21.15.16:0/1889059818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb740106f20 msgr2=0x7fb74019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb740106f20 0x7fb74019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 -- 172.21.15.16:0/1889059818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74019d430 con 0x7fb7401027a0 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb74564e640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb740106f20 0x7fb74019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.829+0000 7fb73ffff640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7401027a0 0x7fb7401989c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00d950 tx=0x7fb72c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.830+0000 7fb73dffb640 1 -- 172.21.15.16:0/1889059818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c017030 con 0x7fb7401027a0 2024-09-15T22:39:16.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.830+0000 7fb73dffb640 1 -- 172.21.15.16:0/1889059818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72c004020 con 0x7fb7401027a0 2024-09-15T22:39:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.830+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74019d720 con 0x7fb7401027a0 2024-09-15T22:39:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.830+0000 7fb73dffb640 1 -- 172.21.15.16:0/1889059818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0047a0 con 0x7fb7401027a0 2024-09-15T22:39:16.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.830+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7401a5000 con 0x7fb7401027a0 2024-09-15T22:39:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.833+0000 7fb74948a740 1 Processor -- start 2024-09-15T22:39:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.833+0000 7fb73dffb640 1 -- 172.21.15.16:0/1889059818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7fb72c0041d0 con 0x7fb7401027a0 2024-09-15T22:39:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.833+0000 7fb74948a740 1 -- start start 2024-09-15T22:39:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb74948a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25e8ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb74948a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25e8d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb74948a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb74948a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c25df32f0 con 0x564c25e8d120 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb74948a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c25d9acf0 con 0x564c25e8db80 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb74948a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564c25def2b0 con 0x564c25e8e840 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb744e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25e8d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb744e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25e8d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49398/0 (socket says 172.21.15.16:49398) 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb744e4d640 1 -- 172.21.15.16:0/1408401020 learned_addr learned my addr 172.21.15.16:0/1408401020 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb74564e640 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25e8ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.834+0000 7fb73ffff640 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb744e4d640 1 -- 172.21.15.16:0/1408401020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 msgr2=0x564c25e8ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb744e4d640 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25e8ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb744e4d640 1 -- 172.21.15.16:0/1408401020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 msgr2=0x564c25ea2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb744e4d640 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb744e4d640 1 -- 172.21.15.16:0/1408401020 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c25d7f3a0 con 0x564c25e8d120 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb73dffb640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7300a3620 0x7fb7300a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb744e4d640 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25e8d560 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb73400bac0 tx=0x7fb73400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf4b20aabd931a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb73dffb640 1 -- 172.21.15.16:0/1889059818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb72c0cfcd0 con 0x7fb7401027a0 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb74564e640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7300a3620 0x7fb7300a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb72affd640 1 -- 172.21.15.16:0/1408401020 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734016070 con 0x564c25e8d120 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb72affd640 1 -- 172.21.15.16:0/1408401020 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb73400dca0 con 0x564c25e8d120 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.835+0000 7fb72affd640 1 -- 172.21.15.16:0/1408401020 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734010460 con 0x564c25e8d120 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 -- 172.21.15.16:0/1408401020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 msgr2=0x564c25e8d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74564e640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7300a3620 0x7fb7300a5ae0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb738004af0 tx=0x7fb738005d80 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25e8d560 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb73400bac0 tx=0x7fb73400bf80 comp rx=0 tx=0).stop 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 -- 172.21.15.16:0/1408401020 shutdown_connections 2024-09-15T22:39:16.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25e8d560 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 --2- 172.21.15.16:0/1408401020 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25e8ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 -- 172.21.15.16:0/1408401020 >> 172.21.15.16:0/1408401020 conn(0x564c25e56310 msgr2=0x564c25e89c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 -- 172.21.15.16:0/1408401020 shutdown_connections 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.836+0000 7fb74948a740 1 -- 172.21.15.16:0/1408401020 wait complete. 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 -- start start 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 Processor -- start 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25ea2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25ea0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb744e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25ea2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564c25ea3930 con 0x564c25e8d120 2024-09-15T22:39:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c25dae060 con 0x564c25e8db80 2024-09-15T22:39:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb744e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25ea2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49414/0 (socket says 172.21.15.16:49414) 2024-09-15T22:39:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb74948a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564c25ea3ab0 con 0x564c25e8e840 2024-09-15T22:39:16.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.837+0000 7fb744e4d640 1 -- 172.21.15.16:0/397831866 learned_addr learned my addr 172.21.15.16:0/397831866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb74948a740 10 client.0 _ll_register_callbacks cb 0x564c25b018c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb73ffff640 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb744e4d640 1 -- 172.21.15.16:0/397831866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 msgr2=0x564c25ea0600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:39:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb744e4d640 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25ea0600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb744e4d640 1 -- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 msgr2=0x564c25ea00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb744e4d640 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb744e4d640 1 -- 172.21.15.16:0/397831866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c25b643c0 con 0x564c25e8d120 2024-09-15T22:39:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb73ffff640 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb744e4d640 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25ea2df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb734005fd0 tx=0x7fb73400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812a99f3a31a6aec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb71ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb71ffff640 1 -- 172.21.15.16:0/397831866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734016040 con 0x564c25e8d120 2024-09-15T22:39:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb71ffff640 1 -- 172.21.15.16:0/397831866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb734003e30 con 0x564c25e8d120 2024-09-15T22:39:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb73400dca0 con 0x564c25e8d120 2024-09-15T22:39:16.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.838+0000 7fb71ffff640 1 -- 172.21.15.16:0/397831866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb73400de50 con 0x564c25e8d120 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 10 client.15633 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c25eaf340 con 0x564c25e8d120 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb71ffff640 1 -- 172.21.15.16:0/397831866 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb73400dca0 con 0x564c25e8d120 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb71ffff640 1 -- 172.21.15.16:0/397831866 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb734021070 con 0x564c25e8d120 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 10 client.15633 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 20 client.15633 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25eb2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 4 client.15633 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c25e00860 con 0x564c25eb07a0 2024-09-15T22:39:16.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.839+0000 7fb73ffff640 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25eb2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.840+0000 7fb71ffff640 10 client.15633 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.840+0000 7fb73ffff640 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25eb2b90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c683599498dcc22 server_cookie=cb73853dec969f1a in_seq=0 out_seq=0 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb71ffff640 1 -- 172.21.15.16:0/397831866 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c25e00860 con 0x564c25eb07a0 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb71ffff640 10 client.15633 handle_command_reply: tid=0 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 client.15633 shutdown 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 msgr2=0x564c25ea2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25ea2df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb734005fd0 tx=0x7fb73400b7d0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 shutdown_connections 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25eb2b90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb71ffff640 0 client.15633 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25ea0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25ea00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 --2- 172.21.15.16:0/397831866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25ea2df0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 >> 172.21.15.16:0/397831866 conn(0x564c25e56310 msgr2=0x564c25e8f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 shutdown_connections 2024-09-15T22:39:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 1 -- 172.21.15.16:0/397831866 wait complete. 2024-09-15T22:39:16.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.841+0000 7fb74948a740 20 client.15633 trim_cache size 0 max 16384 2024-09-15T22:39:16.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.854+0000 7fb74948a740 1 Processor -- start 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.854+0000 7fb74948a740 1 -- start start 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74948a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8d120 0x564c25eacbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74948a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8db80 0x564c25e8fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74948a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8e840 0x564c25e9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74948a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401af600 con 0x564c25e8e840 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74948a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401aced0 con 0x564c25e8d120 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74948a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401aad20 con 0x564c25e8db80 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8e840 0x564c25e9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8e840 0x564c25e9f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49424/0 (socket says 172.21.15.16:49424) 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 -- 172.21.15.16:0/454922359 learned_addr learned my addr 172.21.15.16:0/454922359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb73ffff640 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8db80 0x564c25e8fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb744e4d640 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8d120 0x564c25eacbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 -- 172.21.15.16:0/454922359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8db80 msgr2=0x564c25e8fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8db80 0x564c25e8fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 -- 172.21.15.16:0/454922359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8d120 msgr2=0x564c25eacbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8d120 0x564c25eacbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.855+0000 7fb74564e640 1 -- 172.21.15.16:0/454922359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c25ea3810 con 0x564c25e8e840 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.856+0000 7fb74564e640 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8e840 0x564c25e9f080 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb738031e40 tx=0x7fb738040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36910c46ff369ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.856+0000 7fb72a7fc640 1 -- 172.21.15.16:0/454922359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738031850 con 0x564c25e8e840 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.856+0000 7fb72a7fc640 1 -- 172.21.15.16:0/454922359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb738036050 con 0x564c25e8e840 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.856+0000 7fb72a7fc640 1 -- 172.21.15.16:0/454922359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738030a80 con 0x564c25e8e840 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 -- 172.21.15.16:0/454922359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8e840 msgr2=0x564c25e9f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8e840 0x564c25e9f080 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb738031e40 tx=0x7fb738040000 comp rx=0 tx=0).stop 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 -- 172.21.15.16:0/454922359 shutdown_connections 2024-09-15T22:39:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8e840 0x564c25e9f080 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8db80 0x564c25e8fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 --2- 172.21.15.16:0/454922359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8d120 0x564c25eacbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 -- 172.21.15.16:0/454922359 >> 172.21.15.16:0/454922359 conn(0x564c25e56310 msgr2=0x564c25e93ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 -- 172.21.15.16:0/454922359 shutdown_connections 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 -- 172.21.15.16:0/454922359 wait complete. 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.857+0000 7fb74948a740 1 -- start start 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 1 Processor -- start 2024-09-15T22:39:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25eb73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25eb7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25ea7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ab390 con 0x564c25e8d120 2024-09-15T22:39:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ac800 con 0x564c25e8db80 2024-09-15T22:39:16.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401aba60 con 0x564c25e8e840 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74948a740 10 client.0 _ll_register_callbacks cb 0x7fb738009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb744e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25eb73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb744e4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25eb73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49434/0 (socket says 172.21.15.16:49434) 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb744e4d640 1 -- 172.21.15.16:0/4059418946 learned_addr learned my addr 172.21.15.16:0/4059418946 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb74564e640 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25ea7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.858+0000 7fb73ffff640 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25eb7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb744e4d640 1 -- 172.21.15.16:0/4059418946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 msgr2=0x564c25ea7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb744e4d640 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25ea7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb744e4d640 1 -- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 msgr2=0x564c25eb7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb744e4d640 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25eb7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb744e4d640 1 -- 172.21.15.16:0/4059418946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c25eb81d0 con 0x564c25e8d120 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb744e4d640 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25eb73f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb734004950 tx=0x7fb7340059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822fc43c133bbe6d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb71ffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb71ffff640 1 -- 172.21.15.16:0/4059418946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734035060 con 0x564c25e8d120 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb71ffff640 1 -- 172.21.15.16:0/4059418946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb7340053f0 con 0x564c25e8d120 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.859+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb738036050 con 0x564c25e8d120 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb71ffff640 1 -- 172.21.15.16:0/4059418946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb73403c050 con 0x564c25e8d120 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb74948a740 10 client.15639 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c25ea9350 con 0x564c25e8d120 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb71ffff640 1 -- 172.21.15.16:0/4059418946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb73400d770 con 0x564c25e8d120 2024-09-15T22:39:16.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb71ffff640 1 -- 172.21.15.16:0/4059418946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb734021040 con 0x564c25e8d120 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb74948a740 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb74948a740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb74948a740 20 client.15639 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb74948a740 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25f1d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.860+0000 7fb74948a740 4 client.15639 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.861+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb738036050 con 0x564c25eb07a0 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.861+0000 7fb73ffff640 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25f1d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.861+0000 7fb73ffff640 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25f1d300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8114e65581eaa52a server_cookie=8f148f9e6171fda0 in_seq=0 out_seq=0 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.861+0000 7fb71ffff640 10 client.15639 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.861+0000 7fb71ffff640 1 -- 172.21.15.16:0/4059418946 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb738036050 con 0x564c25eb07a0 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.861+0000 7fb71ffff640 10 client.15639 handle_command_reply: tid=0 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.861+0000 7fb74948a740 1 client.15639 shutdown 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 msgr2=0x564c25eb73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25eb73f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb734004950 tx=0x7fb7340059b0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 shutdown_connections 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564c25eb07a0 0x564c25f1d300 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564c25e8e840 0x564c25ea7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c25e8db80 0x564c25eb7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 --2- 172.21.15.16:0/4059418946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564c25e8d120 0x564c25eb73f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 >> 172.21.15.16:0/4059418946 conn(0x564c25e56310 msgr2=0x564c25e9f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb71ffff640 0 client.15639 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 shutdown_connections 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 1 -- 172.21.15.16:0/4059418946 wait complete. 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.862+0000 7fb74948a740 20 client.15639 trim_cache size 0 max 16384 2024-09-15T22:39:16.870 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7300a3620 msgr2=0x7fb7300a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7300a3620 0x7fb7300a5ae0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb738004af0 tx=0x7fb738005d80 comp rx=0 tx=0).stop 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7401027a0 msgr2=0x7fb7401989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7401027a0 0x7fb7401989c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00d950 tx=0x7fb72c00de10 comp rx=0 tx=0).stop 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 shutdown_connections 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb740106f20 0x7fb74019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7401027a0 0x7fb7401989c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7300a3620 0x7fb7300a5ae0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 --2- 172.21.15.16:0/1889059818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb740101df0 0x7fb740198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 >> 172.21.15.16:0/1889059818 conn(0x7fb7400fd8f0 msgr2=0x7fb7400fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:16.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 shutdown_connections 2024-09-15T22:39:16.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:16.863+0000 7fb7470b1640 1 -- 172.21.15.16:0/1889059818 wait complete. 2024-09-15T22:39:16.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:16 smithi059 ceph-mon[34872]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-15T22:39:18.667 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:39:18.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:39:18.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:18 smithi130 ceph-mon[35742]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-15T22:39:18.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:18 smithi016 ceph-mon[31492]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-15T22:39:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:18 smithi059 ceph-mon[34872]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-15T22:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:19.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 -- 172.21.15.16:0/404853435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f30100e20 msgr2=0x7f8f30101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 --2- 172.21.15.16:0/404853435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f30100e20 0x7f8f30101200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009900 tx=0x7f8f2002a480 comp rx=0 tx=0).stop 2024-09-15T22:39:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 -- 172.21.15.16:0/404853435 shutdown_connections 2024-09-15T22:39:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 --2- 172.21.15.16:0/404853435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f30102ef0 0x7f8f3010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 --2- 172.21.15.16:0/404853435 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 0x7f8f301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 --2- 172.21.15.16:0/404853435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f30100e20 0x7f8f30101200 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 -- 172.21.15.16:0/404853435 >> 172.21.15.16:0/404853435 conn(0x7f8f300fc7d0 msgr2=0x7f8f300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 -- 172.21.15.16:0/404853435 shutdown_connections 2024-09-15T22:39:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.555+0000 7f8f34da1640 1 -- 172.21.15.16:0/404853435 wait complete. 2024-09-15T22:39:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.556+0000 7f8f34da1640 1 Processor -- start 2024-09-15T22:39:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.556+0000 7f8f34da1640 1 -- start start 2024-09-15T22:39:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.556+0000 7f8f34da1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f30100e20 0x7f8f301971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.556+0000 7f8f34da1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 0x7f8f30197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f34da1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f30102ef0 0x7f8f30197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f34da1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30110f50 con 0x7f8f30100e20 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f34da1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30110dd0 con 0x7f8f30102ef0 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 0x7f8f30197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 0x7f8f30197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:40050/0 (socket says 172.21.15.16:40050) 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 -- 172.21.15.16:0/1420150295 learned_addr learned my addr 172.21.15.16:0/1420150295 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301110d0 con 0x7f8f30102570 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f30102ef0 0x7f8f30197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 -- 172.21.15.16:0/1420150295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f30102ef0 msgr2=0x7f8f30197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f30102ef0 0x7f8f30197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 -- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f30100e20 msgr2=0x7f8f301971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f30100e20 0x7f8f301971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.557+0000 7f8f2dd9b640 1 -- 172.21.15.16:0/1420150295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f301a6a60 con 0x7f8f30102570 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.558+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 0x7f8f30197730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400d950 tx=0x7f8f2400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:19.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.558+0000 7f8f1f7fe640 1 -- 172.21.15.16:0/1420150295 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24017070 con 0x7f8f30102570 2024-09-15T22:39:19.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.558+0000 7f8f1f7fe640 1 -- 172.21.15.16:0/1420150295 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f24009c30 con 0x7f8f30102570 2024-09-15T22:39:19.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.558+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f301a6d50 con 0x7f8f30102570 2024-09-15T22:39:19.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.558+0000 7f8f1f7fe640 1 -- 172.21.15.16:0/1420150295 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2400f6b0 con 0x7f8f30102570 2024-09-15T22:39:19.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.558+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f301a7250 con 0x7f8f30102570 2024-09-15T22:39:19.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.561+0000 7f8f1f7fe640 1 -- 172.21.15.16:0/1420150295 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7f8f24010460 con 0x7f8f30102570 2024-09-15T22:39:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 Processor -- start 2024-09-15T22:39:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 -- start start 2024-09-15T22:39:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d67b60 0x555741d67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d68480 0x555741d72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d72d40 0x555741d75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555741ccd280 con 0x555741d72d40 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555741c74b10 con 0x555741d67b60 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f3717a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555741cc9240 con 0x555741d68480 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f2ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d67b60 0x555741d67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f2ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d67b60 0x555741d67f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:58092/0 (socket says 172.21.15.16:58092) 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/1260587840 learned_addr learned my addr 172.21.15.16:0/1260587840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.562+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d68480 0x555741d72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d72d40 0x555741d75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/1260587840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d68480 msgr2=0x555741d72800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d68480 0x555741d72800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/1260587840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d72d40 msgr2=0x555741d75130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d72d40 0x555741d75130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/1260587840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555741c811f0 con 0x555741d67b60 2024-09-15T22:39:19.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d72d40 0x555741d75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:19.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d67b60 0x555741d67f40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009b60 tx=0x7f8f2800e780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16c6450e0ad87a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.563+0000 7f8f1f7fe640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8f180a3550 0x7f8f180a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.564+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8f180a3550 0x7f8f180a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.564+0000 7f8f1f7fe640 1 -- 172.21.15.16:0/1420150295 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f8f240d0250 con 0x7f8f30102570 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.564+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8f180a3550 0x7f8f180a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009230 tx=0x7f8f20000bc0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.565+0000 7f8f1cff9640 1 -- 172.21.15.16:0/1260587840 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28009c50 con 0x555741d67b60 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.565+0000 7f8f1cff9640 1 -- 172.21.15.16:0/1260587840 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f28003e50 con 0x555741d67b60 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.565+0000 7f8f1cff9640 1 -- 172.21.15.16:0/1260587840 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28020620 con 0x555741d67b60 2024-09-15T22:39:19.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 -- 172.21.15.16:0/1260587840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d67b60 msgr2=0x555741d67f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d67b60 0x555741d67f40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009b60 tx=0x7f8f2800e780 comp rx=0 tx=0).stop 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 -- 172.21.15.16:0/1260587840 shutdown_connections 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d72d40 0x555741d75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d68480 0x555741d72800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1260587840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d67b60 0x555741d67f40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 -- 172.21.15.16:0/1260587840 >> 172.21.15.16:0/1260587840 conn(0x555741d640f0 msgr2=0x555741d644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.566+0000 7f8f3717a740 1 -- 172.21.15.16:0/1260587840 shutdown_connections 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.567+0000 7f8f3717a740 1 -- 172.21.15.16:0/1260587840 wait complete. 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.567+0000 7f8f3717a740 1 -- start start 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.567+0000 7f8f3717a740 1 Processor -- start 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d6fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d72d40 0x555741d70700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f3717a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555741d75910 con 0x555741d68480 2024-09-15T22:39:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f3717a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555741c87f10 con 0x555741d72d40 2024-09-15T22:39:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f3717a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555741d75a90 con 0x555741d67b60 2024-09-15T22:39:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f3717a740 10 client.0 _ll_register_callbacks cb 0x555741c50a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:19.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d6fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d72d40 0x555741d70700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d72d40 0x555741d70700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:58098/0 (socket says 172.21.15.16:58098) 2024-09-15T22:39:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/521270888 learned_addr learned my addr 172.21.15.16:0/521270888 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/521270888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 msgr2=0x555741d6fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d6fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/521270888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 msgr2=0x555741d701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.568+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.569+0000 7f8f2ed9d640 1 -- 172.21.15.16:0/521270888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555741a3e6a0 con 0x555741d72d40 2024-09-15T22:39:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.569+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.570+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d72d40 0x555741d70700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f28004500 tx=0x7f8f28017dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac479ad1f0eef49c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:19.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.570+0000 7f8f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:39:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.571+0000 7f8f15ffb640 1 -- 172.21.15.16:0/521270888 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2802f060 con 0x555741d72d40 2024-09-15T22:39:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.571+0000 7f8f15ffb640 1 -- 172.21.15.16:0/521270888 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f8f28003970 con 0x555741d72d40 2024-09-15T22:39:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.571+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f28003e50 con 0x555741d72d40 2024-09-15T22:39:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.571+0000 7f8f15ffb640 1 -- 172.21.15.16:0/521270888 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f28036050 con 0x555741d72d40 2024-09-15T22:39:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.571+0000 7f8f3717a740 10 client.25292 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:19.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.571+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555741d77590 con 0x555741d72d40 2024-09-15T22:39:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.571+0000 7f8f15ffb640 1 -- 172.21.15.16:0/521270888 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f28003b70 con 0x555741d72d40 2024-09-15T22:39:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f15ffb640 1 -- 172.21.15.16:0/521270888 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f28029030 con 0x555741d72d40 2024-09-15T22:39:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f3717a740 10 client.25292 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f3717a740 10 client.25292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f3717a740 20 client.25292 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f3717a740 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d78a50 0x555741d7ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f3717a740 4 client.25292 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:19.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8f28003e50 con 0x555741d78a50 2024-09-15T22:39:19.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.572+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d78a50 0x555741d7ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.573+0000 7f8f15ffb640 10 client.25292 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.573+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d78a50 0x555741d7ae40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca75b2e4c0c47b5 server_cookie=b1d954a29f246217 in_seq=0 out_seq=0 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.573+0000 7f8f15ffb640 1 -- 172.21.15.16:0/521270888 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8f28003e50 con 0x555741d78a50 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.573+0000 7f8f15ffb640 10 client.25292 handle_command_reply: tid=0 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.573+0000 7f8f3717a740 1 client.25292 shutdown 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.573+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d72d40 msgr2=0x555741d70700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.573+0000 7f8f3717a740 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d72d40 0x555741d70700 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8f28004500 tx=0x7f8f28017dc0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 shutdown_connections 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d78a50 0x555741d7ae40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d72d40 0x555741d70700 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 --2- 172.21.15.16:0/521270888 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d6fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 >> 172.21.15.16:0/521270888 conn(0x555741d640f0 msgr2=0x555741d69dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f15ffb640 0 client.25292 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 shutdown_connections 2024-09-15T22:39:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 1 -- 172.21.15.16:0/521270888 wait complete. 2024-09-15T22:39:19.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.574+0000 7f8f3717a740 20 client.25292 trim_cache size 0 max 16384 2024-09-15T22:39:19.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.608+0000 7f8f3717a740 1 Processor -- start 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.608+0000 7f8f3717a740 1 -- start start 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.608+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.608+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.608+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d7dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f3717a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30073df0 con 0x555741d68480 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f3717a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f300716c0 con 0x555741d640f0 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f3717a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3006f510 con 0x555741d67b60 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d70550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:58108/0 (socket says 172.21.15.16:58108) 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2e59c640 1 -- 172.21.15.16:0/1122115521 learned_addr learned my addr 172.21.15.16:0/1122115521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:19.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d7dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2dd9b640 1 -- 172.21.15.16:0/1122115521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 msgr2=0x555741d70550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d70550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2dd9b640 1 -- 172.21.15.16:0/1122115521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 msgr2=0x555741d7dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d7dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2dd9b640 1 -- 172.21.15.16:0/1122115521 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555741d759b0 con 0x555741d67b60 2024-09-15T22:39:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d70550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d7dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.609+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7da80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f240cd4c0 tx=0x7f8f24005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b7e9e389d91f9de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.610+0000 7f8f17fff640 1 -- 172.21.15.16:0/1122115521 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24020040 con 0x555741d67b60 2024-09-15T22:39:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.610+0000 7f8f17fff640 1 -- 172.21.15.16:0/1122115521 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f2409c680 con 0x555741d67b60 2024-09-15T22:39:19.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.610+0000 7f8f17fff640 1 -- 172.21.15.16:0/1122115521 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2409a350 con 0x555741d67b60 2024-09-15T22:39:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 -- 172.21.15.16:0/1122115521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 msgr2=0x555741d7da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7da80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f240cd4c0 tx=0x7f8f24005880 comp rx=0 tx=0).stop 2024-09-15T22:39:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 -- 172.21.15.16:0/1122115521 shutdown_connections 2024-09-15T22:39:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741d7dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7da80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 --2- 172.21.15.16:0/1122115521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d70550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 -- 172.21.15.16:0/1122115521 >> 172.21.15.16:0/1122115521 conn(0x555741d64b70 msgr2=0x555741d69dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 -- 172.21.15.16:0/1122115521 shutdown_connections 2024-09-15T22:39:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 -- 172.21.15.16:0/1122115521 wait complete. 2024-09-15T22:39:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.611+0000 7f8f3717a740 1 -- start start 2024-09-15T22:39:19.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 1 Processor -- start 2024-09-15T22:39:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741e1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3006fb80 con 0x555741d68480 2024-09-15T22:39:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30070ff0 con 0x555741d640f0 2024-09-15T22:39:19.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30070250 con 0x555741d67b60 2024-09-15T22:39:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f3717a740 10 client.0 _ll_register_callbacks cb 0x7f8f28017ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d7bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:58120/0 (socket says 172.21.15.16:58120) 2024-09-15T22:39:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f2e59c640 1 -- 172.21.15.16:0/2735984222 learned_addr learned my addr 172.21.15.16:0/2735984222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.612+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741e1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2e59c640 1 -- 172.21.15.16:0/2735984222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 msgr2=0x555741d7c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2e59c640 1 -- 172.21.15.16:0/2735984222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 msgr2=0x555741e1ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741e1ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2e59c640 1 -- 172.21.15.16:0/2735984222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555741e1f3d0 con 0x555741d640f0 2024-09-15T22:39:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2ed9d640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741e1ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f2e59c640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d7bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f200045c0 tx=0x7f8f20009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9629fef2e59d35d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:19.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:39:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2735984222 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2005a060 con 0x555741d640f0 2024-09-15T22:39:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.613+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2735984222 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f8f20004ad0 con 0x555741d640f0 2024-09-15T22:39:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.614+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f2409c680 con 0x555741d640f0 2024-09-15T22:39:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.614+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2735984222 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8f20061050 con 0x555741d640f0 2024-09-15T22:39:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.614+0000 7f8f3717a740 10 client.25298 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.614+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555741e204b0 con 0x555741d640f0 2024-09-15T22:39:19.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.614+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2735984222 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f2409c680 con 0x555741d640f0 2024-09-15T22:39:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2735984222 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8f2003a040 con 0x555741d640f0 2024-09-15T22:39:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f3717a740 10 client.25298 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f3717a740 10 client.25298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f3717a740 20 client.25298 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f3717a740 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d72d40 0x555741e22c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f3717a740 4 client.25298 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:39:19.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555741cda7f0 con 0x555741d72d40 2024-09-15T22:39:19.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d72d40 0x555741e22c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:19.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f2dd9b640 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d72d40 0x555741e22c20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d341f61fd350b7 server_cookie=ff80537905d325d7 in_seq=0 out_seq=0 2024-09-15T22:39:19.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.615+0000 7f8f15ffb640 10 client.25298 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f15ffb640 1 -- 172.21.15.16:0/2735984222 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555741cda7f0 con 0x555741d72d40 2024-09-15T22:39:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f15ffb640 10 client.25298 handle_command_reply: tid=0 2024-09-15T22:39:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 client.25298 shutdown 2024-09-15T22:39:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 msgr2=0x555741d7bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d7bbc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f200045c0 tx=0x7f8f20009260 comp rx=0 tx=0).stop 2024-09-15T22:39:19.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 shutdown_connections 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555741d72d40 0x555741e22c20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555741d68480 0x555741e1ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555741d67b60 0x555741d7c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 --2- 172.21.15.16:0/2735984222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555741d640f0 0x555741d7bbc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.617+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 >> 172.21.15.16:0/2735984222 conn(0x555741d64b70 msgr2=0x555741d695b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.618+0000 7f8f15ffb640 0 client.25298 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.618+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 shutdown_connections 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.618+0000 7f8f3717a740 1 -- 172.21.15.16:0/2735984222 wait complete. 2024-09-15T22:39:19.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.618+0000 7f8f3717a740 20 client.25298 trim_cache size 0 max 16384 2024-09-15T22:39:19.628 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:39:19.628 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:39:19.628 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "86738c3b-1bf2-438f-bd58-11b275681bf2", 2024-09-15T22:39:19.628 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:39:19.629 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:39:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8f180a3550 msgr2=0x7f8f180a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8f180a3550 0x7f8f180a5a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009230 tx=0x7f8f20000bc0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 msgr2=0x7f8f30197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 0x7f8f30197730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400d950 tx=0x7f8f2400de10 comp rx=0 tx=0).stop 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 shutdown_connections 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f30102ef0 0x7f8f30197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8f180a3550 0x7f8f180a5a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8f30102570 0x7f8f30197730 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 --2- 172.21.15.16:0/1420150295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f30100e20 0x7f8f301971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.619+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 >> 172.21.15.16:0/1420150295 conn(0x7f8f300fc7d0 msgr2=0x7f8f300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.620+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 shutdown_connections 2024-09-15T22:39:19.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:19.620+0000 7f8f34da1640 1 -- 172.21.15.16:0/1420150295 wait complete. 2024-09-15T22:39:20.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:39:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:20 smithi130 ceph-mon[35742]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-15T22:39:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:20 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:39:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:20 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:39:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:20 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:39:20.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:20 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:39:20.801 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:20 smithi016 ceph-mon[31492]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-15T22:39:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:20 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:39:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:20 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:39:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:20 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:39:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:20 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:39:20.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:20 smithi059 ceph-mon[34872]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-15T22:39:20.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:20 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:39:20.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:20 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:39:20.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:20 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:39:20.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:20 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:39:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.314+0000 7f8a102af640 1 -- 172.21.15.16:0/29115787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 msgr2=0x7f8a081067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.314+0000 7f8a102af640 1 --2- 172.21.15.16:0/29115787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a081067f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009900 tx=0x7f89f802a410 comp rx=0 tx=0).stop 2024-09-15T22:39:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 -- 172.21.15.16:0/29115787 shutdown_connections 2024-09-15T22:39:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 --2- 172.21.15.16:0/29115787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a08106f20 0x7f8a0810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 --2- 172.21.15.16:0/29115787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a081067f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 --2- 172.21.15.16:0/29115787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a08101df0 0x7f8a081021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 -- 172.21.15.16:0/29115787 >> 172.21.15.16:0/29115787 conn(0x7f8a080fd8f0 msgr2=0x7f8a080ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 -- 172.21.15.16:0/29115787 shutdown_connections 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 -- 172.21.15.16:0/29115787 wait complete. 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.315+0000 7f8a102af640 1 Processor -- start 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a102af640 1 -- start start 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a102af640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a08101df0 0x7f8a080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a102af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a08074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a102af640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a08106f20 0x7f8a08074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a102af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08111f80 con 0x7f8a081027a0 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a102af640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08111e00 con 0x7f8a08106f20 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a102af640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08112100 con 0x7f8a08101df0 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a0d84a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a08074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a0d84a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a08074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49502/0 (socket says 172.21.15.16:49502) 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a0d84a640 1 -- 172.21.15.16:0/131090442 learned_addr learned my addr 172.21.15.16:0/131090442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a08101df0 0x7f8a080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.316+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a08106f20 0x7f8a08074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a0d84a640 1 -- 172.21.15.16:0/131090442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a08101df0 msgr2=0x7f8a080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a08101df0 0x7f8a080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a0d84a640 1 -- 172.21.15.16:0/131090442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a08106f20 msgr2=0x7f8a08074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a08106f20 0x7f8a08074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a0d84a640 1 -- 172.21.15.16:0/131090442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a081a69c0 con 0x7f8a081027a0 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a08106f20 0x7f8a08074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a08074a30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006010 tx=0x7f89f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f89ff7fe640 1 -- 172.21.15.16:0/131090442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f803f030 con 0x7f8a081027a0 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f89ff7fe640 1 -- 172.21.15.16:0/131090442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f8003ce0 con 0x7f8a081027a0 2024-09-15T22:39:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f89ff7fe640 1 -- 172.21.15.16:0/131090442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8035070 con 0x7f8a081027a0 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081a6c50 con 0x7f8a081027a0 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.317+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081a70e0 con 0x7f8a081027a0 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.320+0000 7f89ff7fe640 1 -- 172.21.15.16:0/131090442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7f89f8042030 con 0x7f8a081027a0 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.320+0000 7f89ff7fe640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f89f40a36a0 0x7f89f40a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.320+0000 7f89ff7fe640 1 -- 172.21.15.16:0/131090442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f89f80f03c0 con 0x7f8a081027a0 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f89f40a36a0 0x7f89f40a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 Processor -- start 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 -- start start 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f89f40a36a0 0x7f89f40a5b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005fd0 tx=0x7f8a0400b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b3744a0 0x55e96b374880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b381a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e96b2da2e0 con 0x55e96b3744a0 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e96b281ce0 con 0x55e96b37f680 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.321+0000 7f8a12688740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e96b2d62a0 con 0x55e96b374dc0 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b3744a0 0x55e96b374880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b3744a0 0x55e96b374880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49504/0 (socket says 172.21.15.16:49504) 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 -- 172.21.15.16:0/2844738894 learned_addr learned my addr 172.21.15.16:0/2844738894 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b381a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 -- 172.21.15.16:0/2844738894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 msgr2=0x55e96b37f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 -- 172.21.15.16:0/2844738894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 msgr2=0x55e96b381a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b381a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 -- 172.21.15.16:0/2844738894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96b266440 con 0x55e96b3744a0 2024-09-15T22:39:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b381a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.322+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b3744a0 0x55e96b374880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400b620 tx=0x7f8a04013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66745625a5b7c8ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.323+0000 7f89fcdea640 1 -- 172.21.15.16:0/2844738894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04003e90 con 0x55e96b3744a0 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.323+0000 7f89fcdea640 1 -- 172.21.15.16:0/2844738894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a04005500 con 0x55e96b3744a0 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.323+0000 7f89fcdea640 1 -- 172.21.15.16:0/2844738894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0401e660 con 0x55e96b3744a0 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 -- 172.21.15.16:0/2844738894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b3744a0 msgr2=0x55e96b374880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b3744a0 0x55e96b374880 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400b620 tx=0x7f8a04013000 comp rx=0 tx=0).stop 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 -- 172.21.15.16:0/2844738894 shutdown_connections 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b381a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 --2- 172.21.15.16:0/2844738894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b3744a0 0x55e96b374880 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 -- 172.21.15.16:0/2844738894 >> 172.21.15.16:0/2844738894 conn(0x55e96b370a30 msgr2=0x55e96b370e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 -- 172.21.15.16:0/2844738894 shutdown_connections 2024-09-15T22:39:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 -- 172.21.15.16:0/2844738894 wait complete. 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 -- start start 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.324+0000 7f8a12688740 1 Processor -- start 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a12688740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a12688740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b37ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a12688740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b38b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a12688740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e96b382250 con 0x55e96b37cf70 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a12688740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e96b295050 con 0x55e96b37f680 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a0e04b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a0e84c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b38b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a0e04b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:40124/0 (socket says 172.21.15.16:40124) 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a0e04b640 1 -- 172.21.15.16:0/2684255629 learned_addr learned my addr 172.21.15.16:0/2684255629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b37ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e96b3823d0 con 0x55e96b374dc0 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.325+0000 7f8a12688740 10 client.0 _ll_register_callbacks cb 0x55e96b2d5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a0e84c640 1 -- 172.21.15.16:0/2684255629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 msgr2=0x55e96b37c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a0e84c640 1 -- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 msgr2=0x55e96b37ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b37ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a0e84c640 1 -- 172.21.15.16:0/2684255629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96b04b3c0 con 0x55e96b37cf70 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b38b6c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8a00005fd0 tx=0x7f8a00011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc5fa8f1674a9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f89f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f89f1ffb640 1 -- 172.21.15.16:0/2684255629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00004d20 con 0x55e96b37cf70 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.326+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a04005500 con 0x55e96b37cf70 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.327+0000 7f89f1ffb640 1 -- 172.21.15.16:0/2684255629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f8a0000f940 con 0x55e96b37cf70 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.327+0000 7f89f1ffb640 1 -- 172.21.15.16:0/2684255629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a00025050 con 0x55e96b37cf70 2024-09-15T22:39:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.327+0000 7f8a12688740 10 client.15660 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.327+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e96b38e2c0 con 0x55e96b37cf70 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.327+0000 7f89f1ffb640 1 -- 172.21.15.16:0/2684255629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a04005500 con 0x55e96b37cf70 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.328+0000 7f89f1ffb640 1 -- 172.21.15.16:0/2684255629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a00012c70 con 0x55e96b37cf70 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.328+0000 7f8a12688740 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.328+0000 7f8a12688740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.328+0000 7f8a12688740 20 client.15660 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.328+0000 7f8a12688740 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b38fab0 0x55e96b391f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.328+0000 7f8a12688740 4 client.15660 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.328+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e96b2e7850 con 0x55e96b38fab0 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.329+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b38fab0 0x55e96b391f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.329+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b38fab0 0x55e96b391f20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a04eaa5c9856ac server_cookie=fa48993fff6bdf5d in_seq=0 out_seq=0 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.329+0000 7f89f1ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f89f1ffb640 1 -- 172.21.15.16:0/2684255629 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e96b2e7850 con 0x55e96b38fab0 2024-09-15T22:39:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f89f1ffb640 10 client.15660 handle_command_reply: tid=0 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 client.15660 shutdown 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 msgr2=0x55e96b38b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b38b6c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8a00005fd0 tx=0x7f8a00011b70 comp rx=0 tx=0).stop 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 shutdown_connections 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b38fab0 0x55e96b391f20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b38b6c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b37ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 --2- 172.21.15.16:0/2684255629 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b37c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 >> 172.21.15.16:0/2684255629 conn(0x55e96b370a30 msgr2=0x55e96b36fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f89f1ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 shutdown_connections 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.330+0000 7f8a12688740 1 -- 172.21.15.16:0/2684255629 wait complete. 2024-09-15T22:39:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.331+0000 7f8a12688740 20 client.15660 trim_cache size 0 max 16384 2024-09-15T22:39:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.365+0000 7f8a12688740 1 Processor -- start 2024-09-15T22:39:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a12688740 1 -- start start 2024-09-15T22:39:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a12688740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b374dc0 0x55e96b37c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a12688740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b37cf70 0x55e96b37e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a12688740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37f680 0x55e96b38b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a12688740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b3a20 con 0x55e96b37f680 2024-09-15T22:39:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a12688740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b12f0 con 0x55e96b374dc0 2024-09-15T22:39:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a12688740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081af140 con 0x55e96b37cf70 2024-09-15T22:39:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37f680 0x55e96b38b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37f680 0x55e96b38b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49526/0 (socket says 172.21.15.16:49526) 2024-09-15T22:39:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 -- 172.21.15.16:0/295409172 learned_addr learned my addr 172.21.15.16:0/295409172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b37cf70 0x55e96b37e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b374dc0 0x55e96b37c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 -- 172.21.15.16:0/295409172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b37cf70 msgr2=0x55e96b37e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b37cf70 0x55e96b37e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 -- 172.21.15.16:0/295409172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b374dc0 msgr2=0x55e96b37c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b374dc0 0x55e96b37c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e84c640 1 -- 172.21.15.16:0/295409172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e96b37b4a0 con 0x55e96b37f680 2024-09-15T22:39:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.366+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b374dc0 0x55e96b37c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.367+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37f680 0x55e96b38b4b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8a00004f10 tx=0x7f8a00009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49fbb3f73016bea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.367+0000 7f89f3fff640 1 -- 172.21.15.16:0/295409172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00005380 con 0x55e96b37f680 2024-09-15T22:39:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.367+0000 7f89f3fff640 1 -- 172.21.15.16:0/295409172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a0002e070 con 0x55e96b37f680 2024-09-15T22:39:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.367+0000 7f89f3fff640 1 -- 172.21.15.16:0/295409172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000fe80 con 0x55e96b37f680 2024-09-15T22:39:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 -- 172.21.15.16:0/295409172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37f680 msgr2=0x55e96b38b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37f680 0x55e96b38b4b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8a00004f10 tx=0x7f8a00009e10 comp rx=0 tx=0).stop 2024-09-15T22:39:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 -- 172.21.15.16:0/295409172 shutdown_connections 2024-09-15T22:39:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37f680 0x55e96b38b4b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b37cf70 0x55e96b37e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 --2- 172.21.15.16:0/295409172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b374dc0 0x55e96b37c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 -- 172.21.15.16:0/295409172 >> 172.21.15.16:0/295409172 conn(0x55e96b370a30 msgr2=0x55e96b37dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 -- 172.21.15.16:0/295409172 shutdown_connections 2024-09-15T22:39:21.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.368+0000 7f8a12688740 1 -- 172.21.15.16:0/295409172 wait complete. 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 -- start start 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 Processor -- start 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b3825e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b382b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a0d84a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b382b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a0d84a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b382b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49530/0 (socket says 172.21.15.16:49530) 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a0d84a640 1 -- 172.21.15.16:0/1644025018 learned_addr learned my addr 172.21.15.16:0/1644025018 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b383060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081af7b0 con 0x55e96b37cf70 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b0c20 con 0x55e96b37f680 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.369+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081afe80 con 0x55e96b374dc0 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a12688740 10 client.0 _ll_register_callbacks cb 0x7f8a000097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:21.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b3825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b383060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0d84a640 1 -- 172.21.15.16:0/1644025018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 msgr2=0x55e96b3825e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b3825e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0d84a640 1 -- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 msgr2=0x55e96b383060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b383060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0d84a640 1 -- 172.21.15.16:0/1644025018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e96b3f88f0 con 0x55e96b37cf70 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0e84c640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b383060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f8a0d84a640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b382b20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f89f80c48a0 tx=0x7f89f80bbf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27b5107321fcbe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.370+0000 7f89f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.371+0000 7f89f1ffb640 1 -- 172.21.15.16:0/1644025018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80d5060 con 0x55e96b37cf70 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.371+0000 7f89f1ffb640 1 -- 172.21.15.16:0/1644025018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f89f80b87d0 con 0x55e96b37cf70 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.371+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a0002e070 con 0x55e96b37cf70 2024-09-15T22:39:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.371+0000 7f89f1ffb640 1 -- 172.21.15.16:0/1644025018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89f8049040 con 0x55e96b37cf70 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.371+0000 7f8a12688740 10 client.15666 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.371+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e96b3f9bd0 con 0x55e96b37cf70 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f89f1ffb640 1 -- 172.21.15.16:0/1644025018 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a0002e070 con 0x55e96b37cf70 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f89f1ffb640 1 -- 172.21.15.16:0/1644025018 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f89f804c060 con 0x55e96b37cf70 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a12688740 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a12688740 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a12688740 20 client.15666 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a12688740 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b3fa300 0x55e96b3fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a12688740 4 client.15666 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e96b37cdc0 con 0x55e96b3fa300 2024-09-15T22:39:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b3fa300 0x55e96b3fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f89f1ffb640 10 client.15666 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.372+0000 7f8a0e04b640 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b3fa300 0x55e96b3fc720 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896968aea90be66f server_cookie=cc50b69d5ccac418 in_seq=0 out_seq=0 2024-09-15T22:39:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f89f1ffb640 1 -- 172.21.15.16:0/1644025018 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e96b37cdc0 con 0x55e96b3fa300 2024-09-15T22:39:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f89f1ffb640 10 client.15666 handle_command_reply: tid=0 2024-09-15T22:39:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 client.15666 shutdown 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 msgr2=0x55e96b382b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b382b20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f89f80c48a0 tx=0x7f89f80bbf30 comp rx=0 tx=0).stop 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 shutdown_connections 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e96b3fa300 0x55e96b3fc720 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e96b37f680 0x55e96b383060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e96b37cf70 0x55e96b382b20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 --2- 172.21.15.16:0/1644025018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e96b374dc0 0x55e96b3825e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 >> 172.21.15.16:0/1644025018 conn(0x55e96b370a30 msgr2=0x55e96b36fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f89f1ffb640 0 client.15666 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 shutdown_connections 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 1 -- 172.21.15.16:0/1644025018 wait complete. 2024-09-15T22:39:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.373+0000 7f8a12688740 20 client.15666 trim_cache size 0 max 16384 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: "86738c3b-1bf2-438f-bd58-11b275681bf2": { 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "86738c3b-1bf2-438f-bd58-11b275681bf2", 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:21.379 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:39:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f89f40a36a0 msgr2=0x7f89f40a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f89f40a36a0 0x7f89f40a5b60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005fd0 tx=0x7f8a0400b390 comp rx=0 tx=0).stop 2024-09-15T22:39:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 msgr2=0x7f8a08074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a08074a30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f89f8006010 tx=0x7f89f80042f0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 shutdown_connections 2024-09-15T22:39:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a08106f20 0x7f8a08074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a081027a0 0x7f8a08074a30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f89f40a36a0 0x7f89f40a5b60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 --2- 172.21.15.16:0/131090442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8a08101df0 0x7f8a080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.374+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 >> 172.21.15.16:0/131090442 conn(0x7f8a080fd8f0 msgr2=0x7f8a080fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.375+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 shutdown_connections 2024-09-15T22:39:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:21.375+0000 7f8a102af640 1 -- 172.21.15.16:0/131090442 wait complete. 2024-09-15T22:39:22.207 INFO:tasks.cephfs.filesystem:scrub status for tag:86738c3b-1bf2-438f-bd58-11b275681bf2 - {'path': '/', 'tag': '86738c3b-1bf2-438f-bd58-11b275681bf2', 'options': 'recursive,force'} 2024-09-15T22:39:22.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:22 smithi130 ceph-mon[35742]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:39:22.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:22 smithi016 ceph-mon[31492]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:39:22.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:22 smithi059 ceph-mon[34872]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:39:23.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:23 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:39:23.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:23 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:39:23.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:23 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:39:24.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:24 smithi130 ceph-mon[35742]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-15T22:39:24.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:24 smithi016 ceph-mon[31492]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-15T22:39:24.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:24 smithi059 ceph-mon[34872]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-15T22:39:26.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:26 smithi130 ceph-mon[35742]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-15T22:39:26.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:26 smithi016 ceph-mon[31492]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-15T22:39:26.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:26 smithi059 ceph-mon[34872]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-15T22:39:28.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:28 smithi130 ceph-mon[35742]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:39:28.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:28 smithi016 ceph-mon[31492]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:39:28.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:28 smithi059 ceph-mon[34872]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-15T22:39:30.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:30 smithi130 ceph-mon[35742]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:30.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:30 smithi016 ceph-mon[31492]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:30.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:30 smithi059 ceph-mon[34872]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:32.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:32 smithi130 ceph-mon[35742]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:39:32.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:32 smithi016 ceph-mon[31492]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:39:32.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:32 smithi059 ceph-mon[34872]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:39:33.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:33 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:33.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:33 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:33.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:33 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:34.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:34.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:39:36.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:36 smithi130 ceph-mon[35742]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:36.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:36 smithi016 ceph-mon[31492]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:36.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:36 smithi059 ceph-mon[34872]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:39:38.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:39:38.358 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:38.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:38 smithi016 ceph-mon[31492]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:39:38.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:38 smithi130 ceph-mon[35742]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:39:38.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.892+0000 7fc664593640 1 -- 172.21.15.16:0/1733103137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 msgr2=0x7fc65c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:38.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.892+0000 7fc664593640 1 --2- 172.21.15.16:0/1733103137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 0x7fc65c10ec70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc654009cd0 tx=0x7fc65402a910 comp rx=0 tx=0).stop 2024-09-15T22:39:38.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.893+0000 7fc664593640 1 -- 172.21.15.16:0/1733103137 shutdown_connections 2024-09-15T22:39:38.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.893+0000 7fc664593640 1 --2- 172.21.15.16:0/1733103137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 0x7fc65c10ec70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:38.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.893+0000 7fc664593640 1 --2- 172.21.15.16:0/1733103137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc65c103780 0x7fc65c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:38.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.893+0000 7fc664593640 1 --2- 172.21.15.16:0/1733103137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c102dd0 0x7fc65c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:38.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.893+0000 7fc664593640 1 -- 172.21.15.16:0/1733103137 >> 172.21.15.16:0/1733103137 conn(0x7fc65c0fc7d0 msgr2=0x7fc65c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:38.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.894+0000 7fc664593640 1 -- 172.21.15.16:0/1733103137 shutdown_connections 2024-09-15T22:39:38.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.894+0000 7fc664593640 1 -- 172.21.15.16:0/1733103137 wait complete. 2024-09-15T22:39:38.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 Processor -- start 2024-09-15T22:39:38.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 -- start start 2024-09-15T22:39:38.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc65c102dd0 0x7fc65c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:38.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c103780 0x7fc65c1a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:38.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 0x7fc65c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:38.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c1186a0 con 0x7fc65c102dd0 2024-09-15T22:39:38.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c118520 con 0x7fc65c108300 2024-09-15T22:39:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.895+0000 7fc664593640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c118820 con 0x7fc65c103780 2024-09-15T22:39:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c103780 0x7fc65c1a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c103780 0x7fc65c1a4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:55248/0 (socket says 172.21.15.16:55248) 2024-09-15T22:39:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 -- 172.21.15.16:0/3546855516 learned_addr learned my addr 172.21.15.16:0/3546855516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:38.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc662b30640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 0x7fc65c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc66232f640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc65c102dd0 0x7fc65c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 -- 172.21.15.16:0/3546855516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 msgr2=0x7fc65c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 0x7fc65c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 -- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc65c102dd0 msgr2=0x7fc65c1a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:38.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc65c102dd0 0x7fc65c1a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc661b2e640 1 -- 172.21.15.16:0/3546855516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c19f080 con 0x7fc65c103780 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.896+0000 7fc66232f640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc65c102dd0 0x7fc65c1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.897+0000 7fc661b2e640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c103780 0x7fc65c1a4cd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009800 tx=0x7fc64c009cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.897+0000 7fc6537fe640 1 -- 172.21.15.16:0/3546855516 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c01c070 con 0x7fc65c103780 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.897+0000 7fc6537fe640 1 -- 172.21.15.16:0/3546855516 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc64c00cc60 con 0x7fc65c103780 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.897+0000 7fc6537fe640 1 -- 172.21.15.16:0/3546855516 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0165e0 con 0x7fc65c103780 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.897+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65c19f370 con 0x7fc65c103780 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.897+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65c1ab660 con 0x7fc65c103780 2024-09-15T22:39:38.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.899+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65c1031b0 con 0x7fc65c103780 2024-09-15T22:39:38.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.901+0000 7fc6537fe640 1 -- 172.21.15.16:0/3546855516 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7fc64c017400 con 0x7fc65c103780 2024-09-15T22:39:38.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.902+0000 7fc6537fe640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6480a3550 0x7fc6480a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.902+0000 7fc6537fe640 1 -- 172.21.15.16:0/3546855516 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fc64c0d0790 con 0x7fc65c103780 2024-09-15T22:39:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.903+0000 7fc66232f640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6480a3550 0x7fc6480a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.903+0000 7fc66232f640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6480a3550 0x7fc6480a5a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc65c19fa90 tx=0x7fc658005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:38.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:38.905+0000 7fc6537fe640 1 -- 172.21.15.16:0/3546855516 <== mon.1 v2:172.21.15.130:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc64c025070 con 0x7fc65c103780 2024-09-15T22:39:38.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:38 smithi059 ceph-mon[34872]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:39:39.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.336+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc65c19fe80 con 0x7fc65c103780 2024-09-15T22:39:39.339 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:39:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:39:39.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.337+0000 7fc6537fe640 1 -- 172.21.15.16:0/3546855516 <== mon.1 v2:172.21.15.130:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fc65c19fe80 con 0x7fc65c103780 2024-09-15T22:39:39.339 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:39:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6480a3550 msgr2=0x7fc6480a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6480a3550 0x7fc6480a5a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc65c19fa90 tx=0x7fc658005f90 comp rx=0 tx=0).stop 2024-09-15T22:39:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c103780 msgr2=0x7fc65c1a4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c103780 0x7fc65c1a4cd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009800 tx=0x7fc64c009cc0 comp rx=0 tx=0).stop 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 shutdown_connections 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc65c108300 0x7fc65c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc6480a3550 0x7fc6480a5a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc65c103780 0x7fc65c1a4cd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 --2- 172.21.15.16:0/3546855516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc65c102dd0 0x7fc65c1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 >> 172.21.15.16:0/3546855516 conn(0x7fc65c0fc7d0 msgr2=0x7fc65c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 shutdown_connections 2024-09-15T22:39:39.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:39.340+0000 7fc664593640 1 -- 172.21.15.16:0/3546855516 wait complete. 2024-09-15T22:39:39.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:39 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3546855516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:39.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:39 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3546855516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:40.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:39 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3546855516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:40.081 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 1a891f29 --release --await' 2024-09-15T22:39:40.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 1a891f29 --release --await 2024-09-15T22:39:40.347 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:40.763 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:40 smithi016 ceph-mon[31492]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:40.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:40 smithi059 ceph-mon[34872]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:41.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:40 smithi130 ceph-mon[35742]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.855+0000 7ff897dfc640 1 -- 172.21.15.16:0/823514634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff890102d30 msgr2=0x7ff890103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.855+0000 7ff897dfc640 1 --2- 172.21.15.16:0/823514634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff890102d30 0x7ff890103110 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009930 tx=0x7ff88c02a420 comp rx=0 tx=0).stop 2024-09-15T22:39:41.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.856+0000 7ff897dfc640 1 -- 172.21.15.16:0/823514634 shutdown_connections 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.856+0000 7ff897dfc640 1 --2- 172.21.15.16:0/823514634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8901082a0 0x7ff89010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.856+0000 7ff897dfc640 1 --2- 172.21.15.16:0/823514634 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8901036e0 0x7ff890107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.856+0000 7ff897dfc640 1 --2- 172.21.15.16:0/823514634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff890102d30 0x7ff890103110 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.856+0000 7ff897dfc640 1 -- 172.21.15.16:0/823514634 >> 172.21.15.16:0/823514634 conn(0x7ff8900fc7d0 msgr2=0x7ff8900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.856+0000 7ff897dfc640 1 -- 172.21.15.16:0/823514634 shutdown_connections 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.856+0000 7ff897dfc640 1 -- 172.21.15.16:0/823514634 wait complete. 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 Processor -- start 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 -- start start 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff890102d30 0x7ff8901985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8901036e0 0x7ff890198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8901082a0 0x7ff89019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff890112100 con 0x7ff8901082a0 2024-09-15T22:39:41.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff890111f80 con 0x7ff890102d30 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff897dfc640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff890112280 con 0x7ff8901036e0 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8901082a0 0x7ff89019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8901082a0 0x7ff89019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44572/0 (socket says 172.21.15.16:44572) 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 -- 172.21.15.16:0/1717646850 learned_addr learned my addr 172.21.15.16:0/1717646850 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff895b98640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff890102d30 0x7ff8901985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 -- 172.21.15.16:0/1717646850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8901036e0 msgr2=0x7ff890198b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8901036e0 0x7ff890198b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 -- 172.21.15.16:0/1717646850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff890102d30 msgr2=0x7ff8901985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff890102d30 0x7ff8901985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff896399640 1 -- 172.21.15.16:0/1717646850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89019d590 con 0x7ff8901082a0 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.857+0000 7ff895b98640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff890102d30 0x7ff8901985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.858+0000 7ff896399640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8901082a0 0x7ff89019ceb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff888009fd0 tx=0x7ff88800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.858+0000 7ff886ffd640 1 -- 172.21.15.16:0/1717646850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888017070 con 0x7ff8901082a0 2024-09-15T22:39:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.858+0000 7ff886ffd640 1 -- 172.21.15.16:0/1717646850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff888003e50 con 0x7ff8901082a0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.858+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89019d880 con 0x7ff8901082a0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.858+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8901008c0 con 0x7ff8901082a0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.858+0000 7ff886ffd640 1 -- 172.21.15.16:0/1717646850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888020060 con 0x7ff8901082a0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.861+0000 7ff886ffd640 1 -- 172.21.15.16:0/1717646850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7ff888003970 con 0x7ff8901082a0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.861+0000 7ff886ffd640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff87c0a34d0 0x7ff87c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.861+0000 7ff89a1d5740 1 Processor -- start 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.862+0000 7ff886ffd640 1 -- 172.21.15.16:0/1717646850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff8880d00d0 con 0x7ff8901082a0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.862+0000 7ff895b98640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff87c0a34d0 0x7ff87c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.862+0000 7ff89a1d5740 1 -- start start 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.862+0000 7ff895b98640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff87c0a34d0 0x7ff87c0a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff88c02a940 tx=0x7ff88c005fd0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fd4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fded00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44578/0 (socket says 172.21.15.16:44578) 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 -- 172.21.15.16:0/3519539578 learned_addr learned my addr 172.21.15.16:0/3519539578 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4f2f39620 con 0x55a4f2fd4980 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4f2ee0e50 con 0x55a4f2fd4060 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a4f2f355e0 con 0x55a4f2fdf240 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895b98640 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fd4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff896399640 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 -- 172.21.15.16:0/3519539578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 msgr2=0x55a4f2fe1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 -- 172.21.15.16:0/3519539578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 msgr2=0x55a4f2fd4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fd4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff895397640 1 -- 172.21.15.16:0/3519539578 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f2ed1d40 con 0x55a4f2fd4980 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.863+0000 7ff896399640 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.864+0000 7ff895397640 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fded00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff880009b10 tx=0x7ff88000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e3ea2e1eb99134 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.864+0000 7ff87bfff640 1 -- 172.21.15.16:0/3519539578 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88000ed50 con 0x55a4f2fd4980 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.864+0000 7ff87bfff640 1 -- 172.21.15.16:0/3519539578 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880005020 con 0x55a4f2fd4980 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff87bfff640 1 -- 172.21.15.16:0/3519539578 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880004b60 con 0x55a4f2fd4980 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 msgr2=0x55a4f2fded00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fded00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff880009b10 tx=0x7ff88000e350 comp rx=0 tx=0).stop 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 shutdown_connections 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fded00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3519539578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fd4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 >> 172.21.15.16:0/3519539578 conn(0x55a4f2fd05f0 msgr2=0x55a4f2fd09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 shutdown_connections 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.865+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3519539578 wait complete. 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 -- start start 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 Processor -- start 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fdc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fdc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fdf240 0x55a4f2fdcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a4f2fe1eb0 con 0x55a4f2fd4980 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a4f2ec1f10 con 0x55a4f2fdf240 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a4f2fe2030 con 0x55a4f2fd4060 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff89a1d5740 10 client.0 _ll_register_callbacks cb 0x55a4f2ee5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff895397640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fdc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff895397640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fdc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44590/0 (socket says 172.21.15.16:44590) 2024-09-15T22:39:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.866+0000 7ff895397640 1 -- 172.21.15.16:0/3223808329 learned_addr learned my addr 172.21.15.16:0/3223808329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff895397640 1 -- 172.21.15.16:0/3223808329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 msgr2=0x55a4f2fdc060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff895397640 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fdc060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff895397640 1 -- 172.21.15.16:0/3223808329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fdf240 msgr2=0x55a4f2fdcae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff895397640 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fdf240 0x55a4f2fdcae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff895397640 1 -- 172.21.15.16:0/3223808329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f2c48580 con 0x55a4f2fd4980 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff895397640 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fdc5a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff88000efd0 tx=0x7ff880005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2f321d8c4d69d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff8797fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff8797fa640 1 -- 172.21.15.16:0/3223808329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880004cb0 con 0x55a4f2fd4980 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff8797fa640 1 -- 172.21.15.16:0/3223808329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff880004020 con 0x55a4f2fd4980 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.867+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff880005020 con 0x55a4f2fd4980 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff8797fa640 1 -- 172.21.15.16:0/3223808329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff880030050 con 0x55a4f2fd4980 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 10 client.15684 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a4f2d12960 con 0x55a4f2fd4980 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff8797fa640 1 -- 172.21.15.16:0/3223808329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880005020 con 0x55a4f2fd4980 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff8797fa640 1 -- 172.21.15.16:0/3223808329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff880022460 con 0x55a4f2fd4980 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 10 client.15684 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 20 client.15684 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f2ff1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 4 client.15684 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.868+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4f2f46b90 con 0x55a4f2feec30 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.869+0000 7ff895b98640 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f2ff1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.869+0000 7ff8797fa640 10 client.15684 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.869+0000 7ff895b98640 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f2ff1020 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56482663dc81a402 server_cookie=43ab36032c2271e0 in_seq=0 out_seq=0 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff8797fa640 1 -- 172.21.15.16:0/3223808329 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4f2f46b90 con 0x55a4f2feec30 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff8797fa640 10 client.15684 handle_command_reply: tid=0 2024-09-15T22:39:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 client.15684 shutdown 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 msgr2=0x55a4f2fdc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fdc5a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff88000efd0 tx=0x7ff880005540 comp rx=0 tx=0).stop 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 shutdown_connections 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f2ff1020 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fdf240 0x55a4f2fdcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff8797fa640 0 client.15684 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2fdc5a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3223808329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fdc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.870+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 >> 172.21.15.16:0/3223808329 conn(0x55a4f2fd05f0 msgr2=0x55a4f2fd6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.871+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 shutdown_connections 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.871+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3223808329 wait complete. 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.871+0000 7ff89a1d5740 20 client.15684 trim_cache size 0 max 16384 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.905+0000 7ff89a1d5740 1 Processor -- start 2024-09-15T22:39:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.905+0000 7ff89a1d5740 1 -- start start 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.905+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fcf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.905+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2ff6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2ff6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2ff6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44592/0 (socket says 172.21.15.16:44592) 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2ff6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 -- 172.21.15.16:0/3951025089 learned_addr learned my addr 172.21.15.16:0/3951025089 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901b3b40 con 0x55a4f2fd4980 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901b1410 con 0x55a4f2fd4060 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901af260 con 0x55a4f2fdf240 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 -- 172.21.15.16:0/3951025089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 msgr2=0x55a4f2ff6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2ff6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 -- 172.21.15.16:0/3951025089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 msgr2=0x55a4f2fcf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895b98640 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fcf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fcf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 -- 172.21.15.16:0/3951025089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4f2fe2000 con 0x55a4f2fd4980 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.906+0000 7ff895397640 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2ff6650 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff88000a010 tx=0x7ff880009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6618ef8745c902fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.907+0000 7ff87b7fe640 1 -- 172.21.15.16:0/3951025089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88003a070 con 0x55a4f2fd4980 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.907+0000 7ff87b7fe640 1 -- 172.21.15.16:0/3951025089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880027070 con 0x55a4f2fd4980 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.907+0000 7ff87b7fe640 1 -- 172.21.15.16:0/3951025089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880004070 con 0x55a4f2fd4980 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 msgr2=0x55a4f2ff6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2ff6650 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff88000a010 tx=0x7ff880009310 comp rx=0 tx=0).stop 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 shutdown_connections 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fdf240 0x55a4f2ff6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fd4980 0x55a4f2ff6650 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/3951025089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4060 0x55a4f2fcf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 >> 172.21.15.16:0/3951025089 conn(0x55a4f2fd05f0 msgr2=0x55a4f2fcef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 shutdown_connections 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.908+0000 7ff89a1d5740 1 -- 172.21.15.16:0/3951025089 wait complete. 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.909+0000 7ff89a1d5740 1 -- start start 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.909+0000 7ff89a1d5740 1 Processor -- start 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fe74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4980 0x55a4f2fe79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff89a1d5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff895b98640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fe74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff895b98640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fe74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:55318/0 (socket says 172.21.15.16:55318) 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff896399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff895397640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4980 0x55a4f2fe79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff896399640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44600/0 (socket says 172.21.15.16:44600) 2024-09-15T22:39:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff895397640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4980 0x55a4f2fe79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:48296/0 (socket says 172.21.15.16:48296) 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff895b98640 1 -- 172.21.15.16:0/134961385 learned_addr learned my addr 172.21.15.16:0/134961385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901af8d0 con 0x55a4f2fdf240 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901b0d40 con 0x55a4f2fd4980 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.910+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901affa0 con 0x55a4f2fd4060 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff896399640 1 -- 172.21.15.16:0/134961385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 msgr2=0x55a4f2fe74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff896399640 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fe74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff896399640 1 -- 172.21.15.16:0/134961385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4980 msgr2=0x55a4f2fe79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff896399640 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4980 0x55a4f2fe79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff896399640 1 -- 172.21.15.16:0/134961385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4f2fe49e0 con 0x55a4f2fdf240 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff895b98640 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fe74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff89a1d5740 10 client.0 _ll_register_callbacks cb 0x7ff8800051d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff880027070 con 0x55a4f2fdf240 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff896399640 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe42a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff8880cd340 tx=0x7ff88809bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef195a191699569 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff8797fa640 10 client.15690 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff8797fa640 1 -- 172.21.15.16:0/134961385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88809be00 con 0x55a4f2fdf240 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.911+0000 7ff8797fa640 1 -- 172.21.15.16:0/134961385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff8880cd620 con 0x55a4f2fdf240 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff8797fa640 1 -- 172.21.15.16:0/134961385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8880cd860 con 0x55a4f2fdf240 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a4f3059df0 con 0x55a4f2fdf240 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff8797fa640 1 -- 172.21.15.16:0/134961385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff88809b250 con 0x55a4f2fdf240 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 10 client.15690 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 20 client.15690 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f305c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:41.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 4 client.15690 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1a891f29", "release": true, "await": true}] 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1a891f29", "release": true, "await": true}) -- 0x55a4f2fcb950 con 0x55a4f2feec30 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.912+0000 7ff895b98640 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f305c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.913+0000 7ff8797fa640 10 client.15690 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.913+0000 7ff895b98640 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f305c5a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5942f08b0f2a2b server_cookie=4f913ec8531e17ac in_seq=0 out_seq=0 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.914+0000 7ff8797fa640 1 -- 172.21.15.16:0/134961385 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a4f2fcb950 con 0x55a4f2feec30 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.914+0000 7ff8797fa640 10 client.15690 handle_command_reply: tid=0 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.914+0000 7ff89a1d5740 1 client.15690 shutdown 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fdf240 msgr2=0x55a4f2fe42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe42a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff8880cd340 tx=0x7ff88809bcd0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 shutdown_connections 2024-09-15T22:39:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a4f2feec30 0x55a4f305c5a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 26, 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 4, 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": { 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "1a891f29": { 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "version": 4, 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "age_ref": 0.0, 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-15T22:39:41.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "timeout": 90.0, 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "expiration": 150.2, 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "members": { 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "file:/": { 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "excluded": false, 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "state": { 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RELEASED", 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: "age": 0.0 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:39:41.349 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a4f2fdf240 0x55a4f2fe42a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a4f2fd4980 0x55a4f2fe79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 --2- 172.21.15.16:0/134961385 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a4f2fd4060 0x55a4f2fe74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 >> 172.21.15.16:0/134961385 conn(0x55a4f2fd05f0 msgr2=0x55a4f2fdc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff8797fa640 0 client.15690 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 shutdown_connections 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 1 -- 172.21.15.16:0/134961385 wait complete. 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.915+0000 7ff89a1d5740 20 client.15690 trim_cache size 0 max 16384 2024-09-15T22:39:41.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff87c0a34d0 msgr2=0x7ff87c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff87c0a34d0 0x7ff87c0a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff88c02a940 tx=0x7ff88c005fd0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8901082a0 msgr2=0x7ff89019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8901082a0 0x7ff89019ceb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff888009fd0 tx=0x7ff88800b260 comp rx=0 tx=0).stop 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 shutdown_connections 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8901082a0 0x7ff89019ceb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff87c0a34d0 0x7ff87c0a5990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8901036e0 0x7ff890198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 --2- 172.21.15.16:0/1717646850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff890102d30 0x7ff8901985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 >> 172.21.15.16:0/1717646850 conn(0x7ff8900fc7d0 msgr2=0x7ff890106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 shutdown_connections 2024-09-15T22:39:41.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:40.916+0000 7ff897dfc640 1 -- 172.21.15.16:0/1717646850 wait complete. 2024-09-15T22:39:42.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:42 smithi016 ceph-mon[31492]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:39:42.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:42 smithi059 ceph-mon[34872]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:39:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:42 smithi130 ceph-mon[35742]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:39:44.020 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1a891f29', total seconds elapsed: 36.2 2024-09-15T22:39:44.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:44 smithi016 ceph-mon[31492]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:44.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:44 smithi059 ceph-mon[34872]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:45.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:44 smithi130 ceph-mon[35742]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:45.283 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 92% (11888/12889) Updating files: 93% (11987/12889) Updating files: 94% (12116/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T22:39:45.888 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-15T22:39:45.889 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-15T22:39:45.893 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T22:39:45.935 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-15T22:39:45.937 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T22:39:45.937 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-15T22:39:46.696 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-15T22:39:46.780 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-15T22:39:46.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:46 smithi016 ceph-mon[31492]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:46.823 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T22:39:46.824 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T22:39:46.824 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-15T22:39:46.865 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T22:39:46.868 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:39:46.869 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T22:39:46.941 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-09-15T22:39:46.942 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-09-15T22:39:46.942 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-09-15T22:39:46.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:46 smithi059 ceph-mon[34872]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:47.004 INFO:tasks.workunit.client.0.smithi016.stdout:test large (16MB) dio write 2024-09-15T22:39:47.005 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'test large (16MB) dio write' 2024-09-15T22:39:47.005 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-09-15T22:39:47.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:46 smithi130 ceph-mon[35742]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:47.086 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records in 2024-09-15T22:39:47.087 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records out 2024-09-15T22:39:47.087 INFO:tasks.workunit.client.0.smithi016.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0795029 s, 211 MB/s 2024-09-15T22:39:47.087 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-15T22:39:47.088 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-15T22:39:47.088 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T22:39:47.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T22:39:47.162 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-09-15T22:39:47.163 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-09-15T22:39:48.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:48 smithi016 ceph-mon[31492]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:39:48.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:48 smithi059 ceph-mon[34872]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:39:49.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:48 smithi130 ceph-mon[35742]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:39:50.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:50 smithi016 ceph-mon[31492]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:50.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:50 smithi059 ceph-mon[34872]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:51.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:50 smithi130 ceph-mon[35742]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:39:52.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:39:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:52.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:52 smithi016 ceph-mon[31492]: pgmap v46: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 478 op/s 2024-09-15T22:39:52.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:52 smithi059 ceph-mon[34872]: pgmap v46: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 478 op/s 2024-09-15T22:39:53.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:52 smithi130 ceph-mon[35742]: pgmap v46: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 478 op/s 2024-09-15T22:39:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.222+0000 7f2948151640 1 -- 172.21.15.16:0/3485196340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 msgr2=0x7f2940110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.222+0000 7f2948151640 1 --2- 172.21.15.16:0/3485196340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f2940110ca0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2938009ca0 tx=0x7f293802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.222+0000 7f2948151640 1 -- 172.21.15.16:0/3485196340 shutdown_connections 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.222+0000 7f2948151640 1 --2- 172.21.15.16:0/3485196340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f2940110ca0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.222+0000 7f2948151640 1 --2- 172.21.15.16:0/3485196340 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 0x7f29401089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.222+0000 7f2948151640 1 --2- 172.21.15.16:0/3485196340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2940103ff0 0x7f29401043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.222+0000 7f2948151640 1 -- 172.21.15.16:0/3485196340 >> 172.21.15.16:0/3485196340 conn(0x7f29400ffab0 msgr2=0x7f2940101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.223+0000 7f2948151640 1 -- 172.21.15.16:0/3485196340 shutdown_connections 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.223+0000 7f2948151640 1 -- 172.21.15.16:0/3485196340 wait complete. 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.223+0000 7f2948151640 1 Processor -- start 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.223+0000 7f2948151640 1 -- start start 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f2948151640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2940103ff0 0x7f294019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f2948151640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 0x7f294019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f2948151640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f294019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f2948151640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2940114130 con 0x7f2940109120 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f2948151640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2940113fb0 con 0x7f2940103ff0 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f2948151640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29401142b0 con 0x7f29401049a0 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 0x7f294019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29466ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f294019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 0x7f294019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:44318/0 (socket says 172.21.15.16:44318) 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 -- 172.21.15.16:0/2382446952 learned_addr learned my addr 172.21.15.16:0/2382446952 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29466ee640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f294019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39072/0 (socket says 172.21.15.16:39072) 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f2945eed640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2940103ff0 0x7f294019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 -- 172.21.15.16:0/2382446952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2940103ff0 msgr2=0x7f294019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2940103ff0 0x7f294019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 -- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 msgr2=0x7f294019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f294019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29456ec640 1 -- 172.21.15.16:0/2382446952 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294019f640 con 0x7f29401049a0 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.224+0000 7f29466ee640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f294019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.225+0000 7f29456ec640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 0x7f294019abd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f293000e9f0 tx=0x7f293000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.225+0000 7f2936ffd640 1 -- 172.21.15.16:0/2382446952 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293000cd80 con 0x7f29401049a0 2024-09-15T22:39:53.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.225+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294019f930 con 0x7f29401049a0 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.225+0000 7f2936ffd640 1 -- 172.21.15.16:0/2382446952 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2930010910 con 0x7f29401049a0 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.225+0000 7f2936ffd640 1 -- 172.21.15.16:0/2382446952 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300188f0 con 0x7f29401049a0 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.225+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29401a7210 con 0x7f29401049a0 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.227+0000 7f294a52a740 1 Processor -- start 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.227+0000 7f2936ffd640 1 -- 172.21.15.16:0/2382446952 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7f2930010ac0 con 0x7f29401049a0 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f294a52a740 1 -- start start 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f294a52a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e135c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f294a52a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135cdc0 0x55b9e1367140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f294a52a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1369a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f294a52a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b9e12c22e0 con 0x55b9e135c4a0 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f294a52a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b9e1269ce0 con 0x55b9e1367680 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f294a52a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b9e12be2a0 con 0x55b9e135cdc0 2024-09-15T22:39:53.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135cdc0 0x55b9e1367140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135cdc0 0x55b9e1367140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:44334/0 (socket says 172.21.15.16:44334) 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 -- 172.21.15.16:0/1597133677 learned_addr learned my addr 172.21.15.16:0/1597133677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f29466ee640 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e135c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f29456ec640 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1369a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 -- 172.21.15.16:0/1597133677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 msgr2=0x55b9e1369a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1369a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 -- 172.21.15.16:0/1597133677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 msgr2=0x55b9e135c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e135c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2945eed640 1 -- 172.21.15.16:0/1597133677 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e124e440 con 0x55b9e135cdc0 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f29456ec640 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1369a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.228+0000 7f2936ffd640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f292c0a34d0 0x7f292c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.229+0000 7f29466ee640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f292c0a34d0 0x7f292c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.229+0000 7f2945eed640 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135cdc0 0x55b9e1367140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f293c00ac20 tx=0x7f293c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0c23c831d1fdd14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.229+0000 7f2936ffd640 1 -- 172.21.15.16:0/2382446952 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f29300d0640 con 0x7f29401049a0 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.229+0000 7f292bfff640 1 -- 172.21.15.16:0/1597133677 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c01d070 con 0x55b9e135cdc0 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.229+0000 7f292bfff640 1 -- 172.21.15.16:0/1597133677 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c018b20 con 0x55b9e135cdc0 2024-09-15T22:39:53.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.229+0000 7f292bfff640 1 -- 172.21.15.16:0/1597133677 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c010c00 con 0x55b9e135cdc0 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 -- 172.21.15.16:0/1597133677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135cdc0 msgr2=0x55b9e1367140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135cdc0 0x55b9e1367140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f293c00ac20 tx=0x7f293c00f790 comp rx=0 tx=0).stop 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 -- 172.21.15.16:0/1597133677 shutdown_connections 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1369a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135cdc0 0x55b9e1367140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 --2- 172.21.15.16:0/1597133677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e135c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 -- 172.21.15.16:0/1597133677 >> 172.21.15.16:0/1597133677 conn(0x55b9e1358a30 msgr2=0x55b9e1358e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f29466ee640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f292c0a34d0 0x7f292c0a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2938004af0 tx=0x7f2938004a60 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:53.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 -- 172.21.15.16:0/1597133677 shutdown_connections 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 -- 172.21.15.16:0/1597133677 wait complete. 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 -- start start 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.230+0000 7f294a52a740 1 Processor -- start 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f294a52a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135c4a0 0x55b9e1366f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f294a52a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135cdc0 0x55b9e1364240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f294a52a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1364780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f294a52a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b9e136a250 con 0x55b9e135cdc0 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f294a52a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b9e127d050 con 0x55b9e1367680 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f294a52a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b9e136a3d0 con 0x55b9e135c4a0 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f29456ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135cdc0 0x55b9e1364240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f2945eed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135c4a0 0x55b9e1366f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f29456ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135cdc0 0x55b9e1364240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39090/0 (socket says 172.21.15.16:39090) 2024-09-15T22:39:53.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f29456ec640 1 -- 172.21.15.16:0/2174950361 learned_addr learned my addr 172.21.15.16:0/2174950361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f294a52a740 10 client.0 _ll_register_callbacks cb 0x55b9e12bd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.231+0000 7f29466ee640 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1364780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29456ec640 1 -- 172.21.15.16:0/2174950361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135c4a0 msgr2=0x55b9e1366f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29456ec640 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135c4a0 0x55b9e1366f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29456ec640 1 -- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 msgr2=0x55b9e1364780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29456ec640 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1364780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29456ec640 1 -- 172.21.15.16:0/2174950361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e10333c0 con 0x55b9e135cdc0 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29456ec640 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135cdc0 0x55b9e1364240 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f293000c6a0 tx=0x7f293009c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf263d3a72660762 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29297fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:53.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29297fa640 1 -- 172.21.15.16:0/2174950361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300d0480 con 0x55b9e135cdc0 2024-09-15T22:39:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29297fa640 1 -- 172.21.15.16:0/2174950361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f29300cdb90 con 0x55b9e135cdc0 2024-09-15T22:39:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293c018b20 con 0x55b9e135cdc0 2024-09-15T22:39:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29297fa640 1 -- 172.21.15.16:0/2174950361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f29300cdd90 con 0x55b9e135cdc0 2024-09-15T22:39:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f294a52a740 10 client.15702 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9e1375b80 con 0x55b9e135cdc0 2024-09-15T22:39:53.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29297fa640 1 -- 172.21.15.16:0/2174950361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c018b20 con 0x55b9e135cdc0 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.232+0000 7f29297fa640 1 -- 172.21.15.16:0/2174950361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f29300d0af0 con 0x55b9e135cdc0 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f294a52a740 10 client.15702 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f294a52a740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f294a52a740 20 client.15702 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f294a52a740 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e1376ed0 0x55b9e13792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f294a52a740 4 client.15702 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9e12cf850 con 0x55b9e1376ed0 2024-09-15T22:39:53.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f2945eed640 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e1376ed0 0x55b9e13792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f2945eed640 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e1376ed0 0x55b9e13792c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74afea0d3fa05057 server_cookie=34cb355bbf0c3a0c in_seq=0 out_seq=0 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.233+0000 7f29297fa640 10 client.15702 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f29297fa640 1 -- 172.21.15.16:0/2174950361 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9e12cf850 con 0x55b9e1376ed0 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f29297fa640 10 client.15702 handle_command_reply: tid=0 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 client.15702 shutdown 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135cdc0 msgr2=0x55b9e1364240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135cdc0 0x55b9e1364240 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f293000c6a0 tx=0x7f293009c420 comp rx=0 tx=0).stop 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 shutdown_connections 2024-09-15T22:39:53.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e1376ed0 0x55b9e13792c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e1367680 0x55b9e1364780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135cdc0 0x55b9e1364240 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 --2- 172.21.15.16:0/2174950361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e135c4a0 0x55b9e1366f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 >> 172.21.15.16:0/2174950361 conn(0x55b9e1358a30 msgr2=0x55b9e135e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f29297fa640 0 client.15702 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 shutdown_connections 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 1 -- 172.21.15.16:0/2174950361 wait complete. 2024-09-15T22:39:53.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.234+0000 7f294a52a740 20 client.15702 trim_cache size 0 max 16384 2024-09-15T22:39:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.252+0000 7f294a52a740 1 Processor -- start 2024-09-15T22:39:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f294a52a740 1 -- start start 2024-09-15T22:39:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f294a52a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f294a52a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1378ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f294a52a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e135f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f294a52a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29401b3a30 con 0x55b9e135c4a0 2024-09-15T22:39:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f294a52a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29401b1300 con 0x55b9e135cdc0 2024-09-15T22:39:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f294a52a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29401af150 con 0x55b9e1367680 2024-09-15T22:39:53.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e135f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f2945eed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e135f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:44344/0 (socket says 172.21.15.16:44344) 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 -- 172.21.15.16:0/4230436028 learned_addr learned my addr 172.21.15.16:0/4230436028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29456ec640 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1378ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 -- 172.21.15.16:0/4230436028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 msgr2=0x55b9e1378ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1378ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 -- 172.21.15.16:0/4230436028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 msgr2=0x55b9e13734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.253+0000 7f29466ee640 1 -- 172.21.15.16:0/4230436028 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e13631f0 con 0x55b9e1367680 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.254+0000 7f29466ee640 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e135f740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2938009ca0 tx=0x7f2938044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e1918a0cadf586 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:53.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.254+0000 7f292b7fe640 1 -- 172.21.15.16:0/4230436028 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938058070 con 0x55b9e1367680 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.254+0000 7f292b7fe640 1 -- 172.21.15.16:0/4230436028 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2938036040 con 0x55b9e1367680 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.254+0000 7f292b7fe640 1 -- 172.21.15.16:0/4230436028 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29380536e0 con 0x55b9e1367680 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 -- 172.21.15.16:0/4230436028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 msgr2=0x55b9e135f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e135f740 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2938009ca0 tx=0x7f2938044000 comp rx=0 tx=0).stop 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 -- 172.21.15.16:0/4230436028 shutdown_connections 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e135f740 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1378ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 --2- 172.21.15.16:0/4230436028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 -- 172.21.15.16:0/4230436028 >> 172.21.15.16:0/4230436028 conn(0x55b9e1358a30 msgr2=0x55b9e135f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:53.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 -- 172.21.15.16:0/4230436028 shutdown_connections 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 -- 172.21.15.16:0/4230436028 wait complete. 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 -- start start 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.255+0000 7f294a52a740 1 Processor -- start 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f294a52a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f294a52a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1361720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f294a52a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e1361c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f294a52a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29401af7c0 con 0x55b9e135c4a0 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f294a52a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29401b0c30 con 0x55b9e135cdc0 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f294a52a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29401afe90 con 0x55b9e1367680 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f294a52a740 10 client.0 _ll_register_callbacks cb 0x7f2930009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39102/0 (socket says 172.21.15.16:39102) 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 -- 172.21.15.16:0/3340198660 learned_addr learned my addr 172.21.15.16:0/3340198660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f29466ee640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e1361c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f29456ec640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1361720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 -- 172.21.15.16:0/3340198660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 msgr2=0x55b9e1361c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e1361c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 -- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 msgr2=0x55b9e1361720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1361720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 -- 172.21.15.16:0/3340198660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e1367300 con 0x55b9e135c4a0 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f29466ee640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e1361c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f2945eed640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13611e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55b9e1359890 tx=0x7f293c00a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3277147e05e91d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.256+0000 7f29297fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f29297fa640 1 -- 172.21.15.16:0/3340198660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c031070 con 0x55b9e135c4a0 2024-09-15T22:39:53.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f29297fa640 1 -- 172.21.15.16:0/3340198660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f293c005450 con 0x55b9e135c4a0 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2938036040 con 0x55b9e135c4a0 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f29297fa640 1 -- 172.21.15.16:0/3340198660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f293c03d050 con 0x55b9e135c4a0 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f294a52a740 10 client.15708 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9e13e1bf0 con 0x55b9e135c4a0 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f29297fa640 1 -- 172.21.15.16:0/3340198660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2938036040 con 0x55b9e135c4a0 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f29297fa640 1 -- 172.21.15.16:0/3340198660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f293c018710 con 0x55b9e135c4a0 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f294a52a740 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f294a52a740 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f294a52a740 20 client.15708 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.257+0000 7f294a52a740 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e13e2040 0x55b9e13e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f294a52a740 4 client.15708 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9e12c9d60 con 0x55b9e13e2040 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f29456ec640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e13e2040 0x55b9e13e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:53.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f29297fa640 10 client.15708 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f29456ec640 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e13e2040 0x55b9e13e4430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb7c4e5b6e1b021 server_cookie=f579e1a453b634fc in_seq=0 out_seq=0 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f29297fa640 1 -- 172.21.15.16:0/3340198660 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b9e12c9d60 con 0x55b9e13e2040 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f29297fa640 10 client.15708 handle_command_reply: tid=0 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.258+0000 7f294a52a740 1 client.15708 shutdown 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 msgr2=0x55b9e13611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13611e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x55b9e1359890 tx=0x7f293c00a8b0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 shutdown_connections 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b9e13e2040 0x55b9e13e4430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b9e1367680 0x55b9e1361c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f29297fa640 0 client.15708 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b9e135cdc0 0x55b9e1361720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 --2- 172.21.15.16:0/3340198660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b9e135c4a0 0x55b9e13611e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 >> 172.21.15.16:0/3340198660 conn(0x55b9e1358a30 msgr2=0x55b9e1357b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:53.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 shutdown_connections 2024-09-15T22:39:53.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 1 -- 172.21.15.16:0/3340198660 wait complete. 2024-09-15T22:39:53.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.259+0000 7f294a52a740 20 client.15708 trim_cache size 0 max 16384 2024-09-15T22:39:53.279 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:39:53.279 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:39:53.279 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:39:53.279 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:39:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f292c0a34d0 msgr2=0x7f292c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f292c0a34d0 0x7f292c0a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2938004af0 tx=0x7f2938004a60 comp rx=0 tx=0).stop 2024-09-15T22:39:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 msgr2=0x7f294019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 0x7f294019abd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f293000e9f0 tx=0x7f293000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 shutdown_connections 2024-09-15T22:39:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2940109120 0x7f294019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f292c0a34d0 0x7f292c0a5990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f29401049a0 0x7f294019abd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 --2- 172.21.15.16:0/2382446952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2940103ff0 0x7f294019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 >> 172.21.15.16:0/2382446952 conn(0x7f29400ffab0 msgr2=0x7f2940100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 shutdown_connections 2024-09-15T22:39:53.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:53.260+0000 7f2948151640 1 -- 172.21.15.16:0/2382446952 wait complete. 2024-09-15T22:39:53.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:39:53.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:39:54.254 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:54.663 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:54 smithi016 ceph-mon[31492]: pgmap v47: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-09-15T22:39:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 -- 172.21.15.16:0/4050343530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 msgr2=0x7f0c041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 --2- 172.21.15.16:0/4050343530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 0x7f0c041067e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4009900 tx=0x7f0bf402a410 comp rx=0 tx=0).stop 2024-09-15T22:39:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 -- 172.21.15.16:0/4050343530 shutdown_connections 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 --2- 172.21.15.16:0/4050343530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c04106f10 0x7f0c0410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 --2- 172.21.15.16:0/4050343530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 0x7f0c041067e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 --2- 172.21.15.16:0/4050343530 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 0x7f0c041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 -- 172.21.15.16:0/4050343530 >> 172.21.15.16:0/4050343530 conn(0x7f0c040fd8c0 msgr2=0x7f0c040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 -- 172.21.15.16:0/4050343530 shutdown_connections 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.747+0000 7f0c0b599640 1 -- 172.21.15.16:0/4050343530 wait complete. 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 Processor -- start 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 -- start start 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 0x7f0c04198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 0x7f0c041989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c04106f10 0x7f0c0419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111f70 con 0x7f0c04102790 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111df0 con 0x7f0c04106f10 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c0b599640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041120f0 con 0x7f0c04101de0 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c08b34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 0x7f0c041989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c09335640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 0x7f0c04198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c09335640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 0x7f0c04198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:44372/0 (socket says 172.21.15.16:44372) 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c09335640 1 -- 172.21.15.16:0/3167554676 learned_addr learned my addr 172.21.15.16:0/3167554676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c09b36640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c04106f10 0x7f0c0419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c08b34640 1 -- 172.21.15.16:0/3167554676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 msgr2=0x7f0c04198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c08b34640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 0x7f0c04198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c08b34640 1 -- 172.21.15.16:0/3167554676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c04106f10 msgr2=0x7f0c0419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c08b34640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c04106f10 0x7f0c0419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c08b34640 1 -- 172.21.15.16:0/3167554676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0419d440 con 0x7f0c04102790 2024-09-15T22:39:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.748+0000 7f0c09335640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 0x7f0c04198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.749+0000 7f0c08b34640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 0x7f0c041989d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005940 tx=0x7f0bf4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.749+0000 7f0bfa7fc640 1 -- 172.21.15.16:0/3167554676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf403f030 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.749+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0419d6d0 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.749+0000 7f0bfa7fc640 1 -- 172.21.15.16:0/3167554676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf4003ad0 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.749+0000 7f0bfa7fc640 1 -- 172.21.15.16:0/3167554676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4035070 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.749+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041a5010 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.750+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c041021c0 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.751+0000 7f0bfa7fc640 1 -- 172.21.15.16:0/3167554676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7f0bf4042030 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.752+0000 7f0bfa7fc640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0bf00a34d0 0x7f0bf00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.752+0000 7f0c09335640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0bf00a34d0 0x7f0bf00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.752+0000 7f0bfa7fc640 1 -- 172.21.15.16:0/3167554676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0bf40f0b80 con 0x7f0c04102790 2024-09-15T22:39:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.753+0000 7f0c09335640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0bf00a34d0 0x7f0bf00a5990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c00011550 tx=0x7f0c00005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.756+0000 7f0bfa7fc640 1 -- 172.21.15.16:0/3167554676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bf40bd080 con 0x7f0c04102790 2024-09-15T22:39:54.830 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:54 smithi130 ceph-mon[35742]: pgmap v47: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-09-15T22:39:54.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:54 smithi059 ceph-mon[34872]: pgmap v47: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-09-15T22:39:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.980+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c0410a1b0 con 0x7f0c04102790 2024-09-15T22:39:54.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.981+0000 7f0bfa7fc640 1 -- 172.21.15.16:0/3167554676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f0bf40bc980 con 0x7f0c04102790 2024-09-15T22:39:54.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:39:54.984 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:39:54.984 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:39:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0bf00a34d0 msgr2=0x7f0bf00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0bf00a34d0 0x7f0bf00a5990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c00011550 tx=0x7f0c00005f90 comp rx=0 tx=0).stop 2024-09-15T22:39:54.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 msgr2=0x7f0c041989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 0x7f0c041989d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4005940 tx=0x7f0bf4004270 comp rx=0 tx=0).stop 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 shutdown_connections 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c04106f10 0x7f0c0419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c04102790 0x7f0c041989d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0bf00a34d0 0x7f0bf00a5990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 --2- 172.21.15.16:0/3167554676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c04101de0 0x7f0c04198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 >> 172.21.15.16:0/3167554676 conn(0x7f0c040fd8c0 msgr2=0x7f0c040fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 shutdown_connections 2024-09-15T22:39:54.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:54.984+0000 7f0c0b599640 1 -- 172.21.15.16:0/3167554676 wait complete. 2024-09-15T22:39:55.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:39:55.840 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:55 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3167554676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:55.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:55 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3167554676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:56.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:55 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3167554676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:39:56.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 -- 172.21.15.16:0/3616705014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 msgr2=0x7f06c410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 --2- 172.21.15.16:0/3616705014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c410ec40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009930 tx=0x7f06bc02a420 comp rx=0 tx=0).stop 2024-09-15T22:39:56.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 -- 172.21.15.16:0/3616705014 shutdown_connections 2024-09-15T22:39:56.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 --2- 172.21.15.16:0/3616705014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c410ec40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 --2- 172.21.15.16:0/3616705014 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f06c41036e0 0x7f06c4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 --2- 172.21.15.16:0/3616705014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c4102d30 0x7f06c4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 -- 172.21.15.16:0/3616705014 >> 172.21.15.16:0/3616705014 conn(0x7f06c40fc7d0 msgr2=0x7f06c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 -- 172.21.15.16:0/3616705014 shutdown_connections 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.424+0000 7f06c8c38640 1 -- 172.21.15.16:0/3616705014 wait complete. 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 Processor -- start 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 -- start start 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f06c4102d30 0x7f06c4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41036e0 0x7f06c4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41120a0 con 0x7f06c41082a0 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4111f20 con 0x7f06c41036e0 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c8c38640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4112220 con 0x7f06c4102d30 2024-09-15T22:39:56.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39136/0 (socket says 172.21.15.16:39136) 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c2d9d640 1 -- 172.21.15.16:0/2359279016 learned_addr learned my addr 172.21.15.16:0/2359279016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c259c640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f06c4102d30 0x7f06c4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41036e0 0x7f06c4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.425+0000 7f06c2d9d640 1 -- 172.21.15.16:0/2359279016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f06c4102d30 msgr2=0x7f06c4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06c2d9d640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f06c4102d30 0x7f06c4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06c2d9d640 1 -- 172.21.15.16:0/2359279016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41036e0 msgr2=0x7f06c4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06c2d9d640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41036e0 0x7f06c4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06c2d9d640 1 -- 172.21.15.16:0/2359279016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41a6990 con 0x7f06c41082a0 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06c2d9d640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c4101410 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f06bc005ab0 tx=0x7f06bc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06b37fe640 1 -- 172.21.15.16:0/2359279016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc03f030 con 0x7f06c41082a0 2024-09-15T22:39:56.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41a6c20 con 0x7f06c41082a0 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06b37fe640 1 -- 172.21.15.16:0/2359279016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06bc0044c0 con 0x7f06c41082a0 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06b37fe640 1 -- 172.21.15.16:0/2359279016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc035070 con 0x7f06c41082a0 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.426+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41a7150 con 0x7f06c41082a0 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.428+0000 7f06b37fe640 1 -- 172.21.15.16:0/2359279016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 25) ==== 138588+0+0 (secure 0 0 0) 0x7f06bc042030 con 0x7f06c41082a0 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 Processor -- start 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 -- start start 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec8823090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8823a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8838f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fec87892f0 con 0x55fec88235d0 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fec8730cf0 con 0x55fec8836b80 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06cb011740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55fec87852b0 con 0x55fec8824b40 2024-09-15T22:39:56.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8823a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec8823090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06b37fe640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06ac0a34d0 0x7f06ac0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8823a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39144/0 (socket says 172.21.15.16:39144) 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06c259c640 1 -- 172.21.15.16:0/3038580470 learned_addr learned my addr 172.21.15.16:0/3038580470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec8823090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:44396/0 (socket says 172.21.15.16:44396) 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.429+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8838f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c2d9d640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06ac0a34d0 0x7f06ac0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c1d9b640 1 -- 172.21.15.16:0/3038580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 msgr2=0x55fec8838f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8838f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c1d9b640 1 -- 172.21.15.16:0/3038580470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 msgr2=0x55fec8823a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06b37fe640 1 -- 172.21.15.16:0/2359279016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f06bc0f05b0 con 0x7f06c41082a0 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8823a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c1d9b640 1 -- 172.21.15.16:0/3038580470 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec87153a0 con 0x55fec8824b40 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c259c640 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8823a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec8823090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f06c4102420 tx=0x7f06b8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f4201a5cff8db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06c2d9d640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06ac0a34d0 0x7f06ac0a5990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0ed880 tx=0x7f06bc031400 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06b0ff9640 1 -- 172.21.15.16:0/3038580470 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004b20 con 0x55fec8824b40 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06b0ff9640 1 -- 172.21.15.16:0/3038580470 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b8004cc0 con 0x55fec8824b40 2024-09-15T22:39:56.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.430+0000 7f06b0ff9640 1 -- 172.21.15.16:0/3038580470 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8005490 con 0x55fec8824b40 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 -- 172.21.15.16:0/3038580470 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 msgr2=0x55fec8823090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec8823090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f06c4102420 tx=0x7f06b8004ab0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 -- 172.21.15.16:0/3038580470 shutdown_connections 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8838f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8823a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 --2- 172.21.15.16:0/3038580470 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec8823090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 -- 172.21.15.16:0/3038580470 >> 172.21.15.16:0/3038580470 conn(0x55fec87ec310 msgr2=0x55fec881fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:56.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 -- 172.21.15.16:0/3038580470 shutdown_connections 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 -- 172.21.15.16:0/3038580470 wait complete. 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.431+0000 7f06cb011740 1 -- start start 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 1 Processor -- start 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec8836540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec88337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8833de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fec8839880 con 0x55fec8824b40 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fec8744060 con 0x55fec8836b80 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55fec8839a00 con 0x55fec88235d0 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06cb011740 10 client.0 _ll_register_callbacks cb 0x55fec84978c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:56.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec88337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec8836540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec88337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39158/0 (socket says 172.21.15.16:39158) 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 -- 172.21.15.16:0/1156795436 learned_addr learned my addr 172.21.15.16:0/1156795436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8833de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 -- 172.21.15.16:0/1156795436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 msgr2=0x55fec8836540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec8836540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 -- 172.21.15.16:0/1156795436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 msgr2=0x55fec8833de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8833de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 -- 172.21.15.16:0/1156795436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec84fa3c0 con 0x55fec8824b40 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06c259c640 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec88337f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f06b4010a50 tx=0x7f06b4010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7481f60dae7c09cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.432+0000 7f06a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:56.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1156795436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b400fbd0 con 0x55fec8824b40 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b8004cc0 con 0x55fec8824b40 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1156795436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f06b400fd70 con 0x55fec8824b40 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1156795436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06b4024050 con 0x55fec8824b40 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 10 client.15726 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fec8845370 con 0x55fec8824b40 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1156795436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b8004cc0 con 0x55fec8824b40 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1156795436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f06b400e650 con 0x55fec8824b40 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:56.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 10 client.15726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 20 client.15726 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec8846880 0x55fec8848ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 4 client.15726 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.433+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fec8796860 con 0x55fec8846880 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.434+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec8846880 0x55fec8848ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.434+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec8846880 0x55fec8848ca0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7a5944dd0c498b5 server_cookie=cb77195342662c2f in_seq=0 out_seq=0 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.434+0000 7f06a9ffb640 10 client.15726 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1156795436 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fec8796860 con 0x55fec8846880 2024-09-15T22:39:56.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06a9ffb640 10 client.15726 handle_command_reply: tid=0 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 client.15726 shutdown 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 msgr2=0x55fec88337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec88337f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f06b4010a50 tx=0x7f06b4010f10 comp rx=0 tx=0).stop 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 shutdown_connections 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec8846880 0x55fec8848ca0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec8833de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec88337f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 --2- 172.21.15.16:0/1156795436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec8836540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 >> 172.21.15.16:0/1156795436 conn(0x55fec87ec310 msgr2=0x55fec8825930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:56.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06a9ffb640 0 client.15726 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 shutdown_connections 2024-09-15T22:39:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 1 -- 172.21.15.16:0/1156795436 wait complete. 2024-09-15T22:39:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.435+0000 7f06cb011740 20 client.15726 trim_cache size 0 max 16384 2024-09-15T22:39:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.448+0000 7f06cb011740 1 Processor -- start 2024-09-15T22:39:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.448+0000 7f06cb011740 1 -- start start 2024-09-15T22:39:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.448+0000 7f06cb011740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8842b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06cb011740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec88327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06cb011740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec884e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06cb011740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41af610 con 0x55fec88235d0 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06cb011740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41acee0 con 0x55fec8836b80 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06cb011740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41aad30 con 0x55fec8824b40 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8842b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8842b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39172/0 (socket says 172.21.15.16:39172) 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 -- 172.21.15.16:0/1859093717 learned_addr learned my addr 172.21.15.16:0/1859093717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c259c640 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec88327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 -- 172.21.15.16:0/1859093717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 msgr2=0x55fec88327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec88327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 -- 172.21.15.16:0/1859093717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 msgr2=0x55fec884e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec884e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 -- 172.21.15.16:0/1859093717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fec8820960 con 0x55fec88235d0 2024-09-15T22:39:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8842b50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011870 tx=0x7f06b8011800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4ffb8bbcdadd5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06abfff640 1 -- 172.21.15.16:0/1859093717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004c40 con 0x55fec88235d0 2024-09-15T22:39:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06abfff640 1 -- 172.21.15.16:0/1859093717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b8004de0 con 0x55fec88235d0 2024-09-15T22:39:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.449+0000 7f06abfff640 1 -- 172.21.15.16:0/1859093717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80054b0 con 0x55fec88235d0 2024-09-15T22:39:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 -- 172.21.15.16:0/1859093717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 msgr2=0x55fec8842b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8842b50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011870 tx=0x7f06b8011800 comp rx=0 tx=0).stop 2024-09-15T22:39:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 -- 172.21.15.16:0/1859093717 shutdown_connections 2024-09-15T22:39:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec884e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec8824b40 0x55fec88327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 --2- 172.21.15.16:0/1859093717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec88235d0 0x55fec8842b50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 -- 172.21.15.16:0/1859093717 >> 172.21.15.16:0/1859093717 conn(0x55fec87ec310 msgr2=0x55fec88320c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 -- 172.21.15.16:0/1859093717 shutdown_connections 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 -- 172.21.15.16:0/1859093717 wait complete. 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 -- start start 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.450+0000 7f06cb011740 1 Processor -- start 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06cb011740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec883ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06cb011740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec883b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06cb011740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec883b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06cb011740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41ab3a0 con 0x55fec8824b40 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06cb011740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41ac810 con 0x55fec8836b80 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06cb011740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41aba70 con 0x55fec88235d0 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06cb011740 10 client.0 _ll_register_callbacks cb 0x7f06b400a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec883b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec883ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec883b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39174/0 (socket says 172.21.15.16:39174) 2024-09-15T22:39:56.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 -- 172.21.15.16:0/1627214844 learned_addr learned my addr 172.21.15.16:0/1627214844 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec883b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 -- 172.21.15.16:0/1627214844 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 msgr2=0x55fec883ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec883ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 -- 172.21.15.16:0/1627214844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 msgr2=0x55fec883b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec883b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.451+0000 7f06c259c640 1 -- 172.21.15.16:0/1627214844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fec88b0740 con 0x55fec8824b40 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06c259c640 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec883b150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f06b400ea70 tx=0x7f06b400a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fe98de83d460ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1627214844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b400fbd0 con 0x55fec8824b40 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1627214844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f06b400fd70 con 0x55fec8824b40 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b8004de0 con 0x55fec8824b40 2024-09-15T22:39:56.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1627214844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f06b403a050 con 0x55fec8824b40 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 10 client.15732 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fec88b1a00 con 0x55fec8824b40 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1627214844 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06b8004de0 con 0x55fec8824b40 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1627214844 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f06b4031970 con 0x55fec8824b40 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 10 client.15732 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 10 client.15732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 20 client.15732 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec88b2120 0x55fec88b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 4 client.15732 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:39:56.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.452+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fec8790d70 con 0x55fec88b2120 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec88b2120 0x55fec88b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06c1d9b640 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec88b2120 0x55fec88b4540 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6fda29f5f88abe16 server_cookie=74a0e59f0c047cd6 in_seq=0 out_seq=0 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06a9ffb640 10 client.15732 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06a9ffb640 1 -- 172.21.15.16:0/1627214844 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fec8790d70 con 0x55fec88b2120 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06a9ffb640 10 client.15732 handle_command_reply: tid=0 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 client.15732 shutdown 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 msgr2=0x55fec883b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec883b150 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f06b400ea70 tx=0x7f06b400a550 comp rx=0 tx=0).stop 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 shutdown_connections 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55fec88b2120 0x55fec88b4540 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fec8836b80 0x55fec883b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fec8824b40 0x55fec883b150 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 --2- 172.21.15.16:0/1627214844 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fec88235d0 0x55fec883ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06a9ffb640 0 client.15732 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.453+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 >> 172.21.15.16:0/1627214844 conn(0x55fec87ec310 msgr2=0x55fec88328e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 shutdown_connections 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06cb011740 1 -- 172.21.15.16:0/1627214844 wait complete. 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06cb011740 20 client.15732 trim_cache size 0 max 16384 2024-09-15T22:39:56.466 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:39:56.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06ac0a34d0 msgr2=0x7f06ac0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06ac0a34d0 0x7f06ac0a5990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0ed880 tx=0x7f06bc031400 comp rx=0 tx=0).stop 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 msgr2=0x7f06c4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c4101410 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f06bc005ab0 tx=0x7f06bc0040c0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 shutdown_connections 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06c41082a0 0x7f06c4101410 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06c41036e0 0x7f06c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06ac0a34d0 0x7f06ac0a5990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 --2- 172.21.15.16:0/2359279016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f06c4102d30 0x7f06c4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 >> 172.21.15.16:0/2359279016 conn(0x7f06c40fc7d0 msgr2=0x7f06c4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 shutdown_connections 2024-09-15T22:39:56.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:56.454+0000 7f06c8c38640 1 -- 172.21.15.16:0/2359279016 wait complete. 2024-09-15T22:39:56.793 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:56 smithi016 ceph-mon[31492]: pgmap v48: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-09-15T22:39:56.845 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:56 smithi059 ceph-mon[34872]: pgmap v48: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-09-15T22:39:57.002 INFO:tasks.workunit.client.0.smithi016.stdout:.......... 2024-09-15T22:39:57.002 INFO:tasks.workunit.client.0.smithi016.stdout:do_o_direct_reads: SUCCESS 2024-09-15T22:39:57.007 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T22:39:57.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T22:39:57.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:56 smithi130 ceph-mon[35742]: pgmap v48: 129 pgs: 129 active+clean; 18 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-09-15T22:39:57.040 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-09-15T22:39:57.040 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-09-15T22:39:57.102 INFO:tasks.workunit.client.0.smithi016.stdout:test read from hole 2024-09-15T22:39:57.102 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'test read from hole' 2024-09-15T22:39:57.103 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-09-15T22:39:57.104 INFO:tasks.workunit.client.0.smithi016.stderr:0+0 records in 2024-09-15T22:39:57.104 INFO:tasks.workunit.client.0.smithi016.stderr:0+0 records out 2024-09-15T22:39:57.104 INFO:tasks.workunit.client.0.smithi016.stderr:0 bytes copied, 3.4756e-05 s, 0.0 kB/s 2024-09-15T22:39:57.104 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-09-15T22:39:57.106 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records in 2024-09-15T22:39:57.106 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records out 2024-09-15T22:39:57.106 INFO:tasks.workunit.client.0.smithi016.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00128712 s, 796 kB/s 2024-09-15T22:39:57.106 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-09-15T22:39:57.107 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records in 2024-09-15T22:39:57.107 INFO:tasks.workunit.client.0.smithi016.stderr:2+0 records out 2024-09-15T22:39:57.107 INFO:tasks.workunit.client.0.smithi016.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000117562 s, 8.7 MB/s 2024-09-15T22:39:57.107 INFO:tasks.workunit.client.0.smithi016.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-09-15T22:39:57.108 INFO:tasks.workunit.client.0.smithi016.stdout:other thing 2024-09-15T22:39:57.108 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'other thing' 2024-09-15T22:39:57.108 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-09-15T22:39:57.108 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records in 2024-09-15T22:39:57.109 INFO:tasks.workunit.client.0.smithi016.stderr:1+0 records out 2024-09-15T22:39:57.109 INFO:tasks.workunit.client.0.smithi016.stderr:500 bytes copied, 5.1432e-05 s, 9.7 MB/s 2024-09-15T22:39:57.109 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-09-15T22:39:57.114 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records in 2024-09-15T22:39:57.115 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records out 2024-09-15T22:39:57.115 INFO:tasks.workunit.client.0.smithi016.stderr:500 bytes copied, 0.00435006 s, 115 kB/s 2024-09-15T22:39:57.115 INFO:tasks.workunit.client.0.smithi016.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-09-15T22:39:57.121 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records in 2024-09-15T22:39:57.121 INFO:tasks.workunit.client.0.smithi016.stderr:0+1 records out 2024-09-15T22:39:57.121 INFO:tasks.workunit.client.0.smithi016.stderr:500 bytes copied, 0.00575752 s, 86.8 kB/s 2024-09-15T22:39:57.121 INFO:tasks.workunit.client.0.smithi016.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-09-15T22:39:57.122 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-15T22:39:57.123 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-15T22:39:57.123 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T22:39:57.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T22:39:57.199 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-09-15T22:39:57.199 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-09-15T22:39:57.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:39:57.298 INFO:tasks.workunit.client.0.smithi016.stdout:writing first 3 bytes of 10k file 2024-09-15T22:39:57.298 INFO:tasks.workunit.client.0.smithi016.stdout:reading O_DIRECT 2024-09-15T22:39:57.298 INFO:tasks.workunit.client.0.smithi016.stdout:got 10000 2024-09-15T22:39:57.299 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T22:39:57.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T22:39:57.372 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-09-15T22:39:57.373 DEBUG:teuthology.orchestra.run.smithi016:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-09-15T22:39:57.536 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.097+0000 7faf4c891640 1 -- 172.21.15.16:0/161692722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 msgr2=0x7faf4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.097+0000 7faf4c891640 1 --2- 172.21.15.16:0/161692722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 0x7faf4810da40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faf40009ca0 tx=0x7faf4002a950 comp rx=0 tx=0).stop 2024-09-15T22:39:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 -- 172.21.15.16:0/161692722 shutdown_connections 2024-09-15T22:39:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 --2- 172.21.15.16:0/161692722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 0x7faf4810da40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 --2- 172.21.15.16:0/161692722 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf48102570 0x7faf481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 --2- 172.21.15.16:0/161692722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 0x7faf48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 -- 172.21.15.16:0/161692722 >> 172.21.15.16:0/161692722 conn(0x7faf480fc7d0 msgr2=0x7faf480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 -- 172.21.15.16:0/161692722 shutdown_connections 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 -- 172.21.15.16:0/161692722 wait complete. 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.098+0000 7faf4c891640 1 Processor -- start 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4c891640 1 -- start start 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4c891640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 0x7faf480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4c891640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf48102570 0x7faf480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4c891640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 0x7faf480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4c891640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf48110f80 con 0x7faf48102ef0 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4c891640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf48110e00 con 0x7faf48100e20 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4c891640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7faf48111100 con 0x7faf48102570 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 0x7faf480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf46d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 0x7faf480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 0x7faf480fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:57098/0 (socket says 172.21.15.16:57098) 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 -- 172.21.15.16:0/2177902080 learned_addr learned my addr 172.21.15.16:0/2177902080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf45d9b640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf48102570 0x7faf480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 -- 172.21.15.16:0/2177902080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf48102570 msgr2=0x7faf480fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf48102570 0x7faf480fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 -- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 msgr2=0x7faf480fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 0x7faf480fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf4659c640 1 -- 172.21.15.16:0/2177902080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf481a6930 con 0x7faf48100e20 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.099+0000 7faf46d9d640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 0x7faf480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.100+0000 7faf4659c640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 0x7faf480fcc80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf3800e9f0 tx=0x7faf3800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.100+0000 7faf377fe640 1 -- 172.21.15.16:0/2177902080 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3800cd80 con 0x7faf48100e20 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.100+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf481a6c20 con 0x7faf48100e20 2024-09-15T22:39:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.100+0000 7faf377fe640 1 -- 172.21.15.16:0/2177902080 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf38010910 con 0x7faf48100e20 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.100+0000 7faf377fe640 1 -- 172.21.15.16:0/2177902080 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380188f0 con 0x7faf48100e20 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.100+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf481a71b0 con 0x7faf48100e20 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.102+0000 7faf377fe640 1 -- 172.21.15.16:0/2177902080 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7faf38010ac0 con 0x7faf48100e20 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.103+0000 7faf377fe640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faf300a33b0 0x7faf300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.103+0000 7faf45d9b640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faf300a33b0 0x7faf300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.103+0000 7faf377fe640 1 -- 172.21.15.16:0/2177902080 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7faf380d04a0 con 0x7faf48100e20 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.103+0000 7faf45d9b640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faf300a33b0 0x7faf300a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf480fe710 tx=0x7faf3c00b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 Processor -- start 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 -- start start 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b225d0 0x55a632b22a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b2d370 0x55a632b2f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a632a882f0 con 0x55a632b2d370 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a632a2fcf0 con 0x55a632b23b40 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a632a842b0 con 0x55a632b225d0 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf46d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b2d370 0x55a632b2f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b22090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:57108/0 (socket says 172.21.15.16:57108) 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.104+0000 7faf4659c640 1 -- 172.21.15.16:0/2541564739 learned_addr learned my addr 172.21.15.16:0/2541564739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf45d9b640 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b225d0 0x55a632b22a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf4659c640 1 -- 172.21.15.16:0/2541564739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b225d0 msgr2=0x55a632b22a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf4659c640 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b225d0 0x55a632b22a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf4659c640 1 -- 172.21.15.16:0/2541564739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b2d370 msgr2=0x55a632b2f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf4659c640 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b2d370 0x55a632b2f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf4659c640 1 -- 172.21.15.16:0/2541564739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a632a143a0 con 0x55a632b23b40 2024-09-15T22:39:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf4659c640 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b22090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf380988d0 tx=0x7faf380a0770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f45b2e734b49c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf34ff9640 1 -- 172.21.15.16:0/2541564739 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380cdab0 con 0x55a632b23b40 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf34ff9640 1 -- 172.21.15.16:0/2541564739 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf38098ae0 con 0x55a632b23b40 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.105+0000 7faf34ff9640 1 -- 172.21.15.16:0/2541564739 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380981c0 con 0x55a632b23b40 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2541564739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 msgr2=0x55a632b22090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b22090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faf380988d0 tx=0x7faf380a0770 comp rx=0 tx=0).stop 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2541564739 shutdown_connections 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b2d370 0x55a632b2f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b225d0 0x55a632b22a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2541564739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b22090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2541564739 >> 172.21.15.16:0/2541564739 conn(0x55a632aeb310 msgr2=0x55a632b1ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2541564739 shutdown_connections 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2541564739 wait complete. 2024-09-15T22:39:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 -- start start 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.106+0000 7faf4ec6a740 1 Processor -- start 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b2cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b23b40 0x55a632b2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b2d370 0x55a632b2a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a632b2ffb0 con 0x55a632b225d0 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a632a43060 con 0x55a632b2d370 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a632b30130 con 0x55a632b23b40 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4ec6a740 10 client.0 _ll_register_callbacks cb 0x55a6327968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b2cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf45d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b23b40 0x55a632b2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b2cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39212/0 (socket says 172.21.15.16:39212) 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf46d9d640 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b2d370 0x55a632b2a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4659c640 1 -- 172.21.15.16:0/482785614 learned_addr learned my addr 172.21.15.16:0/482785614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf45d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b23b40 0x55a632b2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:44450/0 (socket says 172.21.15.16:44450) 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf46d9d640 1 -- 172.21.15.16:0/482785614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b23b40 msgr2=0x55a632b2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf46d9d640 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b23b40 0x55a632b2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf46d9d640 1 -- 172.21.15.16:0/482785614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 msgr2=0x55a632b2cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf46d9d640 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b2cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf46d9d640 1 -- 172.21.15.16:0/482785614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6327f93c0 con 0x55a632b2d370 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf4659c640 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b2cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf46d9d640 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b2d370 0x55a632b2a7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf40005cc0 tx=0x7faf4002f880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab17fbf2bb16d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.107+0000 7faf2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf2dffb640 1 -- 172.21.15.16:0/482785614 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40049060 con 0x55a632b2d370 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf2dffb640 1 -- 172.21.15.16:0/482785614 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7faf40003d80 con 0x55a632b2d370 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf38098ae0 con 0x55a632b2d370 2024-09-15T22:39:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf2dffb640 1 -- 172.21.15.16:0/482785614 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf40050050 con 0x55a632b2d370 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 10 client.25367 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a632b3bb30 con 0x55a632b2d370 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf2dffb640 1 -- 172.21.15.16:0/482785614 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf38098ae0 con 0x55a632b2d370 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf2dffb640 1 -- 172.21.15.16:0/482785614 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf400307d0 con 0x55a632b2d370 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 10 client.25367 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 10 client.25367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 20 client.25367 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632b3cff0 0x55a632b3f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 4 client.25367 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.108+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a632a95860 con 0x55a632b3cff0 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.109+0000 7faf4659c640 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632b3cff0 0x55a632b3f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.109+0000 7faf4659c640 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632b3cff0 0x55a632b3f440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64c89b7ec9905db0 server_cookie=6fe4cf5c6e420142 in_seq=0 out_seq=0 2024-09-15T22:39:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.109+0000 7faf2dffb640 10 client.25367 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.109+0000 7faf2dffb640 1 -- 172.21.15.16:0/482785614 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a632a95860 con 0x55a632b3cff0 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.109+0000 7faf2dffb640 10 client.25367 handle_command_reply: tid=0 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 client.25367 shutdown 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b2d370 msgr2=0x55a632b2a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b2d370 0x55a632b2a7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf40005cc0 tx=0x7faf4002f880 comp rx=0 tx=0).stop 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 shutdown_connections 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632b3cff0 0x55a632b3f440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b2d370 0x55a632b2a7d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf2dffb640 0 client.25367 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b23b40 0x55a632b2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/482785614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b2cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 >> 172.21.15.16:0/482785614 conn(0x55a632aeb310 msgr2=0x55a632b249c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 shutdown_connections 2024-09-15T22:39:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 1 -- 172.21.15.16:0/482785614 wait complete. 2024-09-15T22:39:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.110+0000 7faf4ec6a740 20 client.25367 trim_cache size 0 max 16384 2024-09-15T22:39:58.151 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:39:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 Processor -- start 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 -- start start 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b39250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf481b39e0 con 0x55a632b225d0 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf481b12b0 con 0x55a632b23b40 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7faf481af100 con 0x55a632b2d370 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b39250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b39250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39214/0 (socket says 172.21.15.16:39214) 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf4659c640 1 -- 172.21.15.16:0/3424651780 learned_addr learned my addr 172.21.15.16:0/3424651780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.123+0000 7faf45d9b640 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf46d9d640 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf4659c640 1 -- 172.21.15.16:0/3424651780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 msgr2=0x55a632b25660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf4659c640 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b25660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf4659c640 1 -- 172.21.15.16:0/3424651780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 msgr2=0x55a632b25120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf4659c640 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b25120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf4659c640 1 -- 172.21.15.16:0/3424651780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a632b39790 con 0x55a632b225d0 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf4659c640 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b39250 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf38098e40 tx=0x7faf3809a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683c9e73a69c9b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf2ffff640 1 -- 172.21.15.16:0/3424651780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380d0160 con 0x55a632b225d0 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf2ffff640 1 -- 172.21.15.16:0/3424651780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf3809b940 con 0x55a632b225d0 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.124+0000 7faf2ffff640 1 -- 172.21.15.16:0/3424651780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380b05f0 con 0x55a632b225d0 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 -- 172.21.15.16:0/3424651780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 msgr2=0x55a632b39250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b39250 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf38098e40 tx=0x7faf3809a980 comp rx=0 tx=0).stop 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 -- 172.21.15.16:0/3424651780 shutdown_connections 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/3424651780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b39250 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 -- 172.21.15.16:0/3424651780 >> 172.21.15.16:0/3424651780 conn(0x55a632aeb310 msgr2=0x55a632b29180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 -- 172.21.15.16:0/3424651780 shutdown_connections 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 -- 172.21.15.16:0/3424651780 wait complete. 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.125+0000 7faf4ec6a740 1 -- start start 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 1 Processor -- start 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b35cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b30d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7faf481af770 con 0x55a632b225d0 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf481b0be0 con 0x55a632b23b40 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7faf481afe40 con 0x55a632b2d370 2024-09-15T22:39:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4ec6a740 10 client.0 _ll_register_callbacks cb 0x7faf4002fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b357b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39228/0 (socket says 172.21.15.16:39228) 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 -- 172.21.15.16:0/2933200852 learned_addr learned my addr 172.21.15.16:0/2933200852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf46d9d640 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b30d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf45d9b640 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b35cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 -- 172.21.15.16:0/2933200852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 msgr2=0x55a632b30d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b30d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 -- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 msgr2=0x55a632b35cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b35cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.126+0000 7faf4659c640 1 -- 172.21.15.16:0/2933200852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a632b31560 con 0x55a632b225d0 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf4659c640 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b357b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf380a4b00 tx=0x7faf38009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e31fd558f48bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf2dffb640 1 -- 172.21.15.16:0/2933200852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380ba060 con 0x55a632b225d0 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf2dffb640 1 -- 172.21.15.16:0/2933200852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7faf3809baf0 con 0x55a632b225d0 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf3809b940 con 0x55a632b225d0 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf2dffb640 1 -- 172.21.15.16:0/2933200852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf380b5230 con 0x55a632b225d0 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf4ec6a740 10 client.15750 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a632ba7990 con 0x55a632b225d0 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf2dffb640 1 -- 172.21.15.16:0/2933200852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf3809b940 con 0x55a632b225d0 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf2dffb640 1 -- 172.21.15.16:0/2933200852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf380982e0 con 0x55a632b225d0 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf4ec6a740 10 client.15750 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf4ec6a740 10 client.15750 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.127+0000 7faf4ec6a740 20 client.15750 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632ba7c10 0x55a632baa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf4ec6a740 4 client.15750 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a632a8fd70 con 0x55a632ba7c10 2024-09-15T22:39:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf45d9b640 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632ba7c10 0x55a632baa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf2dffb640 10 client.15750 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf45d9b640 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632ba7c10 0x55a632baa000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a75cf66ef310c29 server_cookie=5bcc748428bba59 in_seq=0 out_seq=0 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf2dffb640 1 -- 172.21.15.16:0/2933200852 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a632a8fd70 con 0x55a632ba7c10 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf2dffb640 10 client.15750 handle_command_reply: tid=0 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf4ec6a740 1 client.15750 shutdown 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 msgr2=0x55a632b357b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.128+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b357b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf380a4b00 tx=0x7faf38009700 comp rx=0 tx=0).stop 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 shutdown_connections 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55a632ba7c10 0x55a632baa000 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a632b2d370 0x55a632b30d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a632b23b40 0x55a632b35cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 --2- 172.21.15.16:0/2933200852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a632b225d0 0x55a632b357b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 >> 172.21.15.16:0/2933200852 conn(0x55a632aeb310 msgr2=0x55a632b2b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf2dffb640 0 client.15750 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 shutdown_connections 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 1 -- 172.21.15.16:0/2933200852 wait complete. 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4ec6a740 20 client.15750 trim_cache size 0 max 16384 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faf300a33b0 msgr2=0x7faf300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4c891640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faf300a33b0 0x7faf300a5870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf480fe710 tx=0x7faf3c00b390 comp rx=0 tx=0).stop 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 msgr2=0x7faf480fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4c891640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 0x7faf480fcc80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faf3800e9f0 tx=0x7faf3800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 shutdown_connections 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.129+0000 7faf4c891640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf48102ef0 0x7faf480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.130+0000 7faf4c891640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faf300a33b0 0x7faf300a5870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.130+0000 7faf4c891640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7faf48102570 0x7faf480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.130+0000 7faf4c891640 1 --2- 172.21.15.16:0/2177902080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf48100e20 0x7faf480fcc80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.130+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 >> 172.21.15.16:0/2177902080 conn(0x7faf480fc7d0 msgr2=0x7faf4806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.130+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 shutdown_connections 2024-09-15T22:39:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:58.130+0000 7faf4c891640 1 -- 172.21.15.16:0/2177902080 wait complete. 2024-09-15T22:39:58.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:39:58.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:58 smithi016 ceph-mon[31492]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-09-15T22:39:58.837 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:39:58 smithi016 ceph-mon[31492]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-15T22:39:58.896 INFO:tasks.workunit.client.0.smithi016.stdout:writing pattern 2024-09-15T22:39:58.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-09-15T22:39:58.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-15T22:39:58.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-09-15T22:39:58.896 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-09-15T22:39:58.897 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-09-15T22:39:58.898 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-09-15T22:39:58.899 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-15T22:39:58.900 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-09-15T22:39:58.901 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-15T22:39:58.902 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-09-15T22:39:58.940 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-15T22:39:58.940 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-09-15T22:39:58.940 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-15T22:39:58.940 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-09-15T22:39:58.940 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-15T22:39:58.940 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-09-15T22:39:58.940 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-09-15T22:39:58.941 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-09-15T22:39:58.942 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-09-15T22:39:58.943 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-09-15T22:39:58.944 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-09-15T22:39:58.945 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-15T22:39:58.946 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-09-15T22:39:58.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:58 smithi059 ceph-mon[34872]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-09-15T22:39:58.947 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:39:58 smithi059 ceph-mon[34872]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-15T22:39:58.965 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-09-15T22:39:58.966 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-09-15T22:39:58.967 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-09-15T22:39:58.968 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-15T22:39:58.969 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-15T22:39:58.970 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-09-15T22:39:58.971 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-09-15T22:39:58.972 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-09-15T22:39:58.992 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-09-15T22:39:58.993 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-15T22:39:58.994 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-15T22:39:58.995 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-15T22:39:58.996 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-15T22:39:58.997 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-15T22:39:58.998 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-09-15T22:39:59.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:58 smithi130 ceph-mon[35742]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-09-15T22:39:59.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:39:58 smithi130 ceph-mon[35742]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-15T22:39:59.035 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-15T22:39:59.036 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-09-15T22:39:59.037 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-09-15T22:39:59.038 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-09-15T22:39:59.039 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-09-15T22:39:59.040 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-15T22:39:59.041 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-09-15T22:39:59.075 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-15T22:39:59.075 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-09-15T22:39:59.075 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-15T22:39:59.075 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-09-15T22:39:59.076 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-09-15T22:39:59.077 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-09-15T22:39:59.078 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-09-15T22:39:59.079 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-09-15T22:39:59.080 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-15T22:39:59.081 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-09-15T22:39:59.101 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-15T22:39:59.101 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-09-15T22:39:59.101 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-15T22:39:59.101 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-09-15T22:39:59.101 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-15T22:39:59.101 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-09-15T22:39:59.101 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-15T22:39:59.102 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-15T22:39:59.103 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-15T22:39:59.104 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-09-15T22:39:59.105 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-09-15T22:39:59.106 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-15T22:39:59.107 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-09-15T22:39:59.108 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:59.134 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-09-15T22:39:59.135 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-09-15T22:39:59.136 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-15T22:39:59.137 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-09-15T22:39:59.138 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-15T22:39:59.139 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-15T22:39:59.140 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-09-15T22:39:59.141 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-09-15T22:39:59.178 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-15T22:39:59.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-09-15T22:39:59.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-09-15T22:39:59.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-09-15T22:39:59.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-15T22:39:59.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-15T22:39:59.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-09-15T22:39:59.210 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-15T22:39:59.210 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-09-15T22:39:59.210 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-15T22:39:59.210 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-09-15T22:39:59.210 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-15T22:39:59.210 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-15T22:39:59.211 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-15T22:39:59.212 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-15T22:39:59.213 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-15T22:39:59.214 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-15T22:39:59.215 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-15T22:39:59.216 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-09-15T22:39:59.233 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-15T22:39:59.233 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-15T22:39:59.234 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-15T22:39:59.235 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-09-15T22:39:59.236 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-09-15T22:39:59.237 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-15T22:39:59.238 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-09-15T22:39:59.239 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-15T22:39:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-09-15T22:39:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-15T22:39:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-09-15T22:39:59.266 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-09-15T22:39:59.267 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-09-15T22:39:59.268 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-09-15T22:39:59.269 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-09-15T22:39:59.270 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-09-15T22:39:59.271 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-09-15T22:39:59.272 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-15T22:39:59.308 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-15T22:39:59.309 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-09-15T22:39:59.310 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-09-15T22:39:59.311 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-09-15T22:39:59.312 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-09-15T22:39:59.313 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-09-15T22:39:59.314 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-15T22:39:59.338 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-09-15T22:39:59.339 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-15T22:39:59.340 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-15T22:39:59.341 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-15T22:39:59.342 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-09-15T22:39:59.343 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-15T22:39:59.344 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-15T22:39:59.345 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-09-15T22:39:59.366 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-09-15T22:39:59.367 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-09-15T22:39:59.368 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-09-15T22:39:59.369 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-15T22:39:59.370 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-15T22:39:59.371 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-09-15T22:39:59.400 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-09-15T22:39:59.401 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-09-15T22:39:59.402 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-15T22:39:59.403 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-09-15T22:39:59.404 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-15T22:39:59.405 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-09-15T22:39:59.406 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-09-15T22:39:59.407 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-15T22:39:59.441 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-09-15T22:39:59.441 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-15T22:39:59.441 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-09-15T22:39:59.442 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-15T22:39:59.443 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-09-15T22:39:59.444 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-09-15T22:39:59.445 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-09-15T22:39:59.446 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-09-15T22:39:59.447 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-15T22:39:59.448 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-09-15T22:39:59.448 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-15T22:39:59.469 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-09-15T22:39:59.469 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-15T22:39:59.469 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-09-15T22:39:59.469 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-15T22:39:59.469 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-09-15T22:39:59.470 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-09-15T22:39:59.471 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-09-15T22:39:59.472 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-15T22:39:59.473 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-15T22:39:59.474 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-09-15T22:39:59.475 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-15T22:39:59.497 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-15T22:39:59.498 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-09-15T22:39:59.499 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-15T22:39:59.500 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-15T22:39:59.501 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-15T22:39:59.502 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-09-15T22:39:59.503 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-15T22:39:59.504 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-09-15T22:39:59.535 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-15T22:39:59.536 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-15T22:39:59.537 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-15T22:39:59.538 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-09-15T22:39:59.539 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-09-15T22:39:59.540 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-15T22:39:59.541 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-09-15T22:39:59.541 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-15T22:39:59.541 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-09-15T22:39:59.541 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-15T22:39:59.541 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-09-15T22:39:59.541 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-15T22:39:59.541 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-09-15T22:39:59.582 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-15T22:39:59.582 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-09-15T22:39:59.583 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-15T22:39:59.584 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-15T22:39:59.585 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-15T22:39:59.586 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-09-15T22:39:59.587 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-15T22:39:59.588 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-09-15T22:39:59.589 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-15T22:39:59.609 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-09-15T22:39:59.610 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-09-15T22:39:59.611 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-15T22:39:59.612 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-15T22:39:59.613 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-09-15T22:39:59.614 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-15T22:39:59.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-09-15T22:39:59.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-15T22:39:59.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-09-15T22:39:59.615 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 -- 172.21.15.16:0/1780018681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5841037a0 msgr2=0x7fb58410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 --2- 172.21.15.16:0/1780018681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5841037a0 0x7fb58410bc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb578009900 tx=0x7fb57802a410 comp rx=0 tx=0).stop 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 -- 172.21.15.16:0/1780018681 shutdown_connections 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 --2- 172.21.15.16:0/1780018681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb58410c2f0 0x7fb58410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 --2- 172.21.15.16:0/1780018681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5841037a0 0x7fb58410bc00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 --2- 172.21.15.16:0/1780018681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 0x7fb5841031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 -- 172.21.15.16:0/1780018681 >> 172.21.15.16:0/1780018681 conn(0x7fb5840fc7d0 msgr2=0x7fb5840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 -- 172.21.15.16:0/1780018681 shutdown_connections 2024-09-15T22:39:59.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.617+0000 7fb588c53640 1 -- 172.21.15.16:0/1780018681 wait complete. 2024-09-15T22:39:59.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.618+0000 7fb588c53640 1 Processor -- start 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.618+0000 7fb588c53640 1 -- start start 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.618+0000 7fb588c53640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 0x7fb584198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb588c53640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5841037a0 0x7fb584198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb588c53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb58410c2f0 0x7fb58419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb588c53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841120b0 con 0x7fb58410c2f0 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb588c53640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb584111f30 con 0x7fb584102df0 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb588c53640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb584112230 con 0x7fb5841037a0 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 0x7fb584198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb582d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb58410c2f0 0x7fb58419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 0x7fb584198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:57166/0 (socket says 172.21.15.16:57166) 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 -- 172.21.15.16:0/3133119432 learned_addr learned my addr 172.21.15.16:0/3133119432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:59.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 -- 172.21.15.16:0/3133119432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5841037a0 msgr2=0x7fb584198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb581d9b640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5841037a0 0x7fb584198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5841037a0 0x7fb584198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 -- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb58410c2f0 msgr2=0x7fb58419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb58410c2f0 0x7fb58419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 -- 172.21.15.16:0/3133119432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58419d540 con 0x7fb584102df0 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb581d9b640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5841037a0 0x7fb584198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.619+0000 7fb58259c640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 0x7fb584198590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb57400e9f0 tx=0x7fb57400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.620+0000 7fb5737fe640 1 -- 172.21.15.16:0/3133119432 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57400cd80 con 0x7fb584102df0 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.620+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb58419d830 con 0x7fb584102df0 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.620+0000 7fb5737fe640 1 -- 172.21.15.16:0/3133119432 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb574010910 con 0x7fb584102df0 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.620+0000 7fb5737fe640 1 -- 172.21.15.16:0/3133119432 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574018980 con 0x7fb584102df0 2024-09-15T22:39:59.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.620+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5841a5110 con 0x7fb584102df0 2024-09-15T22:39:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 Processor -- start 2024-09-15T22:39:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb5737fe640 1 -- 172.21.15.16:0/3133119432 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb574010ac0 con 0x7fb584102df0 2024-09-15T22:39:59.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 -- start start 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2fb40 0x55a93fc2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2e5d0 0x55a93fc2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc3fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a93fb942f0 con 0x55a93fc2e5d0 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a93fb3bcf0 con 0x55a93fc3d780 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.623+0000 7fb58b02c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a93fb902b0 con 0x55a93fc2fb40 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2e5d0 0x55a93fc2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2e5d0 0x55a93fc2ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39250/0 (socket says 172.21.15.16:39250) 2024-09-15T22:39:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 -- 172.21.15.16:0/1772738162 learned_addr learned my addr 172.21.15.16:0/1772738162 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb581d9b640 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc3fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb582d9d640 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2fb40 0x55a93fc2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 -- 172.21.15.16:0/1772738162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2fb40 msgr2=0x55a93fc2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2fb40 0x55a93fc2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 -- 172.21.15.16:0/1772738162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 msgr2=0x55a93fc3fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc3fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 -- 172.21.15.16:0/1772738162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93fb203a0 con 0x55a93fc2e5d0 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb5737fe640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb56c0a33b0 0x7fb56c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb58259c640 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2e5d0 0x55a93fc2ea10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb5740cd7a0 tx=0x7fb5740d6ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ae1f0f563c5a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.624+0000 7fb582d9d640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb56c0a33b0 0x7fb56c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.625+0000 7fb5737fe640 1 -- 172.21.15.16:0/3133119432 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb5740d0440 con 0x7fb584102df0 2024-09-15T22:39:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.625+0000 7fb570ff9640 1 -- 172.21.15.16:0/1772738162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574098660 con 0x55a93fc2e5d0 2024-09-15T22:39:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.625+0000 7fb570ff9640 1 -- 172.21.15.16:0/1772738162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57409cac0 con 0x55a93fc2e5d0 2024-09-15T22:39:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.625+0000 7fb570ff9640 1 -- 172.21.15.16:0/1772738162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57409bd20 con 0x55a93fc2e5d0 2024-09-15T22:39:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 -- 172.21.15.16:0/1772738162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2e5d0 msgr2=0x55a93fc2ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2e5d0 0x55a93fc2ea10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb5740cd7a0 tx=0x7fb5740d6ab0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb582d9d640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb56c0a33b0 0x7fb56c0a5870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb57c007e50 tx=0x7fb57c007520 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 -- 172.21.15.16:0/1772738162 shutdown_connections 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc3fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2e5d0 0x55a93fc2ea10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1772738162 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2fb40 0x55a93fc2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 -- 172.21.15.16:0/1772738162 >> 172.21.15.16:0/1772738162 conn(0x55a93fbf7310 msgr2=0x55a93fc2ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 -- 172.21.15.16:0/1772738162 shutdown_connections 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 -- 172.21.15.16:0/1772738162 wait complete. 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.626+0000 7fb58b02c740 1 -- start start 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 1 Processor -- start 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc3ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc36090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a93fc40480 con 0x55a93fc2fb40 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a93fb4f060 con 0x55a93fc3d780 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a93fc40600 con 0x55a93fc2e5d0 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb581d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc36090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb581d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc36090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39254/0 (socket says 172.21.15.16:39254) 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb581d9b640 1 -- 172.21.15.16:0/1603216499 learned_addr learned my addr 172.21.15.16:0/1603216499 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58b02c740 10 client.0 _ll_register_callbacks cb 0x55a93f8a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb58259c640 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc3ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.627+0000 7fb582d9d640 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb581d9b640 1 -- 172.21.15.16:0/1603216499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 msgr2=0x55a93fc3ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb581d9b640 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc3ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb581d9b640 1 -- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 msgr2=0x55a93fc365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb581d9b640 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc365d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb581d9b640 1 -- 172.21.15.16:0/1603216499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93f9053c0 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb581d9b640 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc36090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb578031550 tx=0x7fb5780317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a168d5b950e96c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb569ffb640 1 -- 172.21.15.16:0/1603216499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578040070 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb569ffb640 1 -- 172.21.15.16:0/1603216499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb578004240 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb57409cac0 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb569ffb640 1 -- 172.21.15.16:0/1603216499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb57804d050 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb58b02c740 10 client.15762 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.628+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a93fc50380 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb569ffb640 1 -- 172.21.15.16:0/1603216499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57409cac0 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb569ffb640 1 -- 172.21.15.16:0/1603216499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb578004ac0 con 0x55a93fc2fb40 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb58b02c740 10 client.15762 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb58b02c740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb58b02c740 20 client.15762 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fc51840 0x55a93fc53c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb58b02c740 4 client.15762 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:39:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a93fba1860 con 0x55a93fc51840 2024-09-15T22:39:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.629+0000 7fb58259c640 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fc51840 0x55a93fc53c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.630+0000 7fb58259c640 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fc51840 0x55a93fc53c80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c857342088e510e2 server_cookie=41f02a1574f6334d in_seq=0 out_seq=0 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.630+0000 7fb569ffb640 10 client.15762 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.630+0000 7fb569ffb640 1 -- 172.21.15.16:0/1603216499 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a93fba1860 con 0x55a93fc51840 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.630+0000 7fb569ffb640 10 client.15762 handle_command_reply: tid=0 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.630+0000 7fb58b02c740 1 client.15762 shutdown 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.630+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 msgr2=0x55a93fc36090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.630+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc36090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb578031550 tx=0x7fb5780317e0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 shutdown_connections 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fc51840 0x55a93fc53c80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc3d780 0x55a93fc365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc36090 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb569ffb640 0 client.15762 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:59.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1603216499 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc3ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 >> 172.21.15.16:0/1603216499 conn(0x55a93fbf7310 msgr2=0x55a93fc30960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:59.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 shutdown_connections 2024-09-15T22:39:59.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 1 -- 172.21.15.16:0/1603216499 wait complete. 2024-09-15T22:39:59.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.631+0000 7fb58b02c740 20 client.15762 trim_cache size 0 max 16384 2024-09-15T22:39:59.638 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-09-15T22:39:59.638 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-15T22:39:59.638 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-09-15T22:39:59.638 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-09-15T22:39:59.639 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-15T22:39:59.640 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-15T22:39:59.641 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-09-15T22:39:59.642 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-15T22:39:59.643 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-09-15T22:39:59.644 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-15T22:39:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.644+0000 7fb58b02c740 1 Processor -- start 2024-09-15T22:39:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.644+0000 7fb58b02c740 1 -- start start 2024-09-15T22:39:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.644+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2e5d0 0x55a93fc34980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc31120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58259c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2e5d0 0x55a93fc34980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc3d780 0x55a93fc31660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58259c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2e5d0 0x55a93fc34980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:57180/0 (socket says 172.21.15.16:57180) 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc31120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc31120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39264/0 (socket says 172.21.15.16:39264) 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841af720 con 0x55a93fc2fb40 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58259c640 1 -- 172.21.15.16:0/2021229505 learned_addr learned my addr 172.21.15.16:0/2021229505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841acff0 con 0x55a93fc2e5d0 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841aae40 con 0x55a93fc3d780 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 -- 172.21.15.16:0/2021229505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc3d780 msgr2=0x55a93fc31660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 --2- 172.21.15.16:0/2021229505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc3d780 0x55a93fc31660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 -- 172.21.15.16:0/2021229505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2e5d0 msgr2=0x55a93fc34980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 --2- 172.21.15.16:0/2021229505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2e5d0 0x55a93fc34980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 -- 172.21.15.16:0/2021229505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93fc372e0 con 0x55a93fc2fb40 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.645+0000 7fb581d9b640 1 --2- 172.21.15.16:0/2021229505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc31120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb57802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc8dffffab12241 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:59.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.646+0000 7fb56bfff640 1 -- 172.21.15.16:0/2021229505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578040070 con 0x55a93fc2fb40 2024-09-15T22:39:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.646+0000 7fb56bfff640 1 -- 172.21.15.16:0/2021229505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb578009520 con 0x55a93fc2fb40 2024-09-15T22:39:59.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.646+0000 7fb56bfff640 1 -- 172.21.15.16:0/2021229505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578035040 con 0x55a93fc2fb40 2024-09-15T22:39:59.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.646+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 msgr2=0x55a93fc31120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.646+0000 7fb58b02c740 1 --2- 172.21.15.16:0/2021229505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc31120 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb578000c00 tx=0x7fb57802a410 comp rx=0 tx=0).stop 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 shutdown_connections 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 --2- 172.21.15.16:0/2021229505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc3d780 0x55a93fc31660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 --2- 172.21.15.16:0/2021229505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc2fb40 0x55a93fc31120 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 --2- 172.21.15.16:0/2021229505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2e5d0 0x55a93fc34980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 >> 172.21.15.16:0/2021229505 conn(0x55a93fbf7310 msgr2=0x55a93fc29d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 shutdown_connections 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 -- 172.21.15.16:0/2021229505 wait complete. 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 -- start start 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 Processor -- start 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc4e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2fb40 0x55a93fc4ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.647+0000 7fb58b02c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc3d780 0x55a93fc32e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58b02c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841ab4b0 con 0x55a93fc3d780 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc4e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb581d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2fb40 0x55a93fc4ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc4e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:44498/0 (socket says 172.21.15.16:44498) 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 -- 172.21.15.16:0/1050724673 learned_addr learned my addr 172.21.15.16:0/1050724673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb582d9d640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc3d780 0x55a93fc32e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841ac920 con 0x55a93fc2fb40 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb5841abb80 con 0x55a93fc2e5d0 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58b02c740 10 client.0 _ll_register_callbacks cb 0x7fb578009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 -- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2fb40 msgr2=0x55a93fc4ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2fb40 0x55a93fc4ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 -- 172.21.15.16:0/1050724673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc3d780 msgr2=0x55a93fc32e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc3d780 0x55a93fc32e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 -- 172.21.15.16:0/1050724673 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93fc33630 con 0x55a93fc2e5d0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb581d9b640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2fb40 0x55a93fc4ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb582d9d640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc3d780 0x55a93fc32e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb58259c640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc4e680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb57400e9c0 tx=0x7fb574018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92153edc195120fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.648+0000 7fb569ffb640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb569ffb640 1 -- 172.21.15.16:0/1050724673 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57409bf00 con 0x55a93fc2e5d0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb569ffb640 1 -- 172.21.15.16:0/1050724673 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb5740d3d60 con 0x55a93fc2e5d0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb578009520 con 0x55a93fc2e5d0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb569ffb640 1 -- 172.21.15.16:0/1050724673 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb57409c190 con 0x55a93fc2e5d0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 10 client.25384 fetch_fsmap learned FSMap version 26 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a93fcbc1d0 con 0x55a93fc2e5d0 2024-09-15T22:39:59.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb569ffb640 1 -- 172.21.15.16:0/1050724673 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb578009520 con 0x55a93fc2e5d0 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb569ffb640 1 -- 172.21.15.16:0/1050724673 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb5740d0120 con 0x55a93fc2e5d0 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 10 client.25384 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 10 client.25384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 20 client.25384 populate_metadata read hostname 'smithi016' 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fcbc560 0x55a93fcbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 4 client.25384 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.649+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a93fb9bd70 con 0x55a93fcbc560 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb581d9b640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fcbc560 0x55a93fcbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb569ffb640 10 client.25384 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb581d9b640 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fcbc560 0x55a93fcbe950 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba349b55beb4b50d server_cookie=a91edf722f46a79e in_seq=0 out_seq=0 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb569ffb640 1 -- 172.21.15.16:0/1050724673 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a93fb9bd70 con 0x55a93fcbc560 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb569ffb640 10 client.25384 handle_command_reply: tid=0 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb58b02c740 1 client.25384 shutdown 2024-09-15T22:39:59.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 msgr2=0x55a93fc4e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.650+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc4e680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb57400e9c0 tx=0x7fb574018420 comp rx=0 tx=0).stop 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 shutdown_connections 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55a93fcbc560 0x55a93fcbe950 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a93fc3d780 0x55a93fc32e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93fc2fb40 0x55a93fc4ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb569ffb640 0 client.25384 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 --2- 172.21.15.16:0/1050724673 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a93fc2e5d0 0x55a93fc4e680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 >> 172.21.15.16:0/1050724673 conn(0x55a93fbf7310 msgr2=0x55a93fc39900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 shutdown_connections 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 1 -- 172.21.15.16:0/1050724673 wait complete. 2024-09-15T22:39:59.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.651+0000 7fb58b02c740 20 client.25384 trim_cache size 0 max 16384 2024-09-15T22:39:59.659 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:39:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb56c0a33b0 msgr2=0x7fb56c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb56c0a33b0 0x7fb56c0a5870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb57c007e50 tx=0x7fb57c007520 comp rx=0 tx=0).stop 2024-09-15T22:39:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 msgr2=0x7fb584198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:59.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 0x7fb584198590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb57400e9f0 tx=0x7fb57400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 shutdown_connections 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb58410c2f0 0x7fb58419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb56c0a33b0 0x7fb56c0a5870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb5841037a0 0x7fb584198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 --2- 172.21.15.16:0/3133119432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb584102df0 0x7fb584198590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 >> 172.21.15.16:0/3133119432 conn(0x7fb5840fc7d0 msgr2=0x7fb5840fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 shutdown_connections 2024-09-15T22:39:59.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:39:59.652+0000 7fb588c53640 1 -- 172.21.15.16:0/3133119432 wait complete. 2024-09-15T22:39:59.682 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-09-15T22:39:59.682 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-15T22:39:59.682 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-15T22:39:59.683 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-15T22:39:59.684 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-15T22:39:59.685 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-15T22:39:59.686 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-15T22:39:59.687 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-15T22:39:59.688 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-09-15T22:39:59.724 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-15T22:39:59.724 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-09-15T22:39:59.724 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-15T22:39:59.724 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-09-15T22:39:59.724 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-15T22:39:59.724 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-09-15T22:39:59.724 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-09-15T22:39:59.725 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-09-15T22:39:59.726 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-09-15T22:39:59.727 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-09-15T22:39:59.728 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-09-15T22:39:59.729 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-15T22:39:59.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-09-15T22:39:59.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-15T22:39:59.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-09-15T22:39:59.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-15T22:39:59.730 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-09-15T22:39:59.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-15T22:39:59.746 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-15T22:39:59.747 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-15T22:39:59.748 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-15T22:39:59.749 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-15T22:39:59.750 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-09-15T22:39:59.751 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-09-15T22:39:59.752 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-09-15T22:39:59.771 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-09-15T22:39:59.772 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-09-15T22:39:59.773 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-15T22:39:59.774 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-09-15T22:39:59.775 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-09-15T22:39:59.776 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-15T22:39:59.809 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-15T22:39:59.810 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-09-15T22:39:59.811 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-15T22:39:59.812 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-09-15T22:39:59.813 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-09-15T22:39:59.814 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-15T22:39:59.815 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-09-15T22:39:59.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-15T22:39:59.849 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-15T22:39:59.850 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-15T22:39:59.851 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-09-15T22:39:59.852 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-09-15T22:39:59.853 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-15T22:39:59.854 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-15T22:39:59.855 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-09-15T22:39:59.873 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-09-15T22:39:59.874 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-09-15T22:39:59.875 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-09-15T22:39:59.876 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-09-15T22:39:59.877 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-09-15T22:39:59.878 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-09-15T22:39:59.879 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-15T22:39:59.880 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-09-15T22:39:59.880 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-15T22:39:59.880 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:00.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:00.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:00.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:00.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:00.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:00.179 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:00.180 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:00.181 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:00.182 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:00.183 INFO:tasks.workunit.client.0.smithi016.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-09-15T22:40:00.184 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-15T22:40:00.185 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-09-15T22:40:00.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:00 smithi016 ceph-mon[31492]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-09-15T22:40:00.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:00 smithi016 ceph-mon[31492]: overall HEALTH_OK 2024-09-15T22:40:00.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:00 smithi059 ceph-mon[34872]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-09-15T22:40:00.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:00 smithi059 ceph-mon[34872]: overall HEALTH_OK 2024-09-15T22:40:01.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:00 smithi130 ceph-mon[35742]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.4 MiB/s wr, 1.15k op/s 2024-09-15T22:40:01.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:00 smithi130 ceph-mon[35742]: overall HEALTH_OK 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-15T22:40:01.315 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-15T22:40:01.316 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-15T22:40:01.317 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-09-15T22:40:01.318 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-09-15T22:40:01.319 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-09-15T22:40:01.320 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-15T22:40:01.321 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-09-15T22:40:01.377 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:40:01.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:40:01.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:02.048 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-15T22:40:02.048 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-09-15T22:40:02.048 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-15T22:40:02.048 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-09-15T22:40:02.048 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-15T22:40:02.048 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-15T22:40:02.049 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-09-15T22:40:02.050 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-09-15T22:40:02.051 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-15T22:40:02.052 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-15T22:40:02.053 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-09-15T22:40:02.054 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-15T22:40:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.133+0000 7f088604e640 1 -- 172.21.15.16:0/3885601725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880100e20 msgr2=0x7f0880101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.133+0000 7f088604e640 1 --2- 172.21.15.16:0/3885601725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880100e20 0x7f0880101200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0870009930 tx=0x7f087002a4d0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 -- 172.21.15.16:0/3885601725 shutdown_connections 2024-09-15T22:40:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 --2- 172.21.15.16:0/3885601725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0880102ef0 0x7f088010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 --2- 172.21.15.16:0/3885601725 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0880102570 0x7f08801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 --2- 172.21.15.16:0/3885601725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880100e20 0x7f0880101200 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 -- 172.21.15.16:0/3885601725 >> 172.21.15.16:0/3885601725 conn(0x7f08800fc7d0 msgr2=0x7f08800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 -- 172.21.15.16:0/3885601725 shutdown_connections 2024-09-15T22:40:02.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 -- 172.21.15.16:0/3885601725 wait complete. 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.134+0000 7f088604e640 1 Processor -- start 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f088604e640 1 -- start start 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f088604e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0880100e20 0x7f0880198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f088604e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880102570 0x7f0880198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f088604e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0880102ef0 0x7f088019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f088604e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0880110fd0 con 0x7f0880102ef0 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f088604e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0880110e50 con 0x7f0880102570 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f088604e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0880111150 con 0x7f0880100e20 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880102570 0x7f0880198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880102570 0x7f0880198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:56524/0 (socket says 172.21.15.16:56524) 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 -- 172.21.15.16:0/2781762960 learned_addr learned my addr 172.21.15.16:0/2781762960 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:02.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0880102ef0 0x7f088019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087f7fe640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0880100e20 0x7f0880198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 -- 172.21.15.16:0/2781762960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0880100e20 msgr2=0x7f0880198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0880100e20 0x7f0880198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 -- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0880102ef0 msgr2=0x7f088019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0880102ef0 0x7f088019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.135+0000 7f087effd640 1 -- 172.21.15.16:0/2781762960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088019d540 con 0x7f0880102570 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.136+0000 7f087effd640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880102570 0x7f0880198ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f087400e9f0 tx=0x7f087400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.136+0000 7f087cff9640 1 -- 172.21.15.16:0/2781762960 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087400cd80 con 0x7f0880102570 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.136+0000 7f087cff9640 1 -- 172.21.15.16:0/2781762960 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0874010910 con 0x7f0880102570 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.136+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088019d830 con 0x7f0880102570 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.136+0000 7f087cff9640 1 -- 172.21.15.16:0/2781762960 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874018990 con 0x7f0880102570 2024-09-15T22:40:02.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.136+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801a5110 con 0x7f0880102570 2024-09-15T22:40:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.138+0000 7f087cff9640 1 -- 172.21.15.16:0/2781762960 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0874010ac0 con 0x7f0880102570 2024-09-15T22:40:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.138+0000 7f0888427740 1 Processor -- start 2024-09-15T22:40:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.138+0000 7f0888427740 1 -- start start 2024-09-15T22:40:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.138+0000 7f0888427740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d525bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f0888427740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d5266800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f0888427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5269130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f0888427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5611d51c1280 con 0x5611d5266d40 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f0888427740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5611d5168b10 con 0x5611d525c480 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f0888427740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5611d51bd240 con 0x5611d525bb60 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087ffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d525bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d5266800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087ffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d525bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:37290/0 (socket says 172.21.15.16:37290) 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d5266800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:56540/0 (socket says 172.21.15.16:56540) 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087ffff640 1 -- 172.21.15.16:0/1667428124 learned_addr learned my addr 172.21.15.16:0/1667428124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087effd640 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5269130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 -- 172.21.15.16:0/1667428124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 msgr2=0x5611d525bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d525bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 -- 172.21.15.16:0/1667428124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 msgr2=0x5611d5269130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5269130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 -- 172.21.15.16:0/1667428124 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d51751f0 con 0x5611d525c480 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087cff9640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f08680a33b0 0x7f08680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087ffff640 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d525bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087ffff640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f08680a33b0 0x7f08680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087f7fe640 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d5266800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f087002a9f0 tx=0x7f0870005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5fef9da7592c6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.139+0000 7f087cff9640 1 -- 172.21.15.16:0/2781762960 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f08740d03f0 con 0x7f0880102570 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f0865ffb640 1 -- 172.21.15.16:0/1667428124 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870035040 con 0x5611d525c480 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f0865ffb640 1 -- 172.21.15.16:0/1667428124 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870005520 con 0x5611d525c480 2024-09-15T22:40:02.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f0865ffb640 1 -- 172.21.15.16:0/1667428124 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08700035f0 con 0x5611d525c480 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f0888427740 1 -- 172.21.15.16:0/1667428124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 msgr2=0x5611d5266800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f087ffff640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f08680a33b0 0x7f08680a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0878011870 tx=0x7f0878009500 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f0888427740 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d5266800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f087002a9f0 tx=0x7f0870005de0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f0888427740 1 -- 172.21.15.16:0/1667428124 shutdown_connections 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.140+0000 7f0888427740 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5269130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.141+0000 7f0888427740 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d5266800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.141+0000 7f0888427740 1 --2- 172.21.15.16:0/1667428124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d525bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.141+0000 7f0888427740 1 -- 172.21.15.16:0/1667428124 >> 172.21.15.16:0/1667428124 conn(0x5611d52580f0 msgr2=0x5611d52584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.141+0000 7f0888427740 1 -- 172.21.15.16:0/1667428124 shutdown_connections 2024-09-15T22:40:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.141+0000 7f0888427740 1 -- 172.21.15.16:0/1667428124 wait complete. 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.141+0000 7f0888427740 1 -- start start 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.141+0000 7f0888427740 1 Processor -- start 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f0888427740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5263be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f0888427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525c480 0x5611d5264120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f0888427740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d5264660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f0888427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5611d5269910 con 0x5611d525c480 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f0888427740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5611d517bf10 con 0x5611d5266d40 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f0888427740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5611d5269a90 con 0x5611d525bb60 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5263be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5263be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:37302/0 (socket says 172.21.15.16:37302) 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 -- 172.21.15.16:0/3302829824 learned_addr learned my addr 172.21.15.16:0/3302829824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f0888427740 10 client.0 _ll_register_callbacks cb 0x5611d5144a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087effd640 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525c480 0x5611d5264120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087ffff640 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d5264660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 -- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 msgr2=0x5611d5264660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d5264660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 -- 172.21.15.16:0/3302829824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525c480 msgr2=0x5611d5264120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525c480 0x5611d5264120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 -- 172.21.15.16:0/3302829824 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d4f326a0 con 0x5611d525bb60 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f087f7fe640 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5263be0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f087002aac0 tx=0x7f08700048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d980eb5dd7dcfce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.142+0000 7f085effd640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f085effd640 1 -- 172.21.15.16:0/3302829824 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870052060 con 0x5611d525bb60 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0870005520 con 0x5611d525bb60 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f085effd640 1 -- 172.21.15.16:0/3302829824 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0870004390 con 0x5611d525bb60 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f085effd640 1 -- 172.21.15.16:0/3302829824 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0870059050 con 0x5611d525bb60 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 10 client.25396 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5611d5269910 con 0x5611d525bb60 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f085effd640 1 -- 172.21.15.16:0/3302829824 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870005520 con 0x5611d525bb60 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f085effd640 1 -- 172.21.15.16:0/3302829824 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0870003960 con 0x5611d525bb60 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 10 client.25396 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 10 client.25396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 20 client.25396 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d527ade0 0x5611d527d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 4 client.25396 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.143+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611d51ce7f0 con 0x5611d527ade0 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.144+0000 7f087effd640 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d527ade0 0x5611d527d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.144+0000 7f085effd640 10 client.25396 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.144+0000 7f087effd640 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d527ade0 0x5611d527d2a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b6e949a243c66f server_cookie=a1ff330f6ae822b1 in_seq=0 out_seq=0 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f085effd640 1 -- 172.21.15.16:0/3302829824 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611d51ce7f0 con 0x5611d527ade0 2024-09-15T22:40:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f085effd640 10 client.25396 handle_command_reply: tid=0 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 client.25396 shutdown 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 msgr2=0x5611d5263be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5263be0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f087002aac0 tx=0x7f08700048f0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 shutdown_connections 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d527ade0 0x5611d527d2a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d5264660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525c480 0x5611d5264120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 --2- 172.21.15.16:0/3302829824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5263be0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 >> 172.21.15.16:0/3302829824 conn(0x5611d52580f0 msgr2=0x5611d525ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f085effd640 0 client.25396 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 shutdown_connections 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.145+0000 7f0888427740 1 -- 172.21.15.16:0/3302829824 wait complete. 2024-09-15T22:40:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.146+0000 7f0888427740 20 client.25396 trim_cache size 0 max 16384 2024-09-15T22:40:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.164+0000 7f0888427740 1 Processor -- start 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.164+0000 7f0888427740 1 -- start start 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f0888427740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5282b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f0888427740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d52830b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f0888427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5256a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f0888427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08801af5f0 con 0x5611d5266d40 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f0888427740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08801acec0 con 0x5611d525c480 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f0888427740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f08801aad10 con 0x5611d525bb60 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5282b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d52830b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5256a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d52830b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:56546/0 (socket says 172.21.15.16:56546) 2024-09-15T22:40:02.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5282b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:37312/0 (socket says 172.21.15.16:37312) 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087effd640 1 -- 172.21.15.16:0/2163002107 learned_addr learned my addr 172.21.15.16:0/2163002107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087ffff640 1 -- 172.21.15.16:0/2163002107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 msgr2=0x5611d5282b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087ffff640 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5282b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087ffff640 1 -- 172.21.15.16:0/2163002107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 msgr2=0x5611d52830b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087ffff640 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d52830b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087ffff640 1 -- 172.21.15.16:0/2163002107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d5257f20 con 0x5611d5266d40 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087f7fe640 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5282b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.165+0000 7f087ffff640 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5256a50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0878005fd0 tx=0x7f0878016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761834a105ef5aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f08657fa640 1 -- 172.21.15.16:0/2163002107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087800f940 con 0x5611d5266d40 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f08657fa640 1 -- 172.21.15.16:0/2163002107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0878004040 con 0x5611d5266d40 2024-09-15T22:40:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f08657fa640 1 -- 172.21.15.16:0/2163002107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087801e590 con 0x5611d5266d40 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f0888427740 1 -- 172.21.15.16:0/2163002107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 msgr2=0x5611d5256a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f0888427740 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5256a50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0878005fd0 tx=0x7f0878016000 comp rx=0 tx=0).stop 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f0888427740 1 -- 172.21.15.16:0/2163002107 shutdown_connections 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f0888427740 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d5266d40 0x5611d5256a50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f0888427740 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d525c480 0x5611d52830b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f0888427740 1 --2- 172.21.15.16:0/2163002107 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525bb60 0x5611d5282b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.166+0000 7f0888427740 1 -- 172.21.15.16:0/2163002107 >> 172.21.15.16:0/2163002107 conn(0x5611d52580f0 msgr2=0x5611d5265b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 -- 172.21.15.16:0/2163002107 shutdown_connections 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 -- 172.21.15.16:0/2163002107 wait complete. 2024-09-15T22:40:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 -- start start 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 Processor -- start 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525bb60 0x5611d52737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525c480 0x5611d526ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d526f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08801ab380 con 0x5611d525bb60 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f08801ac7f0 con 0x5611d5266d40 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f08801aba50 con 0x5611d525c480 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.167+0000 7f0888427740 10 client.0 _ll_register_callbacks cb 0x7f08740d3d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:02.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525bb60 0x5611d52737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d526f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087effd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525c480 0x5611d526ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525bb60 0x5611d52737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59346/0 (socket says 172.21.15.16:59346) 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 -- 172.21.15.16:0/3080863377 learned_addr learned my addr 172.21.15.16:0/3080863377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 -- 172.21.15.16:0/3080863377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525c480 msgr2=0x5611d526ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525c480 0x5611d526ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 -- 172.21.15.16:0/3080863377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 msgr2=0x5611d526f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d526f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 -- 172.21.15.16:0/3080863377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d526f700 con 0x5611d525bb60 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f087f7fe640 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525bb60 0x5611d52737a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0870003ed0 tx=0x7f0870005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa1fbc908ad5409 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f085effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:02.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f085effd640 1 -- 172.21.15.16:0/3080863377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870052070 con 0x5611d525bb60 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f085effd640 1 -- 172.21.15.16:0/3080863377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0870003800 con 0x5611d525bb60 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0878004040 con 0x5611d525bb60 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f085effd640 1 -- 172.21.15.16:0/3080863377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0870066050 con 0x5611d525bb60 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f0888427740 10 client.15786 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.168+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5611d52e5a60 con 0x5611d525bb60 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f085effd640 1 -- 172.21.15.16:0/3080863377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0878004040 con 0x5611d525bb60 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f085effd640 1 -- 172.21.15.16:0/3080863377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f087003c440 con 0x5611d525bb60 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f0888427740 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:02.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f0888427740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f0888427740 20 client.15786 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f0888427740 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d52e60c0 0x5611d52e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f0888427740 4 client.15786 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:40:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5611d51c8d00 con 0x5611d52e60c0 2024-09-15T22:40:02.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.169+0000 7f087effd640 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d52e60c0 0x5611d52e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:02.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.170+0000 7f087effd640 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d52e60c0 0x5611d52e84e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79dd79ec0f6ac777 server_cookie=249feb05eb7e68f4 in_seq=0 out_seq=0 2024-09-15T22:40:02.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.170+0000 7f085effd640 10 client.15786 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:02.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.170+0000 7f085effd640 1 -- 172.21.15.16:0/3080863377 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5611d51c8d00 con 0x5611d52e60c0 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.170+0000 7f085effd640 10 client.15786 handle_command_reply: tid=0 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 client.15786 shutdown 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525bb60 msgr2=0x5611d52737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525bb60 0x5611d52737a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0870003ed0 tx=0x7f0870005b50 comp rx=0 tx=0).stop 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 shutdown_connections 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5611d52e60c0 0x5611d52e84e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f085effd640 0 client.15786 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5611d5266d40 0x5611d526f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5611d525c480 0x5611d526ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 --2- 172.21.15.16:0/3080863377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5611d525bb60 0x5611d52737a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 >> 172.21.15.16:0/3080863377 conn(0x5611d52580f0 msgr2=0x5611d5265220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 shutdown_connections 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 1 -- 172.21.15.16:0/3080863377 wait complete. 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.171+0000 7f0888427740 20 client.15786 trim_cache size 0 max 16384 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7e635ecc-aa95-4a24-aba8-0f49f75716d4", 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:40:02.180 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:02.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f08680a33b0 msgr2=0x7f08680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f08680a33b0 0x7f08680a5870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0878011870 tx=0x7f0878009500 comp rx=0 tx=0).stop 2024-09-15T22:40:02.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880102570 msgr2=0x7f0880198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:02.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880102570 0x7f0880198ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f087400e9f0 tx=0x7f087400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 shutdown_connections 2024-09-15T22:40:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0880102ef0 0x7f088019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0880102570 0x7f0880198ad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f08680a33b0 0x7f08680a5870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 --2- 172.21.15.16:0/2781762960 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0880100e20 0x7f0880198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 >> 172.21.15.16:0/2781762960 conn(0x7f08800fc7d0 msgr2=0x7f088006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 shutdown_connections 2024-09-15T22:40:02.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:02.172+0000 7f088604e640 1 -- 172.21.15.16:0/2781762960 wait complete. 2024-09-15T22:40:02.487 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-09-15T22:40:02.487 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-15T22:40:02.487 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-15T22:40:02.488 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-15T22:40:02.489 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-15T22:40:02.490 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-15T22:40:02.491 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-15T22:40:02.492 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-15T22:40:02.493 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-09-15T22:40:02.494 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-15T22:40:02.494 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-09-15T22:40:02.494 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-15T22:40:02.494 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-09-15T22:40:02.494 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-15T22:40:02.854 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:02 smithi016 ceph-mon[31492]: pgmap v51: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.1 MiB/s wr, 1.45k op/s 2024-09-15T22:40:02.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:02 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:40:02.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:02 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:02.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:02 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:40:02.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:02 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:02.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:02 smithi059 ceph-mon[34872]: pgmap v51: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.1 MiB/s wr, 1.45k op/s 2024-09-15T22:40:02.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:02 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:40:02.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:02 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:02.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:02 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:40:02.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:02 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:03.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:02 smithi130 ceph-mon[35742]: pgmap v51: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.1 MiB/s wr, 1.45k op/s 2024-09-15T22:40:03.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:02 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:40:03.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:02 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:03.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:02 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:40:03.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:02 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:03.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:40:03.415 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-15T22:40:03.416 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-15T22:40:03.417 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-09-15T22:40:03.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-15T22:40:03.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-09-15T22:40:03.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-15T22:40:03.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-09-15T22:40:03.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-15T22:40:03.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-09-15T22:40:03.418 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-15T22:40:03.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-09-15T22:40:03.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-15T22:40:03.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-09-15T22:40:03.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-15T22:40:03.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-09-15T22:40:03.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-15T22:40:03.419 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-09-15T22:40:03.420 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-15T22:40:03.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-09-15T22:40:03.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-15T22:40:03.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-09-15T22:40:03.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-15T22:40:03.421 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-09-15T22:40:03.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-15T22:40:03.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-09-15T22:40:03.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-15T22:40:03.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-09-15T22:40:03.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-15T22:40:03.422 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-09-15T22:40:03.423 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-15T22:40:03.423 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-09-15T22:40:03.423 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-15T22:40:03.423 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-09-15T22:40:03.423 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-15T22:40:03.423 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-09-15T22:40:03.424 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-15T22:40:03.424 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-09-15T22:40:03.424 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-15T22:40:03.424 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-09-15T22:40:03.424 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-15T22:40:03.425 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-09-15T22:40:03.425 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-15T22:40:03.425 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-09-15T22:40:03.425 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-15T22:40:03.425 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-09-15T22:40:03.425 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-09-15T22:40:03.426 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-15T22:40:03.476 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:03.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:03 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:40:03.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:03 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:40:04.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.988+0000 7f5a1f083640 1 -- 172.21.15.16:0/2429844383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 msgr2=0x7f5a181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.988+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2429844383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181067e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009900 tx=0x7f5a0402a410 comp rx=0 tx=0).stop 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.988+0000 7f5a1f083640 1 -- 172.21.15.16:0/2429844383 shutdown_connections 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.988+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2429844383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18106f10 0x7f5a1810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.988+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2429844383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18102790 0x7f5a181067e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.988+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2429844383 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a18101de0 0x7f5a181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.988+0000 7f5a1f083640 1 -- 172.21.15.16:0/2429844383 >> 172.21.15.16:0/2429844383 conn(0x7f5a180fd8e0 msgr2=0x7f5a180ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 -- 172.21.15.16:0/2429844383 shutdown_connections 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 -- 172.21.15.16:0/2429844383 wait complete. 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 Processor -- start 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 -- start start 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a18102790 0x7f5a18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18106f10 0x7f5a18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18111f70 con 0x7f5a18101de0 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18111df0 con 0x7f5a18106f10 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.989+0000 7f5a1f083640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181120f0 con 0x7f5a18102790 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a180744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59372/0 (socket says 172.21.15.16:59372) 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/2229951484 learned_addr learned my addr 172.21.15.16:0/2229951484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a17fff640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a18102790 0x7f5a18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1d620640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18106f10 0x7f5a18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/2229951484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a18102790 msgr2=0x7f5a18074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a18102790 0x7f5a18074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/2229951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18106f10 msgr2=0x7f5a18074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18106f10 0x7f5a18074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/2229951484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a181a68f0 con 0x7f5a18101de0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a180744f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00e9f0 tx=0x7f5a0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a15ffb640 1 -- 172.21.15.16:0/2229951484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c00cd80 con 0x7f5a18101de0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a181a6be0 con 0x7f5a18101de0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a15ffb640 1 -- 172.21.15.16:0/2229951484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c005500 con 0x7f5a18101de0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.990+0000 7f5a15ffb640 1 -- 172.21.15.16:0/2229951484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c018630 con 0x7f5a18101de0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.991+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a181a70a0 con 0x7f5a18101de0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.993+0000 7f5a15ffb640 1 -- 172.21.15.16:0/2229951484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5a0c0046c0 con 0x7f5a18101de0 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.993+0000 7f5a2145c740 1 Processor -- start 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.993+0000 7f5a2145c740 1 -- start start 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681c71a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a2145c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d681bca2e0 con 0x55d681c64dc0 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a2145c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d681b71ce0 con 0x55d681c6f680 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a2145c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d681bc62a0 con 0x55d681c644a0 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1d620640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1d620640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c64880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:37342/0 (socket says 172.21.15.16:37342) 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1d620640 1 -- 172.21.15.16:0/2262692490 learned_addr learned my addr 172.21.15.16:0/2262692490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c6f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59386/0 (socket says 172.21.15.16:59386) 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a15ffb640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5a080a33b0 0x7f5a080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1d620640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5a080a33b0 0x7f5a080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a17fff640 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681c71a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1d620640 1 -- 172.21.15.16:0/2262692490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 msgr2=0x55d681c71a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1d620640 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681c71a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a1d620640 1 -- 172.21.15.16:0/2262692490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 msgr2=0x55d681c6f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.994+0000 7f5a15ffb640 1 -- 172.21.15.16:0/2229951484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5a0c0d05c0 con 0x7f5a18101de0 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a1d620640 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c6f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a1d620640 1 -- 172.21.15.16:0/2262692490 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d681b56440 con 0x55d681c644a0 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a17fff640 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681c71a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a1d620640 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c64880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005c40 tx=0x7f5a10013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30657e4c6cf85c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a1d620640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5a080a33b0 0x7f5a080a5870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a1001d000 tx=0x7f5a1001f000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a02ffd640 1 -- 172.21.15.16:0/2262692490 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1000fde0 con 0x55d681c644a0 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a02ffd640 1 -- 172.21.15.16:0/2262692490 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a10022020 con 0x55d681c644a0 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.995+0000 7f5a02ffd640 1 -- 172.21.15.16:0/2262692490 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10010690 con 0x55d681c644a0 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 -- 172.21.15.16:0/2262692490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 msgr2=0x55d681c64880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c64880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005c40 tx=0x7f5a10013f50 comp rx=0 tx=0).stop 2024-09-15T22:40:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 -- 172.21.15.16:0/2262692490 shutdown_connections 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681c71a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 --2- 172.21.15.16:0/2262692490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c64880 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 -- 172.21.15.16:0/2262692490 >> 172.21.15.16:0/2262692490 conn(0x55d681c60a30 msgr2=0x55d681c60e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 -- 172.21.15.16:0/2262692490 shutdown_connections 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 -- 172.21.15.16:0/2262692490 wait complete. 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.996+0000 7f5a2145c740 1 -- start start 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 1 Processor -- start 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c6c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c6c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d681c72250 con 0x55d681c644a0 2024-09-15T22:40:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d681b85050 con 0x55d681c64dc0 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d681c723d0 con 0x55d681c6f680 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a2145c740 10 client.0 _ll_register_callbacks cb 0x55d681bc5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59396/0 (socket says 172.21.15.16:59396) 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/909505189 learned_addr learned my addr 172.21.15.16:0/909505189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a17fff640 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c6c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1d620640 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c6c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/909505189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 msgr2=0x55d681c6c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c6c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 msgr2=0x55d681c6c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c6c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.997+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/909505189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68193b3c0 con 0x55d681c644a0 2024-09-15T22:40:04.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6efd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c6a0 tx=0x7f5a0c0d3be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd6822a7a9fbbcb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f59f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f59f7fff640 1 -- 172.21.15.16:0/909505189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c098040 con 0x55d681c644a0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f59f7fff640 1 -- 172.21.15.16:0/909505189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5a0c0d01e0 con 0x55d681c644a0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a10022020 con 0x55d681c644a0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f59f7fff640 1 -- 172.21.15.16:0/909505189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c09cbe0 con 0x55d681c644a0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 10 client.15798 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d681c7dbf0 con 0x55d681c644a0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f59f7fff640 1 -- 172.21.15.16:0/909505189 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a10022020 con 0x55d681c644a0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f59f7fff640 1 -- 172.21.15.16:0/909505189 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a0c0cdc60 con 0x55d681c644a0 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 20 client.15798 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:04.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681c81470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 4 client.15798 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.998+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d681bd7850 con 0x55d681c7f000 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.999+0000 7f5a17fff640 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681c81470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.999+0000 7f5a17fff640 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681c81470 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c02fabb5f325bc server_cookie=8ac894da4228eeea in_seq=0 out_seq=0 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:03.999+0000 7f59f7fff640 10 client.15798 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f59f7fff640 1 -- 172.21.15.16:0/909505189 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d681bd7850 con 0x55d681c7f000 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f59f7fff640 10 client.15798 handle_command_reply: tid=0 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 client.15798 shutdown 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 msgr2=0x55d681c6efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6efd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c6a0 tx=0x7f5a0c0d3be0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 shutdown_connections 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681c81470 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c6c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f59f7fff640 0 client.15798 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c6c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 --2- 172.21.15.16:0/909505189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6efd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 >> 172.21.15.16:0/909505189 conn(0x55d681c60a30 msgr2=0x55d681c5fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 shutdown_connections 2024-09-15T22:40:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 1 -- 172.21.15.16:0/909505189 wait complete. 2024-09-15T22:40:04.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.000+0000 7f5a2145c740 20 client.15798 trim_cache size 0 max 16384 2024-09-15T22:40:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 Processor -- start 2024-09-15T22:40:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 -- start start 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c86d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181b39e0 con 0x55d681c644a0 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181b12b0 con 0x55d681c64dc0 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.023+0000 7f5a2145c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181af100 con 0x55d681c6f680 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59406/0 (socket says 172.21.15.16:59406) 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/178589760 learned_addr learned my addr 172.21.15.16:0/178589760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a17fff640 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c86d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1d620640 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/178589760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 msgr2=0x55d681c87270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c87270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/178589760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 msgr2=0x55d681c86d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c86d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 -- 172.21.15.16:0/178589760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d681c6b230 con 0x55d681c644a0 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6ab50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c09c470 tx=0x7f5a0c09c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a9c46bda981a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a027fc640 1 -- 172.21.15.16:0/178589760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c09c4e0 con 0x55d681c644a0 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a027fc640 1 -- 172.21.15.16:0/178589760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c0a0890 con 0x55d681c644a0 2024-09-15T22:40:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.024+0000 7f5a027fc640 1 -- 172.21.15.16:0/178589760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0cdbd0 con 0x55d681c644a0 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 -- 172.21.15.16:0/178589760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 msgr2=0x55d681c6ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6ab50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c09c470 tx=0x7f5a0c09c0e0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 -- 172.21.15.16:0/178589760 shutdown_connections 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c6f680 0x55d681c87270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c64dc0 0x55d681c86d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 --2- 172.21.15.16:0/178589760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c644a0 0x55d681c6ab50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 -- 172.21.15.16:0/178589760 >> 172.21.15.16:0/178589760 conn(0x55d681c60a30 msgr2=0x55d681c5fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 -- 172.21.15.16:0/178589760 shutdown_connections 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 -- 172.21.15.16:0/178589760 wait complete. 2024-09-15T22:40:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.025+0000 7f5a2145c740 1 -- start start 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 1 Processor -- start 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c75a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c75fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681ce8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181af770 con 0x55d681c64dc0 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181b0be0 con 0x55d681c6f680 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181afe40 con 0x55d681c644a0 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a2145c740 10 client.0 _ll_register_callbacks cb 0x7f5a0c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c75fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c75fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59420/0 (socket says 172.21.15.16:59420) 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 -- 172.21.15.16:0/3461532596 learned_addr learned my addr 172.21.15.16:0/3461532596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c75a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a1d620640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681ce8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 -- 172.21.15.16:0/3461532596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 msgr2=0x55d681c75a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c75a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 -- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 msgr2=0x55d681ce8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681ce8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a17fff640 1 -- 172.21.15.16:0/3461532596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d681c6f340 con 0x55d681c64dc0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.026+0000 7f5a1d620640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681ce8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a17fff640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c75fb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009230 tx=0x7f5a04031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12db1fe414a938e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f59f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f59f7fff640 1 -- 172.21.15.16:0/3461532596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04035040 con 0x55d681c64dc0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c0a0890 con 0x55d681c64dc0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f59f7fff640 1 -- 172.21.15.16:0/3461532596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5a04004240 con 0x55d681c64dc0 2024-09-15T22:40:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f59f7fff640 1 -- 172.21.15.16:0/3461532596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a04057050 con 0x55d681c64dc0 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 10 client.15804 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d681cea3c0 con 0x55d681c64dc0 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f59f7fff640 1 -- 172.21.15.16:0/3461532596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c0a0890 con 0x55d681c64dc0 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f59f7fff640 1 -- 172.21.15.16:0/3461532596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a04004b00 con 0x55d681c64dc0 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 10 client.15804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 20 client.15804 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681cecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 4 client.15804 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.027+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d681bd1d60 con 0x55d681c7f000 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681cecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f59f7fff640 10 client.15804 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f5a1ce1f640 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681cecb00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5239f319ec27ee75 server_cookie=ba9be5cc37aadf38 in_seq=0 out_seq=0 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f59f7fff640 1 -- 172.21.15.16:0/3461532596 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d681bd1d60 con 0x55d681c7f000 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f59f7fff640 10 client.15804 handle_command_reply: tid=0 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f5a2145c740 1 client.15804 shutdown 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 msgr2=0x55d681c75fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.028+0000 7f5a2145c740 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c75fb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009230 tx=0x7f5a04031a00 comp rx=0 tx=0).stop 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 shutdown_connections 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d681c7f000 0x55d681cecb00 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d681c6f680 0x55d681ce8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d681c64dc0 0x55d681c75fb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f59f7fff640 0 client.15804 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 --2- 172.21.15.16:0/3461532596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d681c644a0 0x55d681c75a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 >> 172.21.15.16:0/3461532596 conn(0x55d681c60a30 msgr2=0x55d681c6dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 shutdown_connections 2024-09-15T22:40:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 1 -- 172.21.15.16:0/3461532596 wait complete. 2024-09-15T22:40:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a2145c740 20 client.15804 trim_cache size 0 max 16384 2024-09-15T22:40:04.036 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:04.036 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:40:04.036 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:40:04.036 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.029+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5a080a33b0 msgr2=0x7f5a080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5a080a33b0 0x7f5a080a5870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5a1001d000 tx=0x7f5a1001f000 comp rx=0 tx=0).stop 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 msgr2=0x7f5a180744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a180744f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00e9f0 tx=0x7f5a0c00edf0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 shutdown_connections 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a18106f10 0x7f5a18074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5a080a33b0 0x7f5a080a5870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5a18102790 0x7f5a18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 --2- 172.21.15.16:0/2229951484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a18101de0 0x7f5a180744f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 >> 172.21.15.16:0/2229951484 conn(0x7f5a180fd8e0 msgr2=0x7f5a180fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 shutdown_connections 2024-09-15T22:40:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:04.030+0000 7f5a1f083640 1 -- 172.21.15.16:0/2229951484 wait complete. 2024-09-15T22:40:04.038 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:03 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-09-15T22:40:04.566 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-09-15T22:40:04.567 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-15T22:40:04.568 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-09-15T22:40:04.569 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-15T22:40:04.570 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-15T22:40:04.571 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-09-15T22:40:04.572 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-15T22:40:04.572 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-09-15T22:40:04.572 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-15T22:40:04.572 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-09-15T22:40:04.572 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-15T22:40:04.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:04 smithi016 ceph-mon[31492]: pgmap v52: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 781 KiB/s wr, 977 op/s 2024-09-15T22:40:04.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:04 smithi059 ceph-mon[34872]: pgmap v52: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 781 KiB/s wr, 977 op/s 2024-09-15T22:40:04.945 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:40:04.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-15T22:40:04.968 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-15T22:40:04.969 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-15T22:40:04.970 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-15T22:40:04.971 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-15T22:40:04.972 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-15T22:40:04.973 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-09-15T22:40:04.974 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-15T22:40:05.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:04 smithi130 ceph-mon[35742]: pgmap v52: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 781 KiB/s wr, 977 op/s 2024-09-15T22:40:05.220 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:05.401 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-09-15T22:40:05.401 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-15T22:40:05.401 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-09-15T22:40:05.401 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-15T22:40:05.401 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-09-15T22:40:05.401 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-15T22:40:05.402 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-15T22:40:05.403 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-09-15T22:40:05.404 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-09-15T22:40:05.405 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-15T22:40:05.406 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-09-15T22:40:05.407 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-15T22:40:05.408 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-15T22:40:05.409 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-15T22:40:05.410 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-09-15T22:40:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 -- 172.21.15.16:0/873269160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c101de0 msgr2=0x7f5c1c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/873269160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c101de0 0x7f5c1c1021c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009900 tx=0x7f5c0c02a410 comp rx=0 tx=0).stop 2024-09-15T22:40:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 -- 172.21.15.16:0/873269160 shutdown_connections 2024-09-15T22:40:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/873269160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c1c106f10 0x7f5c1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/873269160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c1c102790 0x7f5c1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/873269160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c101de0 0x7f5c1c1021c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 -- 172.21.15.16:0/873269160 >> 172.21.15.16:0/873269160 conn(0x7f5c1c0fd8c0 msgr2=0x7f5c1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 -- 172.21.15.16:0/873269160 shutdown_connections 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.750+0000 7f5c23cd3640 1 -- 172.21.15.16:0/873269160 wait complete. 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 Processor -- start 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 -- start start 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c1c101de0 0x7f5c1c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c102790 0x7f5c1c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c1c106f10 0x7f5c1c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c111f70 con 0x7f5c1c102790 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c111df0 con 0x7f5c1c106f10 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c23cd3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c1120f0 con 0x7f5c1c101de0 2024-09-15T22:40:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c2126e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c102790 0x7f5c1c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c2126e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c102790 0x7f5c1c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59428/0 (socket says 172.21.15.16:59428) 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c2126e640 1 -- 172.21.15.16:0/963646353 learned_addr learned my addr 172.21.15.16:0/963646353 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c21a6f640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c1c101de0 0x7f5c1c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.751+0000 7f5c22270640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c1c106f10 0x7f5c1c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c2126e640 1 -- 172.21.15.16:0/963646353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c1c101de0 msgr2=0x7f5c1c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c2126e640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c1c101de0 0x7f5c1c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c2126e640 1 -- 172.21.15.16:0/963646353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c1c106f10 msgr2=0x7f5c1c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c2126e640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c1c106f10 0x7f5c1c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c2126e640 1 -- 172.21.15.16:0/963646353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c19d4c0 con 0x7f5c1c102790 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c2126e640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c102790 0x7f5c1c198a50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000e9f0 tx=0x7f5c1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c1affd640 1 -- 172.21.15.16:0/963646353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c1000ccc0 con 0x7f5c1c102790 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c1affd640 1 -- 172.21.15.16:0/963646353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c10010910 con 0x7f5c1c102790 2024-09-15T22:40:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1c19d7b0 con 0x7f5c1c102790 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c1affd640 1 -- 172.21.15.16:0/963646353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c100188a0 con 0x7f5c1c102790 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.752+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1c1a5090 con 0x7f5c1c102790 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.753+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1c1021c0 con 0x7f5c1c102790 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.754+0000 7f5c1affd640 1 -- 172.21.15.16:0/963646353 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5c10010ac0 con 0x7f5c1c102790 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.755+0000 7f5c1affd640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5c040a33b0 0x7f5c040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.755+0000 7f5c21a6f640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5c040a33b0 0x7f5c040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.756+0000 7f5c1affd640 1 -- 172.21.15.16:0/963646353 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5c100d04d0 con 0x7f5c1c102790 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.756+0000 7f5c21a6f640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5c040a33b0 0x7f5c040a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009230 tx=0x7f5c0c0058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.761+0000 7f5c1affd640 1 -- 172.21.15.16:0/963646353 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c1009caf0 con 0x7f5c1c102790 2024-09-15T22:40:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.990+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c1c10a120 con 0x7f5c1c102790 2024-09-15T22:40:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.991+0000 7f5c1affd640 1 -- 172.21.15.16:0/963646353 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f5c1009c3f0 con 0x7f5c1c102790 2024-09-15T22:40:05.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:40:05.993 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:40:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:40:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.993+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5c040a33b0 msgr2=0x7f5c040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.993+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5c040a33b0 0x7f5c040a5870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009230 tx=0x7f5c0c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.993+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c102790 msgr2=0x7f5c1c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.993+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c102790 0x7f5c1c198a50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000e9f0 tx=0x7f5c1000edf0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 shutdown_connections 2024-09-15T22:40:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c1c106f10 0x7f5c1c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5c1c102790 0x7f5c1c198a50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5c040a33b0 0x7f5c040a5870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 --2- 172.21.15.16:0/963646353 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5c1c101de0 0x7f5c1c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 >> 172.21.15.16:0/963646353 conn(0x7f5c1c0fd8c0 msgr2=0x7f5c1c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 shutdown_connections 2024-09-15T22:40:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:05.994+0000 7f5c23cd3640 1 -- 172.21.15.16:0/963646353 wait complete. 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-09-15T22:40:06.606 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-15T22:40:06.607 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-15T22:40:06.608 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-15T22:40:06.609 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-09-15T22:40:06.610 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-15T22:40:06.611 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-09-15T22:40:06.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:40:06.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:06 smithi059 ceph-mon[34872]: pgmap v53: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 781 KiB/s wr, 977 op/s 2024-09-15T22:40:06.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:06 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/963646353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:07.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:06 smithi130 ceph-mon[35742]: pgmap v53: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 781 KiB/s wr, 977 op/s 2024-09-15T22:40:07.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:06 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/963646353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:07.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:06 smithi016 ceph-mon[31492]: pgmap v53: 129 pgs: 129 active+clean; 170 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 781 KiB/s wr, 977 op/s 2024-09-15T22:40:07.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:06 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/963646353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-09-15T22:40:07.472 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-09-15T22:40:07.473 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-09-15T22:40:07.474 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-15T22:40:07.475 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-15T22:40:07.476 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-15T22:40:07.477 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-09-15T22:40:07.478 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-15T22:40:07.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.490+0000 7fb7b3684640 1 -- 172.21.15.16:0/2764361355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102ef0 msgr2=0x7fb7ac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.490+0000 7fb7b3684640 1 --2- 172.21.15.16:0/2764361355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102ef0 0x7fb7ac10da40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009ca0 tx=0x7fb7a402a950 comp rx=0 tx=0).stop 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.490+0000 7fb7b3684640 1 -- 172.21.15.16:0/2764361355 shutdown_connections 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.490+0000 7fb7b3684640 1 --2- 172.21.15.16:0/2764361355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102ef0 0x7fb7ac10da40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.490+0000 7fb7b3684640 1 --2- 172.21.15.16:0/2764361355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ac102570 0x7fb7ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.490+0000 7fb7b3684640 1 --2- 172.21.15.16:0/2764361355 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7ac100e20 0x7fb7ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.490+0000 7fb7b3684640 1 -- 172.21.15.16:0/2764361355 >> 172.21.15.16:0/2764361355 conn(0x7fb7ac0fc7d0 msgr2=0x7fb7ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 -- 172.21.15.16:0/2764361355 shutdown_connections 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 -- 172.21.15.16:0/2764361355 wait complete. 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 Processor -- start 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 -- start start 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ac100e20 0x7fb7ac19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102570 0x7fb7ac19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7ac102ef0 0x7fb7ac19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac110f80 con 0x7fb7ac102570 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac110e00 con 0x7fb7ac100e20 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.491+0000 7fb7b3684640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac111100 con 0x7fb7ac102ef0 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102570 0x7fb7ac19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b1420640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ac100e20 0x7fb7ac19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102570 0x7fb7ac19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59446/0 (socket says 172.21.15.16:59446) 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/1567231676 learned_addr learned my addr 172.21.15.16:0/1567231676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b1c21640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7ac102ef0 0x7fb7ac19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/1567231676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7ac102ef0 msgr2=0x7fb7ac19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7ac102ef0 0x7fb7ac19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/1567231676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ac100e20 msgr2=0x7fb7ac19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ac100e20 0x7fb7ac19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/1567231676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac19f610 con 0x7fb7ac102570 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102570 0x7fb7ac19aba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9f0 tx=0x7fb79c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7a27fc640 1 -- 172.21.15.16:0/1567231676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00cd80 con 0x7fb7ac102570 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ac19f900 con 0x7fb7ac102570 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7a27fc640 1 -- 172.21.15.16:0/1567231676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79c010910 con 0x7fb7ac102570 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7a27fc640 1 -- 172.21.15.16:0/1567231676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0188f0 con 0x7fb7ac102570 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.492+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ac1a71e0 con 0x7fb7ac102570 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.495+0000 7fb7a27fc640 1 -- 172.21.15.16:0/1567231676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb79c010ac0 con 0x7fb7ac102570 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.495+0000 7fb7b5a5d740 1 Processor -- start 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.495+0000 7fb7b5a5d740 1 -- start start 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd28b40 0x564a3dd27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd34760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a3dc8d2f0 con 0x564a3dd28b40 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a3dc34cf0 con 0x564a3dd32370 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564a3dc892b0 con 0x564a3dd275d0 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b1c21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd28b40 0x564a3dd27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b1c21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd28b40 0x564a3dd27090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59462/0 (socket says 172.21.15.16:59462) 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b1c21640 1 -- 172.21.15.16:0/2849150581 learned_addr learned my addr 172.21.15.16:0/2849150581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7a27fc640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7980a33b0 0x7fb7980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b1420640 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7a27fc640 1 -- 172.21.15.16:0/1567231676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb79c0d0480 con 0x7fb7ac102570 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd34760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b1420640 1 -- 172.21.15.16:0/2849150581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 msgr2=0x564a3dd34760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.496+0000 7fb7b1420640 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd34760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7b1420640 1 -- 172.21.15.16:0/2849150581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd28b40 msgr2=0x564a3dd27090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7b1420640 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd28b40 0x564a3dd27090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7b1420640 1 -- 172.21.15.16:0/2849150581 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3dc193a0 con 0x564a3dd275d0 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7b1c21640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7980a33b0 0x7fb7980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7b1420640 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd27a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009c20 tx=0x7fb7a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61aaf4ccb6ee1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7977fe640 1 -- 172.21.15.16:0/2849150581 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8009dd0 con 0x564a3dd275d0 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7977fe640 1 -- 172.21.15.16:0/2849150581 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a8005020 con 0x564a3dd275d0 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.497+0000 7fb7977fe640 1 -- 172.21.15.16:0/2849150581 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8004b90 con 0x564a3dd275d0 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2849150581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 msgr2=0x564a3dd27a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b1c21640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7980a33b0 0x7fb7980a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009ca0 tx=0x7fb7a403c040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd27a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009c20 tx=0x7fb7a8011810 comp rx=0 tx=0).stop 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2849150581 shutdown_connections 2024-09-15T22:40:07.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd27a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2849150581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd28b40 0x564a3dd27090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2849150581 >> 172.21.15.16:0/2849150581 conn(0x564a3dcf0310 msgr2=0x564a3dd23c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2849150581 shutdown_connections 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2849150581 wait complete. 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.498+0000 7fb7b5a5d740 1 -- start start 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 1 Processor -- start 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd31d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd2efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd32370 0x564a3dd2f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564a3dd35070 con 0x564a3dd32370 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564a3dc48060 con 0x564a3dd28b40 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564a3dd351f0 con 0x564a3dd275d0 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b5a5d740 10 client.0 _ll_register_callbacks cb 0x564a3d99b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b1c21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd32370 0x564a3dd2f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd2efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd2efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:56648/0 (socket says 172.21.15.16:56648) 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/3032988144 learned_addr learned my addr 172.21.15.16:0/3032988144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b1420640 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd31d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 msgr2=0x564a3dd31d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd31d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/3032988144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd32370 msgr2=0x564a3dd2f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd32370 0x564a3dd2f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/3032988144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3d9fe3c0 con 0x564a3dd28b40 2024-09-15T22:40:07.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.499+0000 7fb7b1c21640 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd32370 0x564a3dd2f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd2efb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00c6a0 tx=0x7fb79c09c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cac9a6c1ceb8e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb794ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb794ff9640 1 -- 172.21.15.16:0/3032988144 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c098b20 con 0x564a3dd28b40 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a8005020 con 0x564a3dd28b40 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb794ff9640 1 -- 172.21.15.16:0/3032988144 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb79c09b900 con 0x564a3dd28b40 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb794ff9640 1 -- 172.21.15.16:0/3032988144 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb79c098cc0 con 0x564a3dd28b40 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb7b5a5d740 10 client.25439 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564a3dd40a60 con 0x564a3dd28b40 2024-09-15T22:40:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb794ff9640 1 -- 172.21.15.16:0/3032988144 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a8005020 con 0x564a3dd28b40 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.500+0000 7fb794ff9640 1 -- 172.21.15.16:0/3032988144 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb79c098390 con 0x564a3dd28b40 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b5a5d740 10 client.25439 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b5a5d740 10 client.25439 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b5a5d740 20 client.25439 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3dd41e70 0x564a3dd442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b5a5d740 4 client.25439 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a3dc9a860 con 0x564a3dd41e70 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b1420640 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3dd41e70 0x564a3dd442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb794ff9640 10 client.25439 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.501+0000 7fb7b1420640 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3dd41e70 0x564a3dd442b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=15f73db3de0b4a37 server_cookie=853dafa114b67bc1 in_seq=0 out_seq=0 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb794ff9640 1 -- 172.21.15.16:0/3032988144 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a3dc9a860 con 0x564a3dd41e70 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb794ff9640 10 client.25439 handle_command_reply: tid=0 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 client.25439 shutdown 2024-09-15T22:40:07.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 msgr2=0x564a3dd2efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd2efb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00c6a0 tx=0x7fb79c09c5f0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 shutdown_connections 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3dd41e70 0x564a3dd442b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd32370 0x564a3dd2f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd2efb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3032988144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd275d0 0x564a3dd31d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb794ff9640 0 client.25439 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 >> 172.21.15.16:0/3032988144 conn(0x564a3dcf0310 msgr2=0x564a3dd29960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 shutdown_connections 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.502+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3032988144 wait complete. 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.503+0000 7fb7b5a5d740 20 client.25439 trim_cache size 0 max 16384 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 Processor -- start 2024-09-15T22:40:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 -- start start 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd28b40 0x564a3dd28300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd2a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1b3a30 con 0x564a3dd275d0 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1b1300 con 0x564a3dd32370 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.517+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1af150 con 0x564a3dd28b40 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1c21640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd2a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59478/0 (socket says 172.21.15.16:59478) 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 -- 172.21.15.16:0/3698617277 learned_addr learned my addr 172.21.15.16:0/3698617277 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd28b40 0x564a3dd28300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 -- 172.21.15.16:0/3698617277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd28b40 msgr2=0x564a3dd28300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd28b40 0x564a3dd28300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 -- 172.21.15.16:0/3698617277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 msgr2=0x564a3dd2a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd2a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 -- 172.21.15.16:0/3698617277 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3dd2a120 con 0x564a3dd275d0 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1c21640 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd2a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb7b1420640 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3e310 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009b40 tx=0x7fb7a8005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8be2c9e97d1c230 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb796ffd640 1 -- 172.21.15.16:0/3698617277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8030070 con 0x564a3dd275d0 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb796ffd640 1 -- 172.21.15.16:0/3698617277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a80035f0 con 0x564a3dd275d0 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.518+0000 7fb796ffd640 1 -- 172.21.15.16:0/3698617277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8004520 con 0x564a3dd275d0 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3698617277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 msgr2=0x564a3dd3e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3e310 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009b40 tx=0x7fb7a8005da0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3698617277 shutdown_connections 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd32370 0x564a3dd2a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd28b40 0x564a3dd28300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/3698617277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3e310 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3698617277 >> 172.21.15.16:0/3698617277 conn(0x564a3dcf0310 msgr2=0x564a3dd1ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3698617277 shutdown_connections 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.519+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/3698617277 wait complete. 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 -- start start 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 Processor -- start 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd3a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd32370 0x564a3dd2b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1af7c0 con 0x564a3dd275d0 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1b0c30 con 0x564a3dd28b40 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac1afe90 con 0x564a3dd32370 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b5a5d740 10 client.0 _ll_register_callbacks cb 0x7fb79c098090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b0c1f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd3a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b1420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b1c21640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd32370 0x564a3dd2b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b0c1f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd3a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:56668/0 (socket says 172.21.15.16:56668) 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.520+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/2521180818 learned_addr learned my addr 172.21.15.16:0/2521180818 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/2521180818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd32370 msgr2=0x564a3dd2b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd32370 0x564a3dd2b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/2521180818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 msgr2=0x564a3dd3a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b0c1f640 1 -- 172.21.15.16:0/2521180818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3dd3b100 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b1420640 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b0c1f640 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd3a9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb79c09c3f0 tx=0x7fb79c0a08e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=265e3e125101d1db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb794ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb794ff9640 1 -- 172.21.15.16:0/2521180818 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c098b20 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a80035f0 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb794ff9640 1 -- 172.21.15.16:0/2521180818 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb79c09bb00 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb794ff9640 1 -- 172.21.15.16:0/2521180818 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb79c0ac050 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b5a5d740 10 client.25445 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564a3ddac130 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb794ff9640 1 -- 172.21.15.16:0/2521180818 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a80035f0 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.521+0000 7fb794ff9640 1 -- 172.21.15.16:0/2521180818 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb79c098200 con 0x564a3dd28b40 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 10 client.25445 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 10 client.25445 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 20 client.25445 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3ddac8b0 0x564a3ddaecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 4 client.25445 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a3dc94d70 con 0x564a3ddac8b0 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b1420640 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3ddac8b0 0x564a3ddaecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b1420640 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3ddac8b0 0x564a3ddaecd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80f64b1a4fa25357 server_cookie=46915e48494567cb in_seq=0 out_seq=0 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb794ff9640 10 client.25445 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb794ff9640 1 -- 172.21.15.16:0/2521180818 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a3dc94d70 con 0x564a3ddac8b0 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb794ff9640 10 client.25445 handle_command_reply: tid=0 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 client.25445 shutdown 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 msgr2=0x564a3dd3a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd3a9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb79c09c3f0 tx=0x7fb79c0a08e0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.546 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:40:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 shutdown_connections 2024-09-15T22:40:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x564a3ddac8b0 0x564a3ddaecd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564a3dd32370 0x564a3dd2b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb794ff9640 0 client.25445 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564a3dd28b40 0x564a3dd3a9c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 --2- 172.21.15.16:0/2521180818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564a3dd275d0 0x564a3dd3a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.522+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 >> 172.21.15.16:0/2521180818 conn(0x564a3dcf0310 msgr2=0x564a3dd2a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 shutdown_connections 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b5a5d740 1 -- 172.21.15.16:0/2521180818 wait complete. 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b5a5d740 20 client.25445 trim_cache size 0 max 16384 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7980a33b0 msgr2=0x7fb7980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7980a33b0 0x7fb7980a5870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009ca0 tx=0x7fb7a403c040 comp rx=0 tx=0).stop 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102570 msgr2=0x7fb7ac19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102570 0x7fb7ac19aba0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00e9f0 tx=0x7fb79c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 shutdown_connections 2024-09-15T22:40:07.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7980a33b0 0x7fb7980a5870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb7ac102ef0 0x7fb7ac19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7ac102570 0x7fb7ac19aba0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 --2- 172.21.15.16:0/1567231676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7ac100e20 0x7fb7ac19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.523+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 >> 172.21.15.16:0/1567231676 conn(0x7fb7ac0fc7d0 msgr2=0x7fb7ac069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:07.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.524+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 shutdown_connections 2024-09-15T22:40:07.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:07.524+0000 7fb7b3684640 1 -- 172.21.15.16:0/1567231676 wait complete. 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-09-15T22:40:07.834 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-09-15T22:40:07.835 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-15T22:40:07.836 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-15T22:40:07.837 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-09-15T22:40:07.838 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-09-15T22:40:07.839 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-15T22:40:07.840 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-09-15T22:40:07.840 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-15T22:40:07.840 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-09-15T22:40:07.840 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-15T22:40:07.840 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-09-15T22:40:08.513 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-15T22:40:08.513 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-09-15T22:40:08.513 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-15T22:40:08.513 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-09-15T22:40:08.513 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-15T22:40:08.513 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-09-15T22:40:08.513 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-09-15T22:40:08.514 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-09-15T22:40:08.515 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-15T22:40:08.516 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-15T22:40:08.517 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-15T22:40:08.518 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-09-15T22:40:08.519 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-15T22:40:08.519 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-09-15T22:40:08.519 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-15T22:40:08.519 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-09-15T22:40:08.519 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-15T22:40:08.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:40:08.827 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:08 smithi016 ceph-mon[31492]: pgmap v54: 129 pgs: 129 active+clean; 102 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 1.2 MiB/s wr, 1.12k op/s 2024-09-15T22:40:08.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:08 smithi059 ceph-mon[34872]: pgmap v54: 129 pgs: 129 active+clean; 102 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 1.2 MiB/s wr, 1.12k op/s 2024-09-15T22:40:09.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:08 smithi130 ceph-mon[35742]: pgmap v54: 129 pgs: 129 active+clean; 102 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 59 MiB/s rd, 1.2 MiB/s wr, 1.12k op/s 2024-09-15T22:40:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 -- 172.21.15.16:0/1040667457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 msgr2=0x7f4c7c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 --2- 172.21.15.16:0/1040667457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c107b80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009900 tx=0x7f4c6c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 -- 172.21.15.16:0/1040667457 shutdown_connections 2024-09-15T22:40:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 --2- 172.21.15.16:0/1040667457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c7c108290 0x7f4c7c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 --2- 172.21.15.16:0/1040667457 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c107b80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 --2- 172.21.15.16:0/1040667457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c7c102d90 0x7f4c7c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 -- 172.21.15.16:0/1040667457 >> 172.21.15.16:0/1040667457 conn(0x7f4c7c0fc7d0 msgr2=0x7f4c7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 -- 172.21.15.16:0/1040667457 shutdown_connections 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.721+0000 7f4c8468f640 1 -- 172.21.15.16:0/1040667457 wait complete. 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 Processor -- start 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 -- start start 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c7c102d90 0x7f4c7c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c7c108290 0x7f4c7c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c112090 con 0x7f4c7c108290 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c111f10 con 0x7f4c7c102d90 2024-09-15T22:40:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c8468f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c112210 con 0x7f4c7c103740 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c81c2a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c81c2a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:37452/0 (socket says 172.21.15.16:37452) 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c82c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c7c108290 0x7f4c7c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.722+0000 7f4c81c2a640 1 -- 172.21.15.16:0/3935719793 learned_addr learned my addr 172.21.15.16:0/3935719793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c8242b640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c7c102d90 0x7f4c7c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c81c2a640 1 -- 172.21.15.16:0/3935719793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c7c102d90 msgr2=0x7f4c7c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c7c102d90 0x7f4c7c1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c81c2a640 1 -- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c7c108290 msgr2=0x7f4c7c101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c7c108290 0x7f4c7c101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c81c2a640 1 -- 172.21.15.16:0/3935719793 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c1a6a70 con 0x7f4c7c103740 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c8242b640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c7c102d90 0x7f4c7c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c100f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005940 tx=0x7f4c6c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c737fe640 1 -- 172.21.15.16:0/3935719793 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c03f070 con 0x7f4c7c103740 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c737fe640 1 -- 172.21.15.16:0/3935719793 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c031c30 con 0x7f4c7c103740 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c1a6d00 con 0x7f4c7c103740 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.723+0000 7f4c737fe640 1 -- 172.21.15.16:0/3935719793 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c035030 con 0x7f4c7c103740 2024-09-15T22:40:09.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.724+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c1a71d0 con 0x7f4c7c103740 2024-09-15T22:40:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.726+0000 7f4c737fe640 1 -- 172.21.15.16:0/3935719793 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4c6c042070 con 0x7f4c7c103740 2024-09-15T22:40:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.727+0000 7f4c737fe640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4c680a3430 0x7f4c680a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.727+0000 7f4c737fe640 1 -- 172.21.15.16:0/3935719793 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f4c6c0f07a0 con 0x7f4c7c103740 2024-09-15T22:40:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.727+0000 7f4c8242b640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4c680a3430 0x7f4c680a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.727+0000 7f4c86a68740 1 Processor -- start 2024-09-15T22:40:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c86a68740 1 -- start start 2024-09-15T22:40:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2272090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2272a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c8242b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2272090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c86a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5a21d82f0 con 0x55f5a227d370 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c86a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f5a217fcf0 con 0x55f5a2273b40 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c86a68740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f5a21d42b0 con 0x55f5a22725d0 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c8242b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2272090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:56680/0 (socket says 172.21.15.16:56680) 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c81c2a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2272a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.728+0000 7f4c8242b640 1 -- 172.21.15.16:0/2662092550 learned_addr learned my addr 172.21.15.16:0/2662092550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c81c2a640 1 -- 172.21.15.16:0/2662092550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 msgr2=0x55f5a2272090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2272090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c81c2a640 1 -- 172.21.15.16:0/2662092550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 msgr2=0x55f5a227f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c81c2a640 1 -- 172.21.15.16:0/2662092550 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a21643a0 con 0x55f5a22725d0 2024-09-15T22:40:09.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2272a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0edb30 tx=0x7f4c6c02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0645840d62e8e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c8242b640 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2272090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c8242b640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4c680a3430 0x7f4c680a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009c30 tx=0x7f4c78005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.729+0000 7f4c70ff9640 1 -- 172.21.15.16:0/2662092550 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c049040 con 0x55f5a22725d0 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.730+0000 7f4c70ff9640 1 -- 172.21.15.16:0/2662092550 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c0bcdc0 con 0x55f5a22725d0 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.730+0000 7f4c70ff9640 1 -- 172.21.15.16:0/2662092550 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0b8d30 con 0x55f5a22725d0 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 -- 172.21.15.16:0/2662092550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 msgr2=0x55f5a2272a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2272a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0edb30 tx=0x7f4c6c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 -- 172.21.15.16:0/2662092550 shutdown_connections 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2272a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 --2- 172.21.15.16:0/2662092550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2272090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 -- 172.21.15.16:0/2662092550 >> 172.21.15.16:0/2662092550 conn(0x55f5a223b310 msgr2=0x55f5a226ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 -- 172.21.15.16:0/2662092550 shutdown_connections 2024-09-15T22:40:09.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.731+0000 7f4c86a68740 1 -- 172.21.15.16:0/2662092550 wait complete. 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 -- start start 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 Processor -- start 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a227ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a227a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f5a2288640 con 0x55f5a227d370 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f5a2193060 con 0x55f5a2273b40 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f5a22887c0 con 0x55f5a22725d0 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.732+0000 7f4c86a68740 10 client.0 _ll_register_callbacks cb 0x55f5a1ee68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59504/0 (socket says 172.21.15.16:59504) 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 -- 172.21.15.16:0/1335021377 learned_addr learned my addr 172.21.15.16:0/1335021377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c8242b640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a227ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 -- 172.21.15.16:0/1335021377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 msgr2=0x55f5a227ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a227ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 -- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 msgr2=0x55f5a227a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a227a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a227a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c82c2c640 1 -- 172.21.15.16:0/1335021377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a1f493c0 con 0x55f5a227d370 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.733+0000 7f4c8242b640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a227ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227a840 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4c74013920 tx=0x7f4c74013de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0522483007058eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c65ffb640 1 -- 172.21.15.16:0/1335021377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74003e40 con 0x55f5a227d370 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c6c0bcdc0 con 0x55f5a227d370 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c65ffb640 1 -- 172.21.15.16:0/1335021377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f4c740119a0 con 0x55f5a227d370 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c65ffb640 1 -- 172.21.15.16:0/1335021377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c74011ba0 con 0x55f5a227d370 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c86a68740 10 client.15837 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:09.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.734+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5a22940f0 con 0x55f5a227d370 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c65ffb640 1 -- 172.21.15.16:0/1335021377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c0bcdc0 con 0x55f5a227d370 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c65ffb640 1 -- 172.21.15.16:0/1335021377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4c7401a070 con 0x55f5a227d370 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c86a68740 10 client.15837 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c86a68740 10 client.15837 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c86a68740 20 client.15837 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c86a68740 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2295500 0x55f5a22978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c86a68740 4 client.15837 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5a21e5860 con 0x55f5a2295500 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c8242b640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2295500 0x55f5a22978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c8242b640 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2295500 0x55f5a22978f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=211953908715be85 server_cookie=1c12f38e501cb124 in_seq=0 out_seq=0 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.735+0000 7f4c65ffb640 10 client.15837 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c65ffb640 1 -- 172.21.15.16:0/1335021377 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5a21e5860 con 0x55f5a2295500 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c65ffb640 10 client.15837 handle_command_reply: tid=0 2024-09-15T22:40:09.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 client.15837 shutdown 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 msgr2=0x55f5a227a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227a840 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4c74013920 tx=0x7f4c74013de0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 shutdown_connections 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2295500 0x55f5a22978f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c65ffb640 0 client.15837 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a227a840 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a227a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 --2- 172.21.15.16:0/1335021377 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a227ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.736+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 >> 172.21.15.16:0/1335021377 conn(0x55f5a223b310 msgr2=0x55f5a2274960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.737+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 shutdown_connections 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.737+0000 7f4c86a68740 1 -- 172.21.15.16:0/1335021377 wait complete. 2024-09-15T22:40:09.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.737+0000 7f4c86a68740 20 client.15837 trim_cache size 0 max 16384 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-09-15T22:40:09.747 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-09-15T22:40:09.748 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-15T22:40:09.749 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-09-15T22:40:09.750 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-09-15T22:40:09.751 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-09-15T22:40:09.752 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-15T22:40:09.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-09-15T22:40:09.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-15T22:40:09.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 Processor -- start 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 -- start start 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2275120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2275660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a229d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1af700 con 0x55f5a227d370 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1acfd0 con 0x55f5a2273b40 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.751+0000 7f4c86a68740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1aae20 con 0x55f5a22725d0 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a229d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a229d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59516/0 (socket says 172.21.15.16:59516) 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 -- 172.21.15.16:0/845428625 learned_addr learned my addr 172.21.15.16:0/845428625 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c8242b640 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2275120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2275660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 -- 172.21.15.16:0/845428625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 msgr2=0x55f5a2275120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2275120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 -- 172.21.15.16:0/845428625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 msgr2=0x55f5a2275660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2275660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 -- 172.21.15.16:0/845428625 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a226e4f0 con 0x55f5a227d370 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a229d1a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4c74000f80 tx=0x7f4c74011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d0417d7db9f822 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c67fff640 1 -- 172.21.15.16:0/845428625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7402f070 con 0x55f5a227d370 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c67fff640 1 -- 172.21.15.16:0/845428625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7401c440 con 0x55f5a227d370 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.752+0000 7f4c67fff640 1 -- 172.21.15.16:0/845428625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7401ce60 con 0x55f5a227d370 2024-09-15T22:40:09.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 -- 172.21.15.16:0/845428625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 msgr2=0x55f5a229d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a229d1a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4c74000f80 tx=0x7f4c74011860 comp rx=0 tx=0).stop 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 -- 172.21.15.16:0/845428625 shutdown_connections 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a227d370 0x55f5a229d1a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a2273b40 0x55f5a2275660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 --2- 172.21.15.16:0/845428625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a2275120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 -- 172.21.15.16:0/845428625 >> 172.21.15.16:0/845428625 conn(0x55f5a223b310 msgr2=0x55f5a22791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 -- 172.21.15.16:0/845428625 shutdown_connections 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.753+0000 7f4c86a68740 1 -- 172.21.15.16:0/845428625 wait complete. 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 -- start start 2024-09-15T22:40:09.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 Processor -- start 2024-09-15T22:40:09.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a228e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a2273b40 0x55f5a2289490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a227d370 0x55f5a22899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ab490 con 0x55f5a2273b40 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1ac900 con 0x55f5a227d370 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c1abb60 con 0x55f5a22725d0 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c86a68740 10 client.0 _ll_register_callbacks cb 0x7f4c74009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c81c2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a2273b40 0x55f5a2289490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c81c2a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a2273b40 0x55f5a2289490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59518/0 (socket says 172.21.15.16:59518) 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c81c2a640 1 -- 172.21.15.16:0/64372342 learned_addr learned my addr 172.21.15.16:0/64372342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c8242b640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a228e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.754+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a227d370 0x55f5a22899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c81c2a640 1 -- 172.21.15.16:0/64372342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 msgr2=0x55f5a228e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a228e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c81c2a640 1 -- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a227d370 msgr2=0x55f5a22899d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a227d370 0x55f5a22899d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c81c2a640 1 -- 172.21.15.16:0/64372342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a228a1b0 con 0x55f5a2273b40 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c82c2c640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a227d370 0x55f5a22899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c81c2a640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a2273b40 0x55f5a2289490 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0bc650 tx=0x7f4c6c0ee040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73afae82d71739c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c65ffb640 1 -- 172.21.15.16:0/64372342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c049040 con 0x55f5a2273b40 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c65ffb640 1 -- 172.21.15.16:0/64372342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f4c6c0ba9c0 con 0x55f5a2273b40 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7401c440 con 0x55f5a2273b40 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c65ffb640 1 -- 172.21.15.16:0/64372342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c6c0f05a0 con 0x55f5a2273b40 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c86a68740 10 client.15843 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5a2300180 con 0x55f5a2273b40 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.755+0000 7f4c65ffb640 1 -- 172.21.15.16:0/64372342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c7401c440 con 0x55f5a2273b40 2024-09-15T22:40:09.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c65ffb640 1 -- 172.21.15.16:0/64372342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4c6c09d030 con 0x55f5a2273b40 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c86a68740 10 client.15843 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c86a68740 10 client.15843 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c86a68740 20 client.15843 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c86a68740 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2300920 0x55f5a2302d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c86a68740 4 client.15843 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5a21dfd70 con 0x55f5a2300920 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c8242b640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2300920 0x55f5a2302d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c8242b640 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2300920 0x55f5a2302d40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=855a8c2d61e9e056 server_cookie=182407e2ea73cb08 in_seq=0 out_seq=0 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.756+0000 7f4c65ffb640 10 client.15843 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c65ffb640 1 -- 172.21.15.16:0/64372342 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5a21dfd70 con 0x55f5a2300920 2024-09-15T22:40:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c65ffb640 10 client.15843 handle_command_reply: tid=0 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 client.15843 shutdown 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a2273b40 msgr2=0x55f5a2289490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a2273b40 0x55f5a2289490 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0bc650 tx=0x7f4c6c0ee040 comp rx=0 tx=0).stop 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 shutdown_connections 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f5a2300920 0x55f5a2302d40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f5a227d370 0x55f5a22899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c65ffb640 0 client.15843 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f5a2273b40 0x55f5a2289490 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 --2- 172.21.15.16:0/64372342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f5a22725d0 0x55f5a228e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 >> 172.21.15.16:0/64372342 conn(0x55f5a223b310 msgr2=0x55f5a227b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 shutdown_connections 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 1 -- 172.21.15.16:0/64372342 wait complete. 2024-09-15T22:40:09.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.757+0000 7f4c86a68740 20 client.15843 trim_cache size 0 max 16384 2024-09-15T22:40:09.765 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:40:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4c680a3430 msgr2=0x7f4c680a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4c680a3430 0x7f4c680a58f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009c30 tx=0x7f4c78005f90 comp rx=0 tx=0).stop 2024-09-15T22:40:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 msgr2=0x7f4c7c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c100f30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005940 tx=0x7f4c6c003c10 comp rx=0 tx=0).stop 2024-09-15T22:40:09.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 shutdown_connections 2024-09-15T22:40:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c7c108290 0x7f4c7c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4c680a3430 0x7f4c680a58f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4c7c103740 0x7f4c7c100f30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 --2- 172.21.15.16:0/3935719793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c7c102d90 0x7f4c7c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 >> 172.21.15.16:0/3935719793 conn(0x7f4c7c0fc7d0 msgr2=0x7f4c7c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.758+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 shutdown_connections 2024-09-15T22:40:09.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:09.759+0000 7f4c8468f640 1 -- 172.21.15.16:0/3935719793 wait complete. 2024-09-15T22:40:10.374 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-15T22:40:10.374 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-15T22:40:10.375 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-15T22:40:10.376 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-15T22:40:10.377 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-15T22:40:10.378 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-09-15T22:40:10.379 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-15T22:40:10.380 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-09-15T22:40:10.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:40:10.736 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-15T22:40:10.737 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-15T22:40:10.738 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-09-15T22:40:10.739 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-09-15T22:40:10.740 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-15T22:40:10.741 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-09-15T22:40:10.742 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-15T22:40:10.803 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:10.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:10 smithi016 ceph-mon[31492]: pgmap v55: 129 pgs: 129 active+clean; 102 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-15T22:40:10.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:10 smithi059 ceph-mon[34872]: pgmap v55: 129 pgs: 129 active+clean; 102 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-15T22:40:11.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:10 smithi130 ceph-mon[35742]: pgmap v55: 129 pgs: 129 active+clean; 102 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 56 MiB/s rd, 1.2 MiB/s wr, 441 op/s 2024-09-15T22:40:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 -- 172.21.15.16:0/3759405448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c102570 msgr2=0x7fce5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 --2- 172.21.15.16:0/3759405448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c102570 0x7fce5c1029b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce50009900 tx=0x7fce5002a480 comp rx=0 tx=0).stop 2024-09-15T22:40:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 -- 172.21.15.16:0/3759405448 shutdown_connections 2024-09-15T22:40:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 --2- 172.21.15.16:0/3759405448 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce5c102ef0 0x7fce5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 --2- 172.21.15.16:0/3759405448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c102570 0x7fce5c1029b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 --2- 172.21.15.16:0/3759405448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce5c100e20 0x7fce5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 -- 172.21.15.16:0/3759405448 >> 172.21.15.16:0/3759405448 conn(0x7fce5c0fc7d0 msgr2=0x7fce5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 -- 172.21.15.16:0/3759405448 shutdown_connections 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.311+0000 7fce63d9c640 1 -- 172.21.15.16:0/3759405448 wait complete. 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 Processor -- start 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 -- start start 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c100e20 0x7fce5c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce5c102570 0x7fce5c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce5c102ef0 0x7fce5c1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c110f80 con 0x7fce5c100e20 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c110e00 con 0x7fce5c102ef0 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce63d9c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c111100 con 0x7fce5c102570 2024-09-15T22:40:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c100e20 0x7fce5c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c100e20 0x7fce5c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59534/0 (socket says 172.21.15.16:59534) 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 -- 172.21.15.16:0/1616970696 learned_addr learned my addr 172.21.15.16:0/1616970696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce62339640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce5c102ef0 0x7fce5c1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61337640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce5c102570 0x7fce5c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 -- 172.21.15.16:0/1616970696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce5c102570 msgr2=0x7fce5c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce5c102570 0x7fce5c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 -- 172.21.15.16:0/1616970696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce5c102ef0 msgr2=0x7fce5c1a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce5c102ef0 0x7fce5c1a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61b38640 1 -- 172.21.15.16:0/1616970696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c1a4910 con 0x7fce5c100e20 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.312+0000 7fce61337640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce5c102570 0x7fce5c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.313+0000 7fce61b38640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c100e20 0x7fce5c199350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d950 tx=0x7fce5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.313+0000 7fce5affd640 1 -- 172.21.15.16:0/1616970696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54017070 con 0x7fce5c100e20 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.313+0000 7fce5affd640 1 -- 172.21.15.16:0/1616970696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce54009c30 con 0x7fce5c100e20 2024-09-15T22:40:11.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.313+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c1a4ae0 con 0x7fce5c100e20 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.313+0000 7fce5affd640 1 -- 172.21.15.16:0/1616970696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5400f6b0 con 0x7fce5c100e20 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.313+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c1a4ff0 con 0x7fce5c100e20 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.315+0000 7fce5affd640 1 -- 172.21.15.16:0/1616970696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fce540109d0 con 0x7fce5c100e20 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.315+0000 7fce66175740 1 Processor -- start 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce66175740 1 -- start start 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce66175740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dcb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce66175740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dcb5d0 0x55ed16dcba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce66175740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dd6370 0x55ed16dd8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce5affd640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fce440a33b0 0x7fce440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce66175740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed16d312f0 con 0x55ed16dd6370 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce66175740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed16cd8cf0 con 0x55ed16dcb5d0 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce66175740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ed16d2d2b0 con 0x55ed16dccb40 2024-09-15T22:40:11.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dd6370 0x55ed16dd8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce62339640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fce440a33b0 0x7fce440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dd6370 0x55ed16dd8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59546/0 (socket says 172.21.15.16:59546) 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 -- 172.21.15.16:0/765409699 learned_addr learned my addr 172.21.15.16:0/765409699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce5affd640 1 -- 172.21.15.16:0/1616970696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fce540cfd10 con 0x7fce5c100e20 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 -- 172.21.15.16:0/765409699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 msgr2=0x55ed16dcb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 --2- 172.21.15.16:0/765409699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dcb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 -- 172.21.15.16:0/765409699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dcb5d0 msgr2=0x55ed16dcba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 --2- 172.21.15.16:0/765409699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dcb5d0 0x55ed16dcba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.316+0000 7fce61337640 1 -- 172.21.15.16:0/765409699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed16cbd3a0 con 0x55ed16dd6370 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.317+0000 7fce62339640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fce440a33b0 0x7fce440a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce4c005fd0 tx=0x7fce4c009b30 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.317+0000 7fce61337640 1 --2- 172.21.15.16:0/765409699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dd6370 0x55ed16dd8760 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce50009900 tx=0x7fce500040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b88052b8ff92c27 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.317+0000 7fce43fff640 1 -- 172.21.15.16:0/765409699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50035030 con 0x55ed16dd6370 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.317+0000 7fce43fff640 1 -- 172.21.15.16:0/765409699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50005520 con 0x55ed16dd6370 2024-09-15T22:40:11.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.317+0000 7fce43fff640 1 -- 172.21.15.16:0/765409699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce500035f0 con 0x55ed16dd6370 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.317+0000 7fce66175740 1 -- 172.21.15.16:0/765409699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dd6370 msgr2=0x55ed16dd8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 --2- 172.21.15.16:0/765409699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dd6370 0x55ed16dd8760 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fce50009900 tx=0x7fce500040f0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 -- 172.21.15.16:0/765409699 shutdown_connections 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 --2- 172.21.15.16:0/765409699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dd6370 0x55ed16dd8760 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 --2- 172.21.15.16:0/765409699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dcb5d0 0x55ed16dcba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 --2- 172.21.15.16:0/765409699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dcb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 -- 172.21.15.16:0/765409699 >> 172.21.15.16:0/765409699 conn(0x55ed16d94310 msgr2=0x55ed16dc7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 -- 172.21.15.16:0/765409699 shutdown_connections 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 -- 172.21.15.16:0/765409699 wait complete. 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 -- start start 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.318+0000 7fce66175740 1 Processor -- start 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce66175740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dd5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce66175740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dd3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce66175740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dd3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce66175740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ed16dd9010 con 0x55ed16dcb5d0 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce66175740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed16cec060 con 0x55ed16dccb40 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce66175740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ed16dd9190 con 0x55ed16dd6370 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce66175740 10 client.0 _ll_register_callbacks cb 0x55ed16a3f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dd5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dd5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59552/0 (socket says 172.21.15.16:59552) 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 -- 172.21.15.16:0/1530914811 learned_addr learned my addr 172.21.15.16:0/1530914811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61337640 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dd3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce62339640 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dd3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 -- 172.21.15.16:0/1530914811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 msgr2=0x55ed16dd3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dd3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 -- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 msgr2=0x55ed16dd3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dd3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 -- 172.21.15.16:0/1530914811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed16aa23c0 con 0x55ed16dcb5d0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce61b38640 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dd5d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce54010fd0 tx=0x7fce5409b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336c371f3d819e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce417fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.319+0000 7fce417fa640 1 -- 172.21.15.16:0/1530914811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54020040 con 0x55ed16dcb5d0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce417fa640 1 -- 172.21.15.16:0/1530914811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fce5409b0e0 con 0x55ed16dcb5d0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce50005520 con 0x55ed16dcb5d0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce417fa640 1 -- 172.21.15.16:0/1530914811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce5409c390 con 0x55ed16dcb5d0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 10 client.15855 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed16de4b10 con 0x55ed16dcb5d0 2024-09-15T22:40:11.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce417fa640 1 -- 172.21.15.16:0/1530914811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50005520 con 0x55ed16dcb5d0 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce417fa640 1 -- 172.21.15.16:0/1530914811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fce540b4070 con 0x55ed16dcb5d0 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 10 client.15855 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 20 client.15855 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16de5fd0 0x55ed16de83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 4 client.15855 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed16d3e860 con 0x55ed16de5fd0 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.320+0000 7fce61337640 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16de5fd0 0x55ed16de83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.321+0000 7fce61337640 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16de5fd0 0x55ed16de83c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd6280fe268854d server_cookie=95dd78042671226c in_seq=0 out_seq=0 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.321+0000 7fce417fa640 10 client.15855 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.321+0000 7fce417fa640 1 -- 172.21.15.16:0/1530914811 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed16d3e860 con 0x55ed16de5fd0 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.321+0000 7fce417fa640 10 client.15855 handle_command_reply: tid=0 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.321+0000 7fce66175740 1 client.15855 shutdown 2024-09-15T22:40:11.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 msgr2=0x55ed16dd5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dd5d90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fce54010fd0 tx=0x7fce5409b890 comp rx=0 tx=0).stop 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 shutdown_connections 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16de5fd0 0x55ed16de83c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dd3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce417fa640 0 client.15855 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dd3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 --2- 172.21.15.16:0/1530914811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dd5d90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 >> 172.21.15.16:0/1530914811 conn(0x55ed16d94310 msgr2=0x55ed16dcd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 shutdown_connections 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 1 -- 172.21.15.16:0/1530914811 wait complete. 2024-09-15T22:40:11.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.322+0000 7fce66175740 20 client.15855 trim_cache size 0 max 16384 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.335+0000 7fce66175740 1 Processor -- start 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.335+0000 7fce66175740 1 -- start start 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce66175740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dc6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce66175740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dc7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce66175740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce66175740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1b17a0 con 0x55ed16dcb5d0 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce66175740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1af070 con 0x55ed16dccb40 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce66175740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1acec0 con 0x55ed16dd6370 2024-09-15T22:40:11.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dc6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dc6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59562/0 (socket says 172.21.15.16:59562) 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 -- 172.21.15.16:0/993748442 learned_addr learned my addr 172.21.15.16:0/993748442 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61337640 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dc7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce62339640 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 -- 172.21.15.16:0/993748442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 msgr2=0x55ed16dce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 -- 172.21.15.16:0/993748442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 msgr2=0x55ed16dc7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dc7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 -- 172.21.15.16:0/993748442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed16dd1ff0 con 0x55ed16dcb5d0 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce62339640 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.336+0000 7fce61b38640 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dc6bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fce540cd1b0 tx=0x7fce54000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84619b66b6beb276 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce437fe640 1 -- 172.21.15.16:0/993748442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54020070 con 0x55ed16dcb5d0 2024-09-15T22:40:11.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce437fe640 1 -- 172.21.15.16:0/993748442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce540a4900 con 0x55ed16dcb5d0 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce437fe640 1 -- 172.21.15.16:0/993748442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54086310 con 0x55ed16dcb5d0 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 -- 172.21.15.16:0/993748442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 msgr2=0x55ed16dc6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dc6bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fce540cd1b0 tx=0x7fce54000bc0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 -- 172.21.15.16:0/993748442 shutdown_connections 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dd6370 0x55ed16dce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dccb40 0x55ed16dc7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 --2- 172.21.15.16:0/993748442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16dc6bf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 -- 172.21.15.16:0/993748442 >> 172.21.15.16:0/993748442 conn(0x55ed16d94310 msgr2=0x55ed16dc64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 -- 172.21.15.16:0/993748442 shutdown_connections 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.337+0000 7fce66175740 1 -- 172.21.15.16:0/993748442 wait complete. 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 -- start start 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 Processor -- start 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16ddebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dd9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dd6370 0x55ed16dda470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1ad530 con 0x55ed16dcb5d0 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1ae9a0 con 0x55ed16dd6370 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1adc00 con 0x55ed16dccb40 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce66175740 10 client.0 _ll_register_callbacks cb 0x7fce54098020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.338+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16ddebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16ddebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59574/0 (socket says 172.21.15.16:59574) 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 -- 172.21.15.16:0/2500938753 learned_addr learned my addr 172.21.15.16:0/2500938753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce62339640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dd6370 0x55ed16dda470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61337640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dd9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 -- 172.21.15.16:0/2500938753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 msgr2=0x55ed16dd9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dd9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 -- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dd6370 msgr2=0x55ed16dda470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dd6370 0x55ed16dda470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 -- 172.21.15.16:0/2500938753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed16ddac50 con 0x55ed16dcb5d0 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce62339640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dd6370 0x55ed16dda470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61337640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dd9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce61b38640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16ddebb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce54099f40 tx=0x7fce5409b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4121fa0b37001f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce417fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce417fa640 1 -- 172.21.15.16:0/2500938753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54020070 con 0x55ed16dcb5d0 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce417fa640 1 -- 172.21.15.16:0/2500938753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fce540860b0 con 0x55ed16dcb5d0 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce540a4900 con 0x55ed16dcb5d0 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce417fa640 1 -- 172.21.15.16:0/2500938753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fce540a0040 con 0x55ed16dcb5d0 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce66175740 10 client.15861 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:11.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.339+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed16e50b90 con 0x55ed16dcb5d0 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce417fa640 1 -- 172.21.15.16:0/2500938753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce540a4900 con 0x55ed16dcb5d0 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce417fa640 1 -- 172.21.15.16:0/2500938753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fce540b4070 con 0x55ed16dcb5d0 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce66175740 10 client.15861 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce66175740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce66175740 20 client.15861 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce66175740 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16e51010 0x55ed16e53400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce66175740 4 client.15861 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed16d38d70 con 0x55ed16e51010 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce61337640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16e51010 0x55ed16e53400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce417fa640 10 client.15861 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce61337640 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16e51010 0x55ed16e53400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36edd45a946a6d4d server_cookie=a3f96f5e0a1a008f in_seq=0 out_seq=0 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.340+0000 7fce417fa640 1 -- 172.21.15.16:0/2500938753 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed16d38d70 con 0x55ed16e51010 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce417fa640 10 client.15861 handle_command_reply: tid=0 2024-09-15T22:40:11.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 client.15861 shutdown 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 msgr2=0x55ed16ddebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16ddebb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce54099f40 tx=0x7fce5409b890 comp rx=0 tx=0).stop 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 shutdown_connections 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ed16e51010 0x55ed16e53400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed16dd6370 0x55ed16dda470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ed16dccb40 0x55ed16dd9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce417fa640 0 client.15861 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 --2- 172.21.15.16:0/2500938753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ed16dcb5d0 0x55ed16ddebb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 >> 172.21.15.16:0/2500938753 conn(0x55ed16d94310 msgr2=0x55ed16dc6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 shutdown_connections 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 1 -- 172.21.15.16:0/2500938753 wait complete. 2024-09-15T22:40:11.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.341+0000 7fce66175740 20 client.15861 trim_cache size 0 max 16384 2024-09-15T22:40:11.352 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:40:11.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fce440a33b0 msgr2=0x7fce440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fce440a33b0 0x7fce440a5870 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce4c005fd0 tx=0x7fce4c009b30 comp rx=0 tx=0).stop 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c100e20 msgr2=0x7fce5c199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c100e20 0x7fce5c199350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d950 tx=0x7fce5400de10 comp rx=0 tx=0).stop 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 shutdown_connections 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce5c102ef0 0x7fce5c1a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fce440a33b0 0x7fce440a5870 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fce5c102570 0x7fce5c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 --2- 172.21.15.16:0/1616970696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c100e20 0x7fce5c199350 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 >> 172.21.15.16:0/1616970696 conn(0x7fce5c0fc7d0 msgr2=0x7fce5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 shutdown_connections 2024-09-15T22:40:11.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:11.342+0000 7fce63d9c640 1 -- 172.21.15.16:0/1616970696 wait complete. 2024-09-15T22:40:11.773 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-15T22:40:11.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-15T22:40:11.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-09-15T22:40:11.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-09-15T22:40:11.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-09-15T22:40:11.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-15T22:40:11.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-09-15T22:40:12.899 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-15T22:40:12.899 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-09-15T22:40:12.899 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-15T22:40:12.899 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-09-15T22:40:12.900 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-15T22:40:12.901 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-15T22:40:12.902 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-09-15T22:40:12.903 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-15T22:40:12.904 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-09-15T22:40:12.905 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-15T22:40:12.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:12 smithi059 ceph-mon[34872]: pgmap v56: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s rd, 1.7 MiB/s wr, 576 op/s 2024-09-15T22:40:13.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:12 smithi130 ceph-mon[35742]: pgmap v56: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s rd, 1.7 MiB/s wr, 576 op/s 2024-09-15T22:40:13.033 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:40:13.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:40:13.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:12 smithi016 ceph-mon[31492]: pgmap v56: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 97 MiB/s rd, 1.7 MiB/s wr, 576 op/s 2024-09-15T22:40:13.304 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-09-15T22:40:13.307 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-15T22:40:13.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-09-15T22:40:13.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-15T22:40:13.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-09-15T22:40:13.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-15T22:40:13.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-09-15T22:40:13.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-15T22:40:13.676 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-09-15T22:40:13.676 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-15T22:40:13.676 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-09-15T22:40:13.676 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-15T22:40:13.676 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-15T22:40:13.677 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-15T22:40:13.678 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-15T22:40:13.679 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-09-15T22:40:13.680 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-09-15T22:40:13.681 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-15T22:40:13.682 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-09-15T22:40:13.682 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-15T22:40:13.682 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-09-15T22:40:13.682 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-15T22:40:13.682 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-09-15T22:40:13.682 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-15T22:40:13.682 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-09-15T22:40:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.167+0000 7fe6b8249640 1 -- 172.21.15.16:0/772661000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0108290 msgr2=0x7fe6b010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.167+0000 7fe6b8249640 1 --2- 172.21.15.16:0/772661000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0108290 0x7fe6b010ec30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8009cd0 tx=0x7fe6a802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.167+0000 7fe6b8249640 1 -- 172.21.15.16:0/772661000 shutdown_connections 2024-09-15T22:40:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.167+0000 7fe6b8249640 1 --2- 172.21.15.16:0/772661000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0108290 0x7fe6b010ec30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.167+0000 7fe6b8249640 1 --2- 172.21.15.16:0/772661000 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6b0103740 0x7fe6b0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.167+0000 7fe6b8249640 1 --2- 172.21.15.16:0/772661000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6b0102d90 0x7fe6b0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.167+0000 7fe6b8249640 1 -- 172.21.15.16:0/772661000 >> 172.21.15.16:0/772661000 conn(0x7fe6b00fc7d0 msgr2=0x7fe6b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.168+0000 7fe6b8249640 1 -- 172.21.15.16:0/772661000 shutdown_connections 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.168+0000 7fe6b8249640 1 -- 172.21.15.16:0/772661000 wait complete. 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.168+0000 7fe6b8249640 1 Processor -- start 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.168+0000 7fe6b8249640 1 -- start start 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b8249640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0102d90 0x7fe6b01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b8249640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6b0103740 0x7fe6b0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b8249640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6b0108290 0x7fe6b0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b8249640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0112120 con 0x7fe6b0102d90 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b8249640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b0111fa0 con 0x7fe6b0108290 2024-09-15T22:40:14.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b8249640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01122a0 con 0x7fe6b0103740 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0102d90 0x7fe6b01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0102d90 0x7fe6b01009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56872/0 (socket says 172.21.15.16:56872) 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/2259343225 learned_addr learned my addr 172.21.15.16:0/2259343225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b67e6640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6b0108290 0x7fe6b0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6b0103740 0x7fe6b0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/2259343225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6b0103740 msgr2=0x7fe6b0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6b0103740 0x7fe6b0100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/2259343225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6b0108290 msgr2=0x7fe6b0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6b0108290 0x7fe6b0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/2259343225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b01a6a80 con 0x7fe6b0102d90 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.169+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0102d90 0x7fe6b01009f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00e9f0 tx=0x7fe6ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.170+0000 7fe6a6ffd640 1 -- 172.21.15.16:0/2259343225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00cd80 con 0x7fe6b0102d90 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.170+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b01a6d70 con 0x7fe6b0102d90 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.170+0000 7fe6a6ffd640 1 -- 172.21.15.16:0/2259343225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac010910 con 0x7fe6b0102d90 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.170+0000 7fe6a6ffd640 1 -- 172.21.15.16:0/2259343225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac018990 con 0x7fe6b0102d90 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.170+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b01a7250 con 0x7fe6b0102d90 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.173+0000 7fe6a6ffd640 1 -- 172.21.15.16:0/2259343225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe6ac010ac0 con 0x7fe6b0102d90 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.173+0000 7fe6ba622740 1 Processor -- start 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.173+0000 7fe6ba622740 1 -- start start 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0deef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6ba622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccc0d54280 con 0x55ccc0def480 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6b5fe5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6a6ffd640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe69c0a33b0 0x7fe69c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6b5fe5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56888/0 (socket says 172.21.15.16:56888) 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/4100090291 learned_addr learned my addr 172.21.15.16:0/4100090291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6ba622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ccc0cfbb10 con 0x55ccc0deeb60 2024-09-15T22:40:14.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6ba622740 1 -- 172.21.15.16:0/4100090291 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ccc0d50240 con 0x55ccc0df9d40 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.174+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6a6ffd640 1 -- 172.21.15.16:0/2259343225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe6ac0d0420 con 0x7fe6b0102d90 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/4100090291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 msgr2=0x55ccc0dfc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/4100090291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 msgr2=0x55ccc0deef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0deef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6b5fe5640 1 -- 172.21.15.16:0/4100090291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc0d081f0 con 0x55ccc0def480 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df9800 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0cd7b0 tx=0x7fe6ac09bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30cd6edf3b501b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe6b67e6640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe69c0a33b0 0x7fe69c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe69bfff640 1 -- 172.21.15.16:0/4100090291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0a5010 con 0x55ccc0def480 2024-09-15T22:40:14.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe69bfff640 1 -- 172.21.15.16:0/4100090291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac0d0160 con 0x55ccc0def480 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.175+0000 7fe69bfff640 1 -- 172.21.15.16:0/4100090291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac09ba50 con 0x55ccc0def480 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 -- 172.21.15.16:0/4100090291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 msgr2=0x55ccc0df9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6b67e6640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe69c0a33b0 0x7fe69c0a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8005e30 tx=0x7fe6a8004a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df9800 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0cd7b0 tx=0x7fe6ac09bdc0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 -- 172.21.15.16:0/4100090291 shutdown_connections 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df9800 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 --2- 172.21.15.16:0/4100090291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0deef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 -- 172.21.15.16:0/4100090291 >> 172.21.15.16:0/4100090291 conn(0x55ccc0deb0f0 msgr2=0x55ccc0deb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 -- 172.21.15.16:0/4100090291 shutdown_connections 2024-09-15T22:40:14.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.176+0000 7fe6ba622740 1 -- 172.21.15.16:0/4100090291 wait complete. 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 -- start start 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 Processor -- start 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0df76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ccc0dfc910 con 0x55ccc0def480 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ccc0d0ef10 con 0x55ccc0deeb60 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ccc0dfca90 con 0x55ccc0df9d40 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6ba622740 10 client.0 _ll_register_callbacks cb 0x55ccc0cd7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6b57e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6b57e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56892/0 (socket says 172.21.15.16:56892) 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6b57e4640 1 -- 172.21.15.16:0/151494863 learned_addr learned my addr 172.21.15.16:0/151494863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6b67e6640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0df76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.177+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6b57e4640 1 -- 172.21.15.16:0/151494863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 msgr2=0x55ccc0df76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0df76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6b57e4640 1 -- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 msgr2=0x55ccc0df6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6b57e4640 1 -- 172.21.15.16:0/151494863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc0ac56a0 con 0x55ccc0def480 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df7190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000bf90 tx=0x7fe6a000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80402aa0e85ec2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6997fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6997fa640 1 -- 172.21.15.16:0/151494863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0018020 con 0x55ccc0def480 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6997fa640 1 -- 172.21.15.16:0/151494863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe6a0014870 con 0x55ccc0def480 2024-09-15T22:40:14.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6ac0d0160 con 0x55ccc0def480 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6997fa640 1 -- 172.21.15.16:0/151494863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6a0023050 con 0x55ccc0def480 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6ba622740 10 client.15873 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.178+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccc0e085a0 con 0x55ccc0def480 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6997fa640 1 -- 172.21.15.16:0/151494863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac0d0160 con 0x55ccc0def480 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6997fa640 1 -- 172.21.15.16:0/151494863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe6a0013bf0 con 0x55ccc0def480 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6ba622740 10 client.15873 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6ba622740 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6ba622740 20 client.15873 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6ba622740 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e09a60 0x55ccc0e0beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6ba622740 4 client.15873 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc0d617f0 con 0x55ccc0e09a60 2024-09-15T22:40:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.179+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e09a60 0x55ccc0e0beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.180+0000 7fe6997fa640 10 client.15873 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.180+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e09a60 0x55ccc0e0beb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca1efb3e47b34b6 server_cookie=f039a19f6e611f27 in_seq=0 out_seq=0 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.180+0000 7fe6997fa640 1 -- 172.21.15.16:0/151494863 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccc0d617f0 con 0x55ccc0e09a60 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.180+0000 7fe6997fa640 10 client.15873 handle_command_reply: tid=0 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.180+0000 7fe6ba622740 1 client.15873 shutdown 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 msgr2=0x55ccc0df7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df7190 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000bf90 tx=0x7fe6a000f670 comp rx=0 tx=0).stop 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 shutdown_connections 2024-09-15T22:40:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e09a60 0x55ccc0e0beb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0df76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0df7190 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6997fa640 0 client.15873 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 --2- 172.21.15.16:0/151494863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 >> 172.21.15.16:0/151494863 conn(0x55ccc0deb0f0 msgr2=0x55ccc0df0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 shutdown_connections 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 1 -- 172.21.15.16:0/151494863 wait complete. 2024-09-15T22:40:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.181+0000 7fe6ba622740 20 client.15873 trim_cache size 0 max 16384 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 Processor -- start 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 -- start start 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0e117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0e11d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01b3af0 con 0x55ccc0def480 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01b13c0 con 0x55ccc0deeb60 2024-09-15T22:40:14.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.199+0000 7fe6ba622740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01af210 con 0x55ccc0df9d40 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0e117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b5fe5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0e117c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56898/0 (socket says 172.21.15.16:56898) 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 -- 172.21.15.16:0/980001757 learned_addr learned my addr 172.21.15.16:0/980001757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b67e6640 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0e11d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 -- 172.21.15.16:0/980001757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 msgr2=0x55ccc0e11d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0e11d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 -- 172.21.15.16:0/980001757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 msgr2=0x55ccc0df1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 -- 172.21.15.16:0/980001757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc0deab90 con 0x55ccc0def480 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b67e6640 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0e11d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0e117c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eeac2d2f1964c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.200+0000 7fe69b7fe640 1 -- 172.21.15.16:0/980001757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0018040 con 0x55ccc0def480 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe69b7fe640 1 -- 172.21.15.16:0/980001757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a002c070 con 0x55ccc0def480 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe69b7fe640 1 -- 172.21.15.16:0/980001757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0014a30 con 0x55ccc0def480 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe6ba622740 1 -- 172.21.15.16:0/980001757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 msgr2=0x55ccc0e117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe6ba622740 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0e117c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0000c00 tx=0x7fe6a000bee0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe6ba622740 1 -- 172.21.15.16:0/980001757 shutdown_connections 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe6ba622740 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0e11d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe6ba622740 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0e117c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe6ba622740 1 --2- 172.21.15.16:0/980001757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0df1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.201+0000 7fe6ba622740 1 -- 172.21.15.16:0/980001757 >> 172.21.15.16:0/980001757 conn(0x55ccc0deb0f0 msgr2=0x55ccc0df8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.202+0000 7fe6ba622740 1 -- 172.21.15.16:0/980001757 shutdown_connections 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.202+0000 7fe6ba622740 1 -- 172.21.15.16:0/980001757 wait complete. 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.202+0000 7fe6ba622740 1 -- start start 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.202+0000 7fe6ba622740 1 Processor -- start 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.202+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0dfdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.202+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0dfe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6ba622740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6ba622740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01af880 con 0x55ccc0def480 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6ba622740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01b0cf0 con 0x55ccc0deeb60 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6ba622740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b01aff50 con 0x55ccc0df9d40 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6ba622740 10 client.0 _ll_register_callbacks cb 0x7fe6a000b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0dfe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b5fe5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0dfdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0dfe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56912/0 (socket says 172.21.15.16:56912) 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b67e6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b5fe5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0dfdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:32814/0 (socket says 172.21.15.16:32814) 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 -- 172.21.15.16:0/1131325467 learned_addr learned my addr 172.21.15.16:0/1131325467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 -- 172.21.15.16:0/1131325467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 msgr2=0x55ccc0dfe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 -- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 msgr2=0x55ccc0dfdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0dfdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 -- 172.21.15.16:0/1131325467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc0e738f0 con 0x55ccc0def480 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0dfdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6b57e4640 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0dfe090 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe6a001c740 tx=0x7fe6a001c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c15521067333d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6997fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6997fa640 1 -- 172.21.15.16:0/1131325467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0035060 con 0x55ccc0def480 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6997fa640 1 -- 172.21.15.16:0/1131325467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe6a001cda0 con 0x55ccc0def480 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.203+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a002c070 con 0x55ccc0def480 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6997fa640 1 -- 172.21.15.16:0/1131325467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6a0018030 con 0x55ccc0def480 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 10 client.15879 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccc0e74bb0 con 0x55ccc0def480 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6997fa640 1 -- 172.21.15.16:0/1131325467 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a002c070 con 0x55ccc0def480 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6997fa640 1 -- 172.21.15.16:0/1131325467 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe6a0006020 con 0x55ccc0def480 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 10 client.15879 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 20 client.15879 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e75220 0x55ccc0e77640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 4 client.15879 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ccc0d5bd00 con 0x55ccc0e75220 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.204+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e75220 0x55ccc0e77640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.205+0000 7fe6b5fe5640 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e75220 0x55ccc0e77640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a2dfb807911c16 server_cookie=3bebd0e27e0f09bc in_seq=0 out_seq=0 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.205+0000 7fe6997fa640 10 client.15879 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.206+0000 7fe6997fa640 1 -- 172.21.15.16:0/1131325467 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ccc0d5bd00 con 0x55ccc0e75220 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.206+0000 7fe6997fa640 10 client.15879 handle_command_reply: tid=0 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.206+0000 7fe6ba622740 1 client.15879 shutdown 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.206+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 msgr2=0x55ccc0dfe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.206+0000 7fe6ba622740 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0dfe090 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe6a001c740 tx=0x7fe6a001c770 comp rx=0 tx=0).stop 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 shutdown_connections 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ccc0e75220 0x55ccc0e77640 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ccc0df9d40 0x55ccc0dfe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ccc0def480 0x55ccc0dfe090 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6997fa640 0 client.15879 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 --2- 172.21.15.16:0/1131325467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ccc0deeb60 0x55ccc0dfdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 >> 172.21.15.16:0/1131325467 conn(0x55ccc0deb0f0 msgr2=0x55ccc0df6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 shutdown_connections 2024-09-15T22:40:14.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 1 -- 172.21.15.16:0/1131325467 wait complete. 2024-09-15T22:40:14.450 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:14.450 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "b145a081-32e3-4953-9900-39c08b53757b", 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6ba622740 20 client.15879 trim_cache size 0 max 16384 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe69c0a33b0 msgr2=0x7fe69c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe69c0a33b0 0x7fe69c0a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8005e30 tx=0x7fe6a8004a40 comp rx=0 tx=0).stop 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0102d90 msgr2=0x7fe6b01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0102d90 0x7fe6b01009f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00e9f0 tx=0x7fe6ac00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 shutdown_connections 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6b0108290 0x7fe6b0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe69c0a33b0 0x7fe69c0a5870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6b0103740 0x7fe6b0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 --2- 172.21.15.16:0/2259343225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0102d90 0x7fe6b01009f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.207+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 >> 172.21.15.16:0/2259343225 conn(0x7fe6b00fc7d0 msgr2=0x7fe6b0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.208+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 shutdown_connections 2024-09-15T22:40:14.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:14.208+0000 7fe6b8249640 1 -- 172.21.15.16:0/2259343225 wait complete. 2024-09-15T22:40:14.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:14 smithi016 ceph-mon[31492]: pgmap v57: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 960 KiB/s wr, 276 op/s 2024-09-15T22:40:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:14 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:40:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:14 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:14 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:40:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:14 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:14.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-15T22:40:14.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-09-15T22:40:14.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-15T22:40:14.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-09-15T22:40:14.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-15T22:40:14.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-15T22:40:14.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-09-15T22:40:14.851 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-09-15T22:40:14.852 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-15T22:40:14.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:14 smithi059 ceph-mon[34872]: pgmap v57: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 960 KiB/s wr, 276 op/s 2024-09-15T22:40:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:14 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:40:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:14 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:14 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:40:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:14 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:15.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:14 smithi130 ceph-mon[35742]: pgmap v57: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 960 KiB/s wr, 276 op/s 2024-09-15T22:40:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:14 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:40:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:14 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:14 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:40:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:14 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:15.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:40:15.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-09-15T22:40:15.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-15T22:40:15.774 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-15T22:40:15.775 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-15T22:40:15.776 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-15T22:40:15.777 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-09-15T22:40:15.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-15T22:40:15.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-09-15T22:40:15.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-15T22:40:15.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-09-15T22:40:15.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-15T22:40:15.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-09-15T22:40:15.778 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-15T22:40:15.779 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-15T22:40:15.780 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-09-15T22:40:15.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-15T22:40:15.781 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-09-15T22:40:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:16.162 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-15T22:40:16.162 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-09-15T22:40:16.162 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-15T22:40:16.162 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-09-15T22:40:16.163 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-15T22:40:16.163 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-09-15T22:40:16.163 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-15T22:40:16.163 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-09-15T22:40:16.163 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-15T22:40:16.163 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-09-15T22:40:16.164 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-15T22:40:16.164 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-09-15T22:40:16.164 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-15T22:40:16.164 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-09-15T22:40:16.164 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-15T22:40:16.165 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-09-15T22:40:16.166 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-15T22:40:16.167 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-15T22:40:16.168 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-09-15T22:40:16.169 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-15T22:40:16.172 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-09-15T22:40:16.172 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-15T22:40:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2338127862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41082a0 msgr2=0x7ff3f410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2338127862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41082a0 0x7ff3f410ec40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009cd0 tx=0x7ff3ec02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2338127862 shutdown_connections 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2338127862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f41082a0 0x7ff3f410ec40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2338127862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3f41036e0 0x7ff3f4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2338127862 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f4102d30 0x7ff3f4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2338127862 >> 172.21.15.16:0/2338127862 conn(0x7ff3f40fc7d0 msgr2=0x7ff3f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2338127862 shutdown_connections 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2338127862 wait complete. 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 Processor -- start 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.359+0000 7ff3fb9b0640 1 -- start start 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f4102d30 0x7ff3f4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f41036e0 0x7ff3f4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3f41082a0 0x7ff3f4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41120a0 con 0x7ff3f4102d30 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4111f20 con 0x7ff3f41082a0 2024-09-15T22:40:16.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f4112220 con 0x7ff3f41036e0 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f4102d30 0x7ff3f4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f4102d30 0x7ff3f4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56938/0 (socket says 172.21.15.16:56938) 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 -- 172.21.15.16:0/2122752252 learned_addr learned my addr 172.21.15.16:0/2122752252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3f41082a0 0x7ff3f4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f41036e0 0x7ff3f4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 -- 172.21.15.16:0/2122752252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f41036e0 msgr2=0x7ff3f4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f41036e0 0x7ff3f4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 -- 172.21.15.16:0/2122752252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3f41082a0 msgr2=0x7ff3f4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3f41082a0 0x7ff3f4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 -- 172.21.15.16:0/2122752252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f41a6a30 con 0x7ff3f4102d30 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3f974c640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f4102d30 0x7ff3f4100990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e9f0 tx=0x7ff3f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3ea7fc640 1 -- 172.21.15.16:0/2122752252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f000cd80 con 0x7ff3f4102d30 2024-09-15T22:40:16.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f41a6d20 con 0x7ff3f4102d30 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3ea7fc640 1 -- 172.21.15.16:0/2122752252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f0005500 con 0x7ff3f4102d30 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3ea7fc640 1 -- 172.21.15.16:0/2122752252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0010690 con 0x7ff3f4102d30 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.360+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f41a7200 con 0x7ff3f4102d30 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.363+0000 7ff3ea7fc640 1 -- 172.21.15.16:0/2122752252 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff3f00046c0 con 0x7ff3f4102d30 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.363+0000 7ff3fdd89740 1 Processor -- start 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.363+0000 7ff3fdd89740 1 -- start start 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c51880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c5c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c5ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561587bb72e0 con 0x561587c51dc0 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561587b5ece0 con 0x561587c5c680 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561587bb32a0 con 0x561587c514a0 2024-09-15T22:40:16.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f974c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c5c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c51880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c51880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:51162/0 (socket says 172.21.15.16:51162) 2024-09-15T22:40:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/2826342480 learned_addr learned my addr 172.21.15.16:0/2826342480 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f974c640 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c5c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56942/0 (socket says 172.21.15.16:56942) 2024-09-15T22:40:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c5ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/2826342480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 msgr2=0x561587c5ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c5ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/2826342480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 msgr2=0x561587c5c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c5c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/2826342480 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561587b43440 con 0x561587c514a0 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3ea7fc640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff3e00a33b0 0x7ff3e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3ea7fc640 1 -- 172.21.15.16:0/2122752252 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff3f00d0450 con 0x7ff3f4102d30 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.364+0000 7ff3f974c640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff3e00a33b0 0x7ff3e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.365+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c51880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009cd0 tx=0x7ff3ec009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ce15fe8cd3f01c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.365+0000 7ff3df7fe640 1 -- 172.21.15.16:0/2826342480 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec048070 con 0x561587c514a0 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.365+0000 7ff3df7fe640 1 -- 172.21.15.16:0/2826342480 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3ec036070 con 0x561587c514a0 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.365+0000 7ff3df7fe640 1 -- 172.21.15.16:0/2826342480 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0043a0 con 0x561587c514a0 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.365+0000 7ff3f974c640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff3e00a33b0 0x7ff3e00a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00cd8f0 tx=0x7ff3f0098040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2826342480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 msgr2=0x561587c51880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c51880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009cd0 tx=0x7ff3ec009a20 comp rx=0 tx=0).stop 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2826342480 shutdown_connections 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c5ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c5c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2826342480 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c51880 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2826342480 >> 172.21.15.16:0/2826342480 conn(0x561587c4da30 msgr2=0x561587c4de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2826342480 shutdown_connections 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2826342480 wait complete. 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 -- start start 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.366+0000 7ff3fdd89740 1 Processor -- start 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c514a0 0x561587c5bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c51dc0 0x561587c59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561587c5f250 con 0x561587c514a0 2024-09-15T22:40:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561587b72050 con 0x561587c5c680 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561587c5f3d0 con 0x561587c51dc0 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 10 client.0 _ll_register_callbacks cb 0x561587bb2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c514a0 0x561587c5bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c51dc0 0x561587c59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c514a0 0x561587c5bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56946/0 (socket says 172.21.15.16:56946) 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c51dc0 0x561587c59270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:51178/0 (socket says 172.21.15.16:51178) 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/1980518326 learned_addr learned my addr 172.21.15.16:0/1980518326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 -- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 msgr2=0x561587c597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 -- 172.21.15.16:0/1980518326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c514a0 msgr2=0x561587c5bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c514a0 0x561587c5bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 -- 172.21.15.16:0/1980518326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615879283c0 con 0x561587c51dc0 2024-09-15T22:40:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c51dc0 0x561587c59270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4102420 tx=0x7ff3e4012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba8be6f383a2296d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1980518326 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4004030 con 0x561587c51dc0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3ec036070 con 0x561587c51dc0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.367+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1980518326 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff3e40041d0 con 0x561587c51dc0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.368+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1980518326 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3e40043d0 con 0x561587c51dc0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.368+0000 7ff3fdd89740 10 client.25501 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.368+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561587c6ac90 con 0x561587c51dc0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.368+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1980518326 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3ec036070 con 0x561587c51dc0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.368+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1980518326 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3e4003a90 con 0x561587c51dc0 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3fdd89740 10 client.25501 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3fdd89740 10 client.25501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3fdd89740 20 client.25501 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587c6c0a0 0x561587c6e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3fdd89740 4 client.25501 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561587bc4850 con 0x561587c6c0a0 2024-09-15T22:40:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587c6c0a0 0x561587c6e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3dcff9640 10 client.25501 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.369+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587c6c0a0 0x561587c6e4e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca016af01e251912 server_cookie=d7b8382ba001ecd6 in_seq=0 out_seq=0 2024-09-15T22:40:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1980518326 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561587bc4850 con 0x561587c6c0a0 2024-09-15T22:40:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3dcff9640 10 client.25501 handle_command_reply: tid=0 2024-09-15T22:40:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 client.25501 shutdown 2024-09-15T22:40:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c51dc0 msgr2=0x561587c59270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c51dc0 0x561587c59270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4102420 tx=0x7ff3e4012b60 comp rx=0 tx=0).stop 2024-09-15T22:40:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 shutdown_connections 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587c6c0a0 0x561587c6e4e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c5c680 0x561587c597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c51dc0 0x561587c59270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3dcff9640 0 client.25501 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1980518326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c514a0 0x561587c5bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 >> 172.21.15.16:0/1980518326 conn(0x561587c4da30 msgr2=0x561587c4cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 shutdown_connections 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.370+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1980518326 wait complete. 2024-09-15T22:40:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.371+0000 7ff3fdd89740 20 client.25501 trim_cache size 0 max 16384 2024-09-15T22:40:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 Processor -- start 2024-09-15T22:40:16.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 -- start start 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c514a0 0x561587c684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c73db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c5c680 0x561587c742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41b3a80 con 0x561587c51dc0 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41b1350 con 0x561587c514a0 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41af1a0 con 0x561587c5c680 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3f8f4b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c73db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c514a0 0x561587c684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c514a0 0x561587c684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:32852/0 (socket says 172.21.15.16:32852) 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/2784783609 learned_addr learned my addr 172.21.15.16:0/2784783609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:16.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.389+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c5c680 0x561587c742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3f8f4b640 1 -- 172.21.15.16:0/2784783609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c5c680 msgr2=0x561587c742f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c5c680 0x561587c742f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3f8f4b640 1 -- 172.21.15.16:0/2784783609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c514a0 msgr2=0x561587c684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c514a0 0x561587c684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3f8f4b640 1 -- 172.21.15.16:0/2784783609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561587c51bf0 con 0x561587c51dc0 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c73db0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff3e401c500 tx=0x7ff3e4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f72919417af576 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3deffd640 1 -- 172.21.15.16:0/2784783609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e402e070 con 0x561587c51dc0 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3deffd640 1 -- 172.21.15.16:0/2784783609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e400fdd0 con 0x561587c51dc0 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.390+0000 7ff3deffd640 1 -- 172.21.15.16:0/2784783609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4005070 con 0x561587c51dc0 2024-09-15T22:40:16.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2784783609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 msgr2=0x561587c73db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c73db0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff3e401c500 tx=0x7ff3e4005db0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2784783609 shutdown_connections 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c5c680 0x561587c742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c51dc0 0x561587c73db0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/2784783609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c514a0 0x561587c684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2784783609 >> 172.21.15.16:0/2784783609 conn(0x561587c4da30 msgr2=0x561587c58360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2784783609 shutdown_connections 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 -- 172.21.15.16:0/2784783609 wait complete. 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 -- start start 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.391+0000 7ff3fdd89740 1 Processor -- start 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c64b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c51dc0 0x561587c5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c5c680 0x561587c603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41af810 con 0x561587c5c680 2024-09-15T22:40:16.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41b0c80 con 0x561587c51dc0 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f41afee0 con 0x561587c514a0 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 10 client.0 _ll_register_callbacks cb 0x7ff3e4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c5c680 0x561587c603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c5c680 0x561587c603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56970/0 (socket says 172.21.15.16:56970) 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/1836262729 learned_addr learned my addr 172.21.15.16:0/1836262729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c51dc0 0x561587c5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c64b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/1836262729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 msgr2=0x561587c64b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c64b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c51dc0 msgr2=0x561587c5feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c51dc0 0x561587c5feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 -- 172.21.15.16:0/1836262729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561587c609e0 con 0x561587c5c680 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3f9f4d640 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c5c680 0x561587c603f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006010 tx=0x7ff3ec009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c96b50bc276aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:16.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1836262729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec048040 con 0x561587c5c680 2024-09-15T22:40:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1836262729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff3ec0318c0 con 0x561587c5c680 2024-09-15T22:40:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.392+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e400fdd0 con 0x561587c5c680 2024-09-15T22:40:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1836262729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff3ec0316f0 con 0x561587c5c680 2024-09-15T22:40:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 10 client.15897 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561587cd6b70 con 0x561587c5c680 2024-09-15T22:40:16.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1836262729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3e400fdd0 con 0x561587c5c680 2024-09-15T22:40:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1836262729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3ec04e040 con 0x561587c5c680 2024-09-15T22:40:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 10 client.15897 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:16.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 20 client.15897 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587cd6e00 0x561587cd91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 4 client.15897 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561587bbed60 con 0x561587cd6e00 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.393+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587cd6e00 0x561587cd91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3f8f4b640 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587cd6e00 0x561587cd91f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ee0aa524443d2a server_cookie=de936de8c7df7526 in_seq=0 out_seq=0 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3dcff9640 10 client.15897 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3dcff9640 1 -- 172.21.15.16:0/1836262729 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561587bbed60 con 0x561587cd6e00 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3dcff9640 10 client.15897 handle_command_reply: tid=0 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 client.15897 shutdown 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c5c680 msgr2=0x561587c603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c5c680 0x561587c603f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006010 tx=0x7ff3ec009a20 comp rx=0 tx=0).stop 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 shutdown_connections 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561587cd6e00 0x561587cd91f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561587c5c680 0x561587c603f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561587c51dc0 0x561587c5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 --2- 172.21.15.16:0/1836262729 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561587c514a0 0x561587c64b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 >> 172.21.15.16:0/1836262729 conn(0x561587c4da30 msgr2=0x561587c52ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3dcff9640 0 client.15897 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.394+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 shutdown_connections 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fdd89740 1 -- 172.21.15.16:0/1836262729 wait complete. 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fdd89740 20 client.15897 trim_cache size 0 max 16384 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stdout: "b145a081-32e3-4953-9900-39c08b53757b": { 2024-09-15T22:40:16.400 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:40:16.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "b145a081-32e3-4953-9900-39c08b53757b", 2024-09-15T22:40:16.401 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:40:16.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:40:16.401 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:40:16.401 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff3e00a33b0 msgr2=0x7ff3e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff3e00a33b0 0x7ff3e00a5870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff3f00cd8f0 tx=0x7ff3f0098040 comp rx=0 tx=0).stop 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f4102d30 msgr2=0x7ff3f4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f4102d30 0x7ff3f4100990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e9f0 tx=0x7ff3f000edf0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 shutdown_connections 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3f41082a0 0x7ff3f4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff3e00a33b0 0x7ff3e00a5870 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff3f41036e0 0x7ff3f4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 --2- 172.21.15.16:0/2122752252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3f4102d30 0x7ff3f4100990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.395+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 >> 172.21.15.16:0/2122752252 conn(0x7ff3f40fc7d0 msgr2=0x7ff3f4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.396+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 shutdown_connections 2024-09-15T22:40:16.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:16.396+0000 7ff3fb9b0640 1 -- 172.21.15.16:0/2122752252 wait complete. 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-15T22:40:16.704 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-15T22:40:16.705 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-15T22:40:16.706 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-09-15T22:40:16.707 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-15T22:40:16.708 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-09-15T22:40:16.709 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-15T22:40:16.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:16 smithi059 ceph-mon[34872]: pgmap v58: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 960 KiB/s wr, 276 op/s 2024-09-15T22:40:17.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:16 smithi016 ceph-mon[31492]: pgmap v58: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 960 KiB/s wr, 276 op/s 2024-09-15T22:40:17.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:16 smithi130 ceph-mon[35742]: pgmap v58: 129 pgs: 129 active+clean; 318 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 82 MiB/s rd, 960 KiB/s wr, 276 op/s 2024-09-15T22:40:17.052 INFO:tasks.cephfs.filesystem:scrub status for tag:b145a081-32e3-4953-9900-39c08b53757b - {'path': '/', 'tag': 'b145a081-32e3-4953-9900-39c08b53757b', 'options': 'recursive,force'} 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-15T22:40:17.845 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-15T22:40:17.846 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-15T22:40:17.847 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-15T22:40:17.848 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-09-15T22:40:17.849 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-15T22:40:17.850 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-09-15T22:40:18.587 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-15T22:40:18.587 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-09-15T22:40:18.587 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-15T22:40:18.587 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-09-15T22:40:18.587 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-15T22:40:18.587 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-09-15T22:40:18.588 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-15T22:40:18.589 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-15T22:40:18.590 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-15T22:40:18.591 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-09-15T22:40:18.592 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-15T22:40:18.593 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-09-15T22:40:18.593 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-15T22:40:18.593 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-09-15T22:40:18.593 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-15T22:40:18.593 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-09-15T22:40:18.593 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-09-15T22:40:18.919 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-15T22:40:18.920 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-09-15T22:40:18.921 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-09-15T22:40:18.922 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-09-15T22:40:18.923 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-15T22:40:18.924 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-15T22:40:18.925 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-09-15T22:40:18.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:18 smithi059 ceph-mon[34872]: pgmap v59: 129 pgs: 129 active+clean; 297 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 1.4 MiB/s wr, 415 op/s 2024-09-15T22:40:18.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:18 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:40:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:18 smithi130 ceph-mon[35742]: pgmap v59: 129 pgs: 129 active+clean; 297 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 1.4 MiB/s wr, 415 op/s 2024-09-15T22:40:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:18 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:40:19.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:18 smithi016 ceph-mon[31492]: pgmap v59: 129 pgs: 129 active+clean; 297 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 127 MiB/s rd, 1.4 MiB/s wr, 415 op/s 2024-09-15T22:40:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:18 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:40:19.727 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-15T22:40:19.727 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-09-15T22:40:19.728 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-09-15T22:40:19.729 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-15T22:40:19.730 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-09-15T22:40:19.731 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-09-15T22:40:19.732 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-15T22:40:19.733 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-09-15T22:40:19.734 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-15T22:40:20.800 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-09-15T22:40:20.801 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-09-15T22:40:20.802 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-15T22:40:20.803 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-09-15T22:40:20.804 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-15T22:40:20.805 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-09-15T22:40:20.806 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-15T22:40:20.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:20 smithi059 ceph-mon[34872]: pgmap v60: 129 pgs: 129 active+clean; 297 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 910 KiB/s wr, 273 op/s 2024-09-15T22:40:21.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:20 smithi130 ceph-mon[35742]: pgmap v60: 129 pgs: 129 active+clean; 297 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 910 KiB/s wr, 273 op/s 2024-09-15T22:40:21.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:20 smithi016 ceph-mon[31492]: pgmap v60: 129 pgs: 129 active+clean; 297 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 910 KiB/s wr, 273 op/s 2024-09-15T22:40:21.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-09-15T22:40:21.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-15T22:40:21.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-09-15T22:40:21.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-15T22:40:21.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-09-15T22:40:21.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-15T22:40:21.308 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-15T22:40:21.309 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-15T22:40:21.310 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-15T22:40:21.311 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-15T22:40:21.312 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-15T22:40:21.313 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-15T22:40:21.314 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-09-15T22:40:21.687 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-09-15T22:40:21.688 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-09-15T22:40:21.689 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:21.690 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:21.691 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-15T22:40:21.692 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:21.693 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:21.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-15T22:40:21.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:21.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:21.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:21.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-15T22:40:21.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:21.694 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:21.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:21.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-15T22:40:21.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:21.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:21.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:21.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-15T22:40:21.753 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:21.754 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:21.754 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:21.754 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-15T22:40:21.754 INFO:tasks.workunit.client.0.smithi016.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:21.754 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:21.754 INFO:tasks.workunit.client.0.smithi016.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:21.754 INFO:tasks.workunit.client.0.smithi016.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-15T22:40:21.754 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T22:40:21.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T22:40:21.785 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-09-15T22:40:21.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T22:40:22.149 DEBUG:teuthology.parallel:result is None 2024-09-15T22:40:22.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T22:40:22.181 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T22:40:22.182 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-15T22:40:22.192 INFO:tasks.quiescer:joining Quiescers 2024-09-15T22:40:22.192 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-15T22:40:22.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:40:22.540 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:22.855 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:22 smithi016 ceph-mon[31492]: pgmap v61: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.3 MiB/s wr, 413 op/s 2024-09-15T22:40:22.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:22 smithi059 ceph-mon[34872]: pgmap v61: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.3 MiB/s wr, 413 op/s 2024-09-15T22:40:23.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:22 smithi130 ceph-mon[35742]: pgmap v61: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 134 MiB/s rd, 1.3 MiB/s wr, 413 op/s 2024-09-15T22:40:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 -- 172.21.15.16:0/2023462483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0106a80 msgr2=0x7f00f0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2023462483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0106a80 0x7f00f0106e60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009930 tx=0x7f00e002a420 comp rx=0 tx=0).stop 2024-09-15T22:40:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 -- 172.21.15.16:0/2023462483 shutdown_connections 2024-09-15T22:40:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2023462483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00f0101420 0x7f00f010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2023462483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0100a80 0x7f00f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2023462483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0106a80 0x7f00f0106e60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 -- 172.21.15.16:0/2023462483 >> 172.21.15.16:0/2023462483 conn(0x7f00f00fc7a0 msgr2=0x7f00f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 -- 172.21.15.16:0/2023462483 shutdown_connections 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.077+0000 7f00f57b0640 1 -- 172.21.15.16:0/2023462483 wait complete. 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 Processor -- start 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 -- start start 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0100a80 0x7f00f01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0101420 0x7f00f0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00f0106a80 0x7f00f01a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0110e80 con 0x7f00f0100a80 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0110d00 con 0x7f00f0101420 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.078+0000 7f00f57b0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f00f0111000 con 0x7f00f0106a80 2024-09-15T22:40:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0100a80 0x7f00f01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0100a80 0x7f00f01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52338/0 (socket says 172.21.15.16:52338) 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 -- 172.21.15.16:0/2373223694 learned_addr learned my addr 172.21.15.16:0/2373223694 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00ee7fc640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0101420 0x7f00f0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00ef7fe640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00f0106a80 0x7f00f01a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 -- 172.21.15.16:0/2373223694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00f0106a80 msgr2=0x7f00f01a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00f0106a80 0x7f00f01a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 -- 172.21.15.16:0/2373223694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0101420 msgr2=0x7f00f0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0101420 0x7f00f0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 -- 172.21.15.16:0/2373223694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f01a48b0 con 0x7f00f0100a80 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00ef7fe640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00f0106a80 0x7f00f01a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00ee7fc640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0101420 0x7f00f0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00eeffd640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0100a80 0x7f00f01992f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009230 tx=0x7f00e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00d7fff640 1 -- 172.21.15.16:0/2373223694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e003f030 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f01a4a80 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00d7fff640 1 -- 172.21.15.16:0/2373223694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0003ad0 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00d7fff640 1 -- 172.21.15.16:0/2373223694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035070 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.079+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f01a4f30 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.080+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f0100ee0 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.082+0000 7f00d7fff640 1 -- 172.21.15.16:0/2373223694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f00e0042030 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.082+0000 7f00d7fff640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f00d80a33b0 0x7f00d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.082+0000 7f00ee7fc640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f00d80a33b0 0x7f00d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.082+0000 7f00d7fff640 1 -- 172.21.15.16:0/2373223694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f00e00f0620 con 0x7f00f0100a80 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.083+0000 7f00ee7fc640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f00d80a33b0 0x7f00d80a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00e40058c0 tx=0x7f00e4005850 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.086+0000 7f00d7fff640 1 -- 172.21.15.16:0/2373223694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00e00bcca0 con 0x7f00f0100a80 2024-09-15T22:40:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.300+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00f0108ff0 con 0x7f00f0100a80 2024-09-15T22:40:23.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.301+0000 7f00d7fff640 1 -- 172.21.15.16:0/2373223694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f00e00bc5a0 con 0x7f00f0100a80 2024-09-15T22:40:23.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:40:23.303 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:40:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:40:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f00d80a33b0 msgr2=0x7f00d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f00d80a33b0 0x7f00d80a5870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00e40058c0 tx=0x7f00e4005850 comp rx=0 tx=0).stop 2024-09-15T22:40:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0100a80 msgr2=0x7f00f01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0100a80 0x7f00f01992f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009230 tx=0x7f00e0004270 comp rx=0 tx=0).stop 2024-09-15T22:40:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 shutdown_connections 2024-09-15T22:40:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f00d80a33b0 0x7f00d80a5870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f00f0106a80 0x7f00f01a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00f0101420 0x7f00f0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.303+0000 7f00f57b0640 1 --2- 172.21.15.16:0/2373223694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f00f0100a80 0x7f00f01992f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.304+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 >> 172.21.15.16:0/2373223694 conn(0x7f00f00fc7a0 msgr2=0x7f00f00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.304+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 shutdown_connections 2024-09-15T22:40:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:23.304+0000 7f00f57b0640 1 -- 172.21.15.16:0/2373223694 wait complete. 2024-09-15T22:40:23.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:23 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2373223694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:24.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:23 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2373223694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:24.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:23 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2373223694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:24.331 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --cancel --all' 2024-09-15T22:40:24.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --cancel --all 2024-09-15T22:40:24.617 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:24.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:24 smithi016 ceph-mon[31492]: pgmap v62: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 880 KiB/s wr, 278 op/s 2024-09-15T22:40:24.930 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:24 smithi130 ceph-mon[35742]: pgmap v62: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 880 KiB/s wr, 278 op/s 2024-09-15T22:40:24.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:24 smithi059 ceph-mon[34872]: pgmap v62: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 880 KiB/s wr, 278 op/s 2024-09-15T22:40:25.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 -- 172.21.15.16:0/2386398922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 msgr2=0x7f0774103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2386398922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f0774103110 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0764009930 tx=0x7f076402a420 comp rx=0 tx=0).stop 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 -- 172.21.15.16:0/2386398922 shutdown_connections 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2386398922 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07741082a0 0x7f077410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2386398922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07741036e0 0x7f0774107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2386398922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f0774103110 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 -- 172.21.15.16:0/2386398922 >> 172.21.15.16:0/2386398922 conn(0x7f07740fc7d0 msgr2=0x7f07740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 -- 172.21.15.16:0/2386398922 shutdown_connections 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.083+0000 7f0779a7c640 1 -- 172.21.15.16:0/2386398922 wait complete. 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 Processor -- start 2024-09-15T22:40:25.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 -- start start 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f07741985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07741036e0 0x7f0774198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07741082a0 0x7f077419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07741120a0 con 0x7f0774102d30 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0774111f20 con 0x7f07741082a0 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f0779a7c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0774112220 con 0x7f07741036e0 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.084+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f07741985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f07741985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52348/0 (socket says 172.21.15.16:52348) 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 -- 172.21.15.16:0/2216270614 learned_addr learned my addr 172.21.15.16:0/2216270614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f0773fff640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07741082a0 0x7f077419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f0772ffd640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07741036e0 0x7f0774198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 -- 172.21.15.16:0/2216270614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07741036e0 msgr2=0x7f0774198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07741036e0 0x7f0774198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 -- 172.21.15.16:0/2216270614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07741082a0 msgr2=0x7f077419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07741082a0 0x7f077419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 -- 172.21.15.16:0/2216270614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077419d550 con 0x7f0774102d30 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f07737fe640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f07741985a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0764009a50 tx=0x7f0764004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f0770ff9640 1 -- 172.21.15.16:0/2216270614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076403f030 con 0x7f0774102d30 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077419d7e0 con 0x7f0774102d30 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f0770ff9640 1 -- 172.21.15.16:0/2216270614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0764003ad0 con 0x7f0774102d30 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f0770ff9640 1 -- 172.21.15.16:0/2216270614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764035070 con 0x7f0774102d30 2024-09-15T22:40:25.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.085+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07741a5120 con 0x7f0774102d30 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.088+0000 7f0770ff9640 1 -- 172.21.15.16:0/2216270614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0764042030 con 0x7f0774102d30 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 Processor -- start 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 -- start start 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514888a00 0x562514888de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514889320 0x5625148936a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514895fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5625147ed200 con 0x562514889320 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562514793dc0 con 0x562514888a00 2024-09-15T22:40:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f077be55740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5625147e91c0 con 0x562514893be0 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514889320 0x5625148936a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514889320 0x5625148936a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52358/0 (socket says 172.21.15.16:52358) 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f07737fe640 1 -- 172.21.15.16:0/3257917904 learned_addr learned my addr 172.21.15.16:0/3257917904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f0773fff640 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514888a00 0x562514888de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.089+0000 7f0772ffd640 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514895fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0770ff9640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f075c0a33b0 0x7f075c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 -- 172.21.15.16:0/3257917904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 msgr2=0x562514895fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514895fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 -- 172.21.15.16:0/3257917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514889320 msgr2=0x5625148936a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514889320 0x5625148936a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 -- 172.21.15.16:0/3257917904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251455e620 con 0x562514888a00 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f07737fe640 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514889320 0x5625148936a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:25.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0770ff9640 1 -- 172.21.15.16:0/2216270614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f07640f07b0 con 0x7f0774102d30 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f075c0a33b0 0x7f075c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514888a00 0x562514888de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f076c00ad50 tx=0x7f076c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1eb95339b75d2ca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0759ffb640 1 -- 172.21.15.16:0/3257917904 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c019b00 con 0x562514888a00 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0759ffb640 1 -- 172.21.15.16:0/3257917904 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076c010760 con 0x562514888a00 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0759ffb640 1 -- 172.21.15.16:0/3257917904 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c016340 con 0x562514888a00 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.090+0000 7f0773fff640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f075c0a33b0 0x7f075c0a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f076c01e000 tx=0x7f076c01f000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 -- 172.21.15.16:0/3257917904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514888a00 msgr2=0x562514888de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514888a00 0x562514888de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f076c00ad50 tx=0x7f076c00f790 comp rx=0 tx=0).stop 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 -- 172.21.15.16:0/3257917904 shutdown_connections 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514895fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514889320 0x5625148936a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 --2- 172.21.15.16:0/3257917904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514888a00 0x562514888de0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 -- 172.21.15.16:0/3257917904 >> 172.21.15.16:0/3257917904 conn(0x562514884f90 msgr2=0x562514885370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 -- 172.21.15.16:0/3257917904 shutdown_connections 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.091+0000 7f077be55740 1 -- 172.21.15.16:0/3257917904 wait complete. 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 -- start start 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 Processor -- start 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x562514893490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x562514890740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514890c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5625148967b0 con 0x562514888a00 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562514787c50 con 0x562514889320 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562514896930 con 0x562514893be0 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f077be55740 10 client.0 _ll_register_callbacks cb 0x562514799ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x562514893490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x562514893490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52368/0 (socket says 172.21.15.16:52368) 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f07737fe640 1 -- 172.21.15.16:0/1104807427 learned_addr learned my addr 172.21.15.16:0/1104807427 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f0773fff640 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514890c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.092+0000 7f0772ffd640 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x562514890740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f07737fe640 1 -- 172.21.15.16:0/1104807427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 msgr2=0x562514890c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f07737fe640 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514890c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f07737fe640 1 -- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 msgr2=0x562514890740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f07737fe640 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x562514890740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f07737fe640 1 -- 172.21.15.16:0/1104807427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562514891480 con 0x562514888a00 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f07737fe640 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x562514893490 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f07640bc6c0 tx=0x7f07640314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d4c148c46b52d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f0752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f0752ffd640 1 -- 172.21.15.16:0/1104807427 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764049070 con 0x562514888a00 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f0752ffd640 1 -- 172.21.15.16:0/1104807427 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f07640ede20 con 0x562514888a00 2024-09-15T22:40:25.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f076c010760 con 0x562514888a00 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f0752ffd640 1 -- 172.21.15.16:0/1104807427 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07640b89d0 con 0x562514888a00 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f077be55740 10 client.15915 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5625148a22a0 con 0x562514888a00 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.093+0000 7f0752ffd640 1 -- 172.21.15.16:0/1104807427 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076c010760 con 0x562514888a00 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f0752ffd640 1 -- 172.21.15.16:0/1104807427 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f07640ef030 con 0x562514888a00 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f077be55740 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f077be55740 10 client.15915 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f077be55740 20 client.15915 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f077be55740 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5625148a3690 0x5625148a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f077be55740 4 client.15915 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:25.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625147fa770 con 0x5625148a3690 2024-09-15T22:40:25.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f0772ffd640 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5625148a3690 0x5625148a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f0772ffd640 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5625148a3690 0x5625148a5ae0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b411dcc6b1266c6c server_cookie=76240c8518d56edf in_seq=0 out_seq=0 2024-09-15T22:40:25.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.094+0000 7f0752ffd640 10 client.15915 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f0752ffd640 1 -- 172.21.15.16:0/1104807427 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625147fa770 con 0x5625148a3690 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f0752ffd640 10 client.15915 handle_command_reply: tid=0 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 client.15915 shutdown 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 msgr2=0x562514893490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x562514893490 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f07640bc6c0 tx=0x7f07640314b0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 shutdown_connections 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5625148a3690 0x5625148a5ae0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x562514890c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x562514890740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 --2- 172.21.15.16:0/1104807427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x562514893490 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 >> 172.21.15.16:0/1104807427 conn(0x562514884f90 msgr2=0x5625148841d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f0752ffd640 0 client.15915 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 shutdown_connections 2024-09-15T22:40:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.095+0000 7f077be55740 1 -- 172.21.15.16:0/1104807427 wait complete. 2024-09-15T22:40:25.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.096+0000 7f077be55740 20 client.15915 trim_cache size 0 max 16384 2024-09-15T22:40:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 Processor -- start 2024-09-15T22:40:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 -- start start 2024-09-15T22:40:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x56251488bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x5625148ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07741b1820 con 0x562514888a00 2024-09-15T22:40:25.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07741af0f0 con 0x562514889320 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f077be55740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f07741acf40 con 0x562514893be0 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52376/0 (socket says 172.21.15.16:52376) 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f07737fe640 1 -- 172.21.15.16:0/1090322170 learned_addr learned my addr 172.21.15.16:0/1090322170 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f0773fff640 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x5625148ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f0772ffd640 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x56251488bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f07737fe640 1 -- 172.21.15.16:0/1090322170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 msgr2=0x5625148ab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.113+0000 7f07737fe640 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x5625148ab3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f07737fe640 1 -- 172.21.15.16:0/1090322170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 msgr2=0x56251488bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f07737fe640 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x56251488bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f07737fe640 1 -- 172.21.15.16:0/1090322170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562514884dc0 con 0x562514888a00 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f0772ffd640 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x56251488bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:25.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f07737fe640 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489fa10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0764009900 tx=0x7f076403bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbd183b700fd5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:25.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f07597fa640 1 -- 172.21.15.16:0/1090322170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07640da070 con 0x562514888a00 2024-09-15T22:40:25.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f07597fa640 1 -- 172.21.15.16:0/1090322170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07640bc730 con 0x562514888a00 2024-09-15T22:40:25.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.114+0000 7f07597fa640 1 -- 172.21.15.16:0/1090322170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764049070 con 0x562514888a00 2024-09-15T22:40:25.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 -- 172.21.15.16:0/1090322170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 msgr2=0x56251489fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489fa10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0764009900 tx=0x7f076403bf90 comp rx=0 tx=0).stop 2024-09-15T22:40:25.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 -- 172.21.15.16:0/1090322170 shutdown_connections 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514893be0 0x5625148ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514889320 0x56251488bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 --2- 172.21.15.16:0/1090322170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489fa10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 -- 172.21.15.16:0/1090322170 >> 172.21.15.16:0/1090322170 conn(0x562514884f90 msgr2=0x56251488b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 -- 172.21.15.16:0/1090322170 shutdown_connections 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 -- 172.21.15.16:0/1090322170 wait complete. 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 -- start start 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 Processor -- start 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.115+0000 7f077be55740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f077be55740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514889320 0x56251489c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f077be55740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514893be0 0x562514897100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f077be55740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f07741ad5b0 con 0x562514888a00 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f077be55740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f07741aea20 con 0x562514893be0 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f077be55740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f07741adc80 con 0x562514889320 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f077be55740 10 client.0 _ll_register_callbacks cb 0x7f07640f6240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52384/0 (socket says 172.21.15.16:52384) 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 -- 172.21.15.16:0/2106173121 learned_addr learned my addr 172.21.15.16:0/2106173121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f0772ffd640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514889320 0x56251489c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f0773fff640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514893be0 0x562514897100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 -- 172.21.15.16:0/2106173121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514889320 msgr2=0x56251489c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514889320 0x56251489c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 -- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514893be0 msgr2=0x562514897100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514893be0 0x562514897100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 -- 172.21.15.16:0/2106173121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562514897900 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f0773fff640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514893be0 0x562514897100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f07737fe640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489bc30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f07640bc430 tx=0x7f07640314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaffe954e276f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f0752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f0752ffd640 1 -- 172.21.15.16:0/2106173121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07640da070 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f0752ffd640 1 -- 172.21.15.16:0/2106173121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0764031ba0 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.116+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07640bc730 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f0752ffd640 1 -- 172.21.15.16:0/2106173121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0764049040 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 10 client.15921 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562514897900 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f0752ffd640 1 -- 172.21.15.16:0/2106173121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07640bc730 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f0752ffd640 1 -- 172.21.15.16:0/2106173121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f07640ef070 con 0x562514888a00 2024-09-15T22:40:25.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 10 client.15921 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 10 client.15921 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 20 client.15921 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56251490e080 0x562514910470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 4 client.15921 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5625148a5e70 con 0x56251490e080 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.117+0000 7f0772ffd640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56251490e080 0x562514910470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f0772ffd640 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56251490e080 0x562514910470 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c28e981b3517487 server_cookie=bb1ab94ec28fcaf3 in_seq=0 out_seq=0 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f0752ffd640 10 client.15921 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f0752ffd640 1 -- 172.21.15.16:0/2106173121 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x5625148a5e70 con 0x56251490e080 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f0752ffd640 10 client.15921 handle_command_reply: tid=0 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 client.15921 shutdown 2024-09-15T22:40:25.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 msgr2=0x56251489bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489bc30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f07640bc430 tx=0x7f07640314b0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 shutdown_connections 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56251490e080 0x562514910470 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562514893be0 0x562514897100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562514889320 0x56251489c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f0752ffd640 0 client.15921 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 --2- 172.21.15.16:0/2106173121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562514888a00 0x56251489bc30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 >> 172.21.15.16:0/2106173121 conn(0x562514884f90 msgr2=0x5625148927c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 shutdown_connections 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.118+0000 7f077be55740 1 -- 172.21.15.16:0/2106173121 wait complete. 2024-09-15T22:40:25.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f077be55740 20 client.15921 trim_cache size 0 max 16384 2024-09-15T22:40:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:25.126 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 26, 2024-09-15T22:40:25.126 INFO:teuthology.orchestra.run.smithi016.stdout: "leader": 24500, 2024-09-15T22:40:25.126 INFO:teuthology.orchestra.run.smithi016.stdout: "set_version": 4, 2024-09-15T22:40:25.126 INFO:teuthology.orchestra.run.smithi016.stdout: "sets": {} 2024-09-15T22:40:25.126 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f075c0a33b0 msgr2=0x7f075c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f075c0a33b0 0x7f075c0a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f076c01e000 tx=0x7f076c01f000 comp rx=0 tx=0).stop 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 msgr2=0x7f07741985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f07741985a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0764009a50 tx=0x7f0764004270 comp rx=0 tx=0).stop 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 shutdown_connections 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f07741082a0 0x7f077419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f075c0a33b0 0x7f075c0a5870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f07741036e0 0x7f0774198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 --2- 172.21.15.16:0/2216270614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0774102d30 0x7f07741985a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 >> 172.21.15.16:0/2216270614 conn(0x7f07740fc7d0 msgr2=0x7f0774105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 shutdown_connections 2024-09-15T22:40:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:25.119+0000 7f0779a7c640 1 -- 172.21.15.16:0/2216270614 wait complete. 2024-09-15T22:40:25.878 INFO:tasks.quiescer:done joining Quiescers 2024-09-15T22:40:25.878 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-15T22:40:25.888 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-15T22:40:25.888 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:40:25.888 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:40:25.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-15T22:40:25.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-15T22:40:26.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:26 smithi059 ceph-mon[34872]: pgmap v63: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 880 KiB/s wr, 278 op/s 2024-09-15T22:40:27.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:26 smithi130 ceph-mon[35742]: pgmap v63: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 880 KiB/s wr, 278 op/s 2024-09-15T22:40:27.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:26 smithi016 ceph-mon[31492]: pgmap v63: 129 pgs: 129 active+clean; 300 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 880 KiB/s wr, 278 op/s 2024-09-15T22:40:28.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:28 smithi059 ceph-mon[34872]: pgmap v64: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 1.0 MiB/s wr, 331 op/s 2024-09-15T22:40:29.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:28 smithi130 ceph-mon[35742]: pgmap v64: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 1.0 MiB/s wr, 331 op/s 2024-09-15T22:40:29.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:28 smithi016 ceph-mon[31492]: pgmap v64: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 1.0 MiB/s wr, 331 op/s 2024-09-15T22:40:30.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:30 smithi059 ceph-mon[34872]: pgmap v65: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 590 KiB/s wr, 191 op/s 2024-09-15T22:40:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:30 smithi130 ceph-mon[35742]: pgmap v65: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 590 KiB/s wr, 191 op/s 2024-09-15T22:40:31.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:30 smithi016 ceph-mon[31492]: pgmap v65: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 590 KiB/s wr, 191 op/s 2024-09-15T22:40:32.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:32 smithi059 ceph-mon[34872]: pgmap v66: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 593 KiB/s wr, 195 op/s 2024-09-15T22:40:33.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:32 smithi130 ceph-mon[35742]: pgmap v66: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 593 KiB/s wr, 195 op/s 2024-09-15T22:40:33.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:32 smithi016 ceph-mon[31492]: pgmap v66: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 593 KiB/s wr, 195 op/s 2024-09-15T22:40:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:34 smithi059 ceph-mon[34872]: pgmap v67: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 146 KiB/s wr, 55 op/s 2024-09-15T22:40:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:34.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:34 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:35.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:34 smithi130 ceph-mon[35742]: pgmap v67: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 146 KiB/s wr, 55 op/s 2024-09-15T22:40:35.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:35.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:35.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:34 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:35.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:34 smithi016 ceph-mon[31492]: pgmap v67: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 146 KiB/s wr, 55 op/s 2024-09-15T22:40:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:35.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:34 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:35.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:35 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:35.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:35 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:35.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:35 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:36.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:35 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:35 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:35 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:36.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:35 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:35 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:36.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:35 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:40:36.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:36 smithi059 ceph-mon[34872]: pgmap v68: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 146 KiB/s wr, 55 op/s 2024-09-15T22:40:37.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:36 smithi130 ceph-mon[35742]: pgmap v68: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 146 KiB/s wr, 55 op/s 2024-09-15T22:40:37.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:36 smithi016 ceph-mon[31492]: pgmap v68: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 146 KiB/s wr, 55 op/s 2024-09-15T22:40:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:38 smithi059 ceph-mon[34872]: pgmap v69: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 146 KiB/s wr, 58 op/s 2024-09-15T22:40:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:38 smithi130 ceph-mon[35742]: pgmap v69: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 146 KiB/s wr, 58 op/s 2024-09-15T22:40:39.059 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:38 smithi016 ceph-mon[31492]: pgmap v69: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 146 KiB/s wr, 58 op/s 2024-09-15T22:40:40.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:40 smithi059 ceph-mon[34872]: pgmap v70: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-15T22:40:41.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:40 smithi130 ceph-mon[35742]: pgmap v70: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-15T22:40:41.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:40 smithi016 ceph-mon[31492]: pgmap v70: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-15T22:40:42.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:42 smithi059 ceph-mon[34872]: pgmap v71: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-15T22:40:43.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:42 smithi130 ceph-mon[35742]: pgmap v71: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-15T22:40:43.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:42 smithi016 ceph-mon[31492]: pgmap v71: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-15T22:40:44.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:44 smithi059 ceph-mon[34872]: pgmap v72: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:45.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:44 smithi130 ceph-mon[35742]: pgmap v72: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:45.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:44 smithi016 ceph-mon[31492]: pgmap v72: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:46.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:46 smithi059 ceph-mon[34872]: pgmap v73: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:47.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:46 smithi130 ceph-mon[35742]: pgmap v73: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:47.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:40:47.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:46 smithi016 ceph-mon[31492]: pgmap v73: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 -- 172.21.15.16:0/3920218339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8101420 msgr2=0x7efcb810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 --2- 172.21.15.16:0/3920218339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8101420 0x7efcb810d950 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009cd0 tx=0x7efcb002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 -- 172.21.15.16:0/3920218339 shutdown_connections 2024-09-15T22:40:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 --2- 172.21.15.16:0/3920218339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8101420 0x7efcb810d950 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 --2- 172.21.15.16:0/3920218339 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcb8100a80 0x7efcb8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 --2- 172.21.15.16:0/3920218339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb8106a80 0x7efcb8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 -- 172.21.15.16:0/3920218339 >> 172.21.15.16:0/3920218339 conn(0x7efcb80fc7a0 msgr2=0x7efcb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 -- 172.21.15.16:0/3920218339 shutdown_connections 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 -- 172.21.15.16:0/3920218339 wait complete. 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.796+0000 7efcbc935640 1 Processor -- start 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcbc935640 1 -- start start 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcbc935640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8100a80 0x7efcb8102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcbc935640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcb8101420 0x7efcb8102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcbc935640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb8106a80 0x7efcb81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcbc935640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8110e80 con 0x7efcb8100a80 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcbc935640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8110d00 con 0x7efcb8106a80 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcbc935640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8111000 con 0x7efcb8101420 2024-09-15T22:40:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8100a80 0x7efcb8102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8100a80 0x7efcb8102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52670/0 (socket says 172.21.15.16:52670) 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 -- 172.21.15.16:0/3872588302 learned_addr learned my addr 172.21.15.16:0/3872588302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcb8101420 0x7efcb8102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb8106a80 0x7efcb81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 -- 172.21.15.16:0/3872588302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcb8101420 msgr2=0x7efcb8102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcb8101420 0x7efcb8102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 -- 172.21.15.16:0/3872588302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb8106a80 msgr2=0x7efcb81030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb8106a80 0x7efcb81030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 -- 172.21.15.16:0/3872588302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb81a6900 con 0x7efcb8100a80 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb8106a80 0x7efcb81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.797+0000 7efcb659c640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8100a80 0x7efcb8102650 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efca800d950 tx=0x7efca800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.798+0000 7efca77fe640 1 -- 172.21.15.16:0/3872588302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8017070 con 0x7efcb8100a80 2024-09-15T22:40:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.798+0000 7efca77fe640 1 -- 172.21.15.16:0/3872588302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca8009c30 con 0x7efcb8100a80 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.798+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81a6bf0 con 0x7efcb8100a80 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.798+0000 7efca77fe640 1 -- 172.21.15.16:0/3872588302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca800f6b0 con 0x7efcb8100a80 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.798+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81a7180 con 0x7efcb8100a80 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.800+0000 7efca77fe640 1 -- 172.21.15.16:0/3872588302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efca80109d0 con 0x7efcb8100a80 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.800+0000 7efcbed0e740 1 Processor -- start 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.800+0000 7efcbed0e740 1 -- start start 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.800+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x557883953880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.800+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557883953dc0 0x55788395e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55788395e680 0x557883960a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcbed0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5578838b92e0 con 0x557883953dc0 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcbed0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557883860ce0 con 0x5578839534a0 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcbed0e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5578838b52a0 con 0x55788395e680 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557883953dc0 0x55788395e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efca77fe640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efca00a3430 0x7efca00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557883953dc0 0x55788395e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52682/0 (socket says 172.21.15.16:52682) 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb659c640 1 -- 172.21.15.16:0/504344023 learned_addr learned my addr 172.21.15.16:0/504344023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55788395e680 0x557883960a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x557883953880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efca00a3430 0x7efca00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efca77fe640 1 -- 172.21.15.16:0/3872588302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7efca80cfba0 con 0x7efcb8100a80 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb5d9b640 1 -- 172.21.15.16:0/504344023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 msgr2=0x557883953880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x557883953880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb5d9b640 1 -- 172.21.15.16:0/504344023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557883953dc0 msgr2=0x55788395e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557883953dc0 0x55788395e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb5d9b640 1 -- 172.21.15.16:0/504344023 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557883845440 con 0x55788395e680 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x557883953880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb659c640 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557883953dc0 0x55788395e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.801+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55788395e680 0x557883960a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efcac00bbc0 tx=0x7efcac0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813b318ba57904ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.802+0000 7efca4ff9640 1 -- 172.21.15.16:0/504344023 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac004030 con 0x55788395e680 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.802+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efca00a3430 0x7efca00a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcb0004af0 tx=0x7efcb00095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.802+0000 7efca4ff9640 1 -- 172.21.15.16:0/504344023 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcac0041d0 con 0x55788395e680 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.802+0000 7efca4ff9640 1 -- 172.21.15.16:0/504344023 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac0116a0 con 0x55788395e680 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.802+0000 7efcbed0e740 1 -- 172.21.15.16:0/504344023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55788395e680 msgr2=0x557883960a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.802+0000 7efcbed0e740 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55788395e680 0x557883960a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efcac00bbc0 tx=0x7efcac0107a0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.802+0000 7efcbed0e740 1 -- 172.21.15.16:0/504344023 shutdown_connections 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55788395e680 0x557883960a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557883953dc0 0x55788395e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 --2- 172.21.15.16:0/504344023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x557883953880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 -- 172.21.15.16:0/504344023 >> 172.21.15.16:0/504344023 conn(0x55788394fa30 msgr2=0x55788394fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 -- 172.21.15.16:0/504344023 shutdown_connections 2024-09-15T22:40:47.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 -- 172.21.15.16:0/504344023 wait complete. 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 -- start start 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 Processor -- start 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x55788395df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x55788395b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x55788395b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557883961250 con 0x55788395e680 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557883874050 con 0x5578839534a0 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5578839613d0 con 0x557883953dc0 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.803+0000 7efcbed0e740 10 client.0 _ll_register_callbacks cb 0x5578838b4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x55788395b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x55788395b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52686/0 (socket says 172.21.15.16:52686) 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb5d9b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x55788395b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 -- 172.21.15.16:0/3665361596 learned_addr learned my addr 172.21.15.16:0/3665361596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb659c640 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x55788395df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 -- 172.21.15.16:0/3665361596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 msgr2=0x55788395b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x55788395b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 -- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 msgr2=0x55788395df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x55788395df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 -- 172.21.15.16:0/3665361596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55788362a3c0 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x55788395b710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efcb0003bf0 tx=0x7efcb002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d66bc2b121521e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efc9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efc9dffb640 1 -- 172.21.15.16:0/3665361596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb004d070 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcac0041d0 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efc9dffb640 1 -- 172.21.15.16:0/3665361596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7efcb0004450 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.804+0000 7efc9dffb640 1 -- 172.21.15.16:0/3665361596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efcb0004650 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 10 client.15933 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55788396cbd0 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efc9dffb640 1 -- 172.21.15.16:0/3665361596 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcac0041d0 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efc9dffb640 1 -- 172.21.15.16:0/3665361596 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efcb0030070 con 0x55788395e680 2024-09-15T22:40:47.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 10 client.15933 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 20 client.15933 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55788396e090 0x5578839704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 4 client.15933 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578838c6850 con 0x55788396e090 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.805+0000 7efcb659c640 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55788396e090 0x5578839704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efc9dffb640 10 client.15933 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcb659c640 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55788396e090 0x5578839704e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d158c47b7037656d server_cookie=b5e06332e5a28d7e in_seq=0 out_seq=0 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efc9dffb640 1 -- 172.21.15.16:0/3665361596 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578838c6850 con 0x55788396e090 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efc9dffb640 10 client.15933 handle_command_reply: tid=0 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 client.15933 shutdown 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 msgr2=0x55788395b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x55788395b710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efcb0003bf0 tx=0x7efcb002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 shutdown_connections 2024-09-15T22:40:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55788396e090 0x5578839704e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x55788395b710 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x55788395b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 --2- 172.21.15.16:0/3665361596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5578839534a0 0x55788395df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efc9dffb640 0 client.15933 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.806+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 >> 172.21.15.16:0/3665361596 conn(0x55788394fa30 msgr2=0x557883955760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.807+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 shutdown_connections 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.807+0000 7efcbed0e740 1 -- 172.21.15.16:0/3665361596 wait complete. 2024-09-15T22:40:47.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.807+0000 7efcbed0e740 20 client.15933 trim_cache size 0 max 16384 2024-09-15T22:40:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.824+0000 7efcbed0e740 1 Processor -- start 2024-09-15T22:40:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.824+0000 7efcbed0e740 1 -- start start 2024-09-15T22:40:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5578839534a0 0x55788395d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557883953dc0 0x557883975db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x5578839762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcbed0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b3a00 con 0x55788395e680 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcbed0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b12d0 con 0x557883953dc0 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcbed0e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81af120 con 0x5578839534a0 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x5578839762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x5578839762f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52700/0 (socket says 172.21.15.16:52700) 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 -- 172.21.15.16:0/800584869 learned_addr learned my addr 172.21.15.16:0/800584869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb659c640 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5578839534a0 0x55788395d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557883953dc0 0x557883975db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 -- 172.21.15.16:0/800584869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5578839534a0 msgr2=0x55788395d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5578839534a0 0x55788395d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 -- 172.21.15.16:0/800584869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557883953dc0 msgr2=0x557883975db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557883953dc0 0x557883975db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 -- 172.21.15.16:0/800584869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55788394f860 con 0x55788395e680 2024-09-15T22:40:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557883953dc0 0x557883975db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x5578839762f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efcb00526c0 tx=0x7efcb0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d93ab14bd3ba379 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.825+0000 7efc9ffff640 1 -- 172.21.15.16:0/800584869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb004d040 con 0x55788395e680 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efc9ffff640 1 -- 172.21.15.16:0/800584869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0043070 con 0x55788395e680 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efc9ffff640 1 -- 172.21.15.16:0/800584869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0030ce0 con 0x55788395e680 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 -- 172.21.15.16:0/800584869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 msgr2=0x5578839762f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x5578839762f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efcb00526c0 tx=0x7efcb0032f40 comp rx=0 tx=0).stop 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 -- 172.21.15.16:0/800584869 shutdown_connections 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55788395e680 0x5578839762f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557883953dc0 0x557883975db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 --2- 172.21.15.16:0/800584869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5578839534a0 0x55788395d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 -- 172.21.15.16:0/800584869 >> 172.21.15.16:0/800584869 conn(0x55788394fa30 msgr2=0x55788395ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 -- 172.21.15.16:0/800584869 shutdown_connections 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.826+0000 7efcbed0e740 1 -- 172.21.15.16:0/800584869 wait complete. 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 -- start start 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 Processor -- start 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578839534a0 0x5578839669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x557883966f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55788395e680 0x557883963f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81af790 con 0x5578839534a0 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b0c00 con 0x55788395e680 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81afe60 con 0x557883953dc0 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcbed0e740 10 client.0 _ll_register_callbacks cb 0x7efcb0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcb659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578839534a0 0x5578839669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcb659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578839534a0 0x5578839669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52712/0 (socket says 172.21.15.16:52712) 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcb659c640 1 -- 172.21.15.16:0/1944715928 learned_addr learned my addr 172.21.15.16:0/1944715928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcb6d9d640 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55788395e680 0x557883963f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x557883966f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.827+0000 7efcb659c640 1 -- 172.21.15.16:0/1944715928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 msgr2=0x557883966f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcb659c640 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x557883966f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcb659c640 1 -- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55788395e680 msgr2=0x557883963f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcb659c640 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55788395e680 0x557883963f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcb659c640 1 -- 172.21.15.16:0/1944715928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578839647b0 con 0x5578839534a0 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcb659c640 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578839534a0 0x5578839669e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efca80cd590 tx=0x7efca8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb889254185f5e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efc9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efc9dffb640 1 -- 172.21.15.16:0/1944715928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8020040 con 0x5578839534a0 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efc9dffb640 1 -- 172.21.15.16:0/1944715928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7efca80cd210 con 0x5578839534a0 2024-09-15T22:40:47.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb0043070 con 0x5578839534a0 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efc9dffb640 1 -- 172.21.15.16:0/1944715928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efca809de60 con 0x5578839534a0 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcbed0e740 10 client.15939 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5578839d8a20 con 0x5578839534a0 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efc9dffb640 1 -- 172.21.15.16:0/1944715928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb0043070 con 0x5578839534a0 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.828+0000 7efc9dffb640 1 -- 172.21.15.16:0/1944715928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efca80ba350 con 0x5578839534a0 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 10 client.15939 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 20 client.15939 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5578839d90f0 0x5578839db4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 4 client.15939 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557883970870 con 0x5578839d90f0 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5578839d90f0 0x5578839db4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:47.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcb5d9b640 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5578839d90f0 0x5578839db4e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1d84d0857f7549 server_cookie=39b5470871c3f5f in_seq=0 out_seq=0 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efc9dffb640 10 client.15939 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efc9dffb640 1 -- 172.21.15.16:0/1944715928 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557883970870 con 0x5578839d90f0 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efc9dffb640 10 client.15939 handle_command_reply: tid=0 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 1 client.15939 shutdown 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578839534a0 msgr2=0x5578839669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.829+0000 7efcbed0e740 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578839534a0 0x5578839669e0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efca80cd590 tx=0x7efca8086040 comp rx=0 tx=0).stop 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 shutdown_connections 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5578839d90f0 0x5578839db4e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55788395e680 0x557883963f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557883953dc0 0x557883966f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 --2- 172.21.15.16:0/1944715928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5578839534a0 0x5578839669e0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 >> 172.21.15.16:0/1944715928 conn(0x55788394fa30 msgr2=0x55788395d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efc9dffb640 0 client.15939 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:47.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 shutdown_connections 2024-09-15T22:40:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 1 -- 172.21.15.16:0/1944715928 wait complete. 2024-09-15T22:40:47.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbed0e740 20 client.15939 trim_cache size 0 max 16384 2024-09-15T22:40:47.837 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:47.837 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:40:47.837 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:40:47.837 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efca00a3430 msgr2=0x7efca00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efca00a3430 0x7efca00a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcb0004af0 tx=0x7efcb00095c0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8100a80 msgr2=0x7efcb8102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8100a80 0x7efcb8102650 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efca800d950 tx=0x7efca800de10 comp rx=0 tx=0).stop 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 shutdown_connections 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb8106a80 0x7efcb81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efca00a3430 0x7efca00a58f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efcb8101420 0x7efcb8102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 --2- 172.21.15.16:0/3872588302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcb8100a80 0x7efcb8102650 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.830+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 >> 172.21.15.16:0/3872588302 conn(0x7efcb80fc7a0 msgr2=0x7efcb806a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.831+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 shutdown_connections 2024-09-15T22:40:47.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:47.831+0000 7efcbc935640 1 -- 172.21.15.16:0/3872588302 wait complete. 2024-09-15T22:40:48.668 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:40:48.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:40:48.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:48 smithi016 ceph-mon[31492]: pgmap v74: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:48.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:48 smithi059 ceph-mon[34872]: pgmap v74: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:48 smithi130 ceph-mon[35742]: pgmap v74: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 -- 172.21.15.16:0/1881958747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 msgr2=0x7f865810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 --2- 172.21.15.16:0/1881958747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865810ea90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8650009900 tx=0x7f865002a410 comp rx=0 tx=0).stop 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 -- 172.21.15.16:0/1881958747 shutdown_connections 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 --2- 172.21.15.16:0/1881958747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865810ea90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 --2- 172.21.15.16:0/1881958747 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8658102790 0x7f86581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 --2- 172.21.15.16:0/1881958747 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8658101de0 0x7f86581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 -- 172.21.15.16:0/1881958747 >> 172.21.15.16:0/1881958747 conn(0x7f86580fd8c0 msgr2=0x7f86580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 -- 172.21.15.16:0/1881958747 shutdown_connections 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.390+0000 7f865f180640 1 -- 172.21.15.16:0/1881958747 wait complete. 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 Processor -- start 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 -- start start 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8658101de0 0x7f865819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8658102790 0x7f865819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111f70 con 0x7f8658106f10 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8658111df0 con 0x7f8658101de0 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865f180640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f86581120f0 con 0x7f8658102790 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865d71d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f8657fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8658102790 0x7f865819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865d71d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52732/0 (socket says 172.21.15.16:52732) 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f8657fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8658102790 0x7f865819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:43890/0 (socket says 172.21.15.16:43890) 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865d71d640 1 -- 172.21.15.16:0/3878831169 learned_addr learned my addr 172.21.15.16:0/3878831169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.391+0000 7f865cf1c640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8658101de0 0x7f865819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865d71d640 1 -- 172.21.15.16:0/3878831169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8658102790 msgr2=0x7f865819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865d71d640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8658102790 0x7f865819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865d71d640 1 -- 172.21.15.16:0/3878831169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8658101de0 msgr2=0x7f865819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865d71d640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8658101de0 0x7f865819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865d71d640 1 -- 172.21.15.16:0/3878831169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865819f620 con 0x7f8658106f10 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865d71d640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865819ef40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8650005ab0 tx=0x7f8650004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f8655ffb640 1 -- 172.21.15.16:0/3878831169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865003f030 con 0x7f8658106f10 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f8655ffb640 1 -- 172.21.15.16:0/3878831169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8650003ad0 con 0x7f8658106f10 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865819f8b0 con 0x7f8658106f10 2024-09-15T22:40:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f8655ffb640 1 -- 172.21.15.16:0/3878831169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650035070 con 0x7f8658106f10 2024-09-15T22:40:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.392+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86581a71f0 con 0x7f8658106f10 2024-09-15T22:40:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.393+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86581021c0 con 0x7f8658106f10 2024-09-15T22:40:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.395+0000 7f8655ffb640 1 -- 172.21.15.16:0/3878831169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8650042030 con 0x7f8658106f10 2024-09-15T22:40:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.395+0000 7f8655ffb640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f86440a3430 0x7f86440a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.395+0000 7f865cf1c640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f86440a3430 0x7f86440a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.395+0000 7f8655ffb640 1 -- 172.21.15.16:0/3878831169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f86500f08f0 con 0x7f8658106f10 2024-09-15T22:40:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.396+0000 7f865cf1c640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f86440a3430 0x7f86440a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8640005fd0 tx=0x7f864000b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:49.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.400+0000 7f8655ffb640 1 -- 172.21.15.16:0/3878831169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86500bc920 con 0x7f8658106f10 2024-09-15T22:40:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.613+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86581a73c0 con 0x7f8658106f10 2024-09-15T22:40:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.614+0000 7f8655ffb640 1 -- 172.21.15.16:0/3878831169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f86500c0c20 con 0x7f8658106f10 2024-09-15T22:40:49.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:40:49.616 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:40:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:40:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f86440a3430 msgr2=0x7f86440a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f86440a3430 0x7f86440a58f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8640005fd0 tx=0x7f864000b390 comp rx=0 tx=0).stop 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 msgr2=0x7f865819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865819ef40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8650005ab0 tx=0x7f8650004270 comp rx=0 tx=0).stop 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 shutdown_connections 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8658106f10 0x7f865819ef40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f86440a3430 0x7f86440a58f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8658102790 0x7f865819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 --2- 172.21.15.16:0/3878831169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8658101de0 0x7f865819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 >> 172.21.15.16:0/3878831169 conn(0x7f86580fd8c0 msgr2=0x7f86580feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 shutdown_connections 2024-09-15T22:40:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:49.617+0000 7f865f180640 1 -- 172.21.15.16:0/3878831169 wait complete. 2024-09-15T22:40:49.929 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:49 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3878831169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:49.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:49 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3878831169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:50.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:49 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3878831169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:40:50.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:40:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:50.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:50 smithi016 ceph-mon[31492]: pgmap v75: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:50.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:50 smithi059 ceph-mon[34872]: pgmap v75: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:51.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:50 smithi130 ceph-mon[35742]: pgmap v75: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 -- 172.21.15.16:0/1710336820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 msgr2=0x7f5d9010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1710336820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9010ec70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009cd0 tx=0x7f5d8802a8d0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 -- 172.21.15.16:0/1710336820 shutdown_connections 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1710336820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9010ec70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1710336820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d90103780 0x7f5d90107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1710336820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5d90102dd0 0x7f5d901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 -- 172.21.15.16:0/1710336820 >> 172.21.15.16:0/1710336820 conn(0x7f5d900fc7d0 msgr2=0x7f5d900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 -- 172.21.15.16:0/1710336820 shutdown_connections 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.095+0000 7f5d968fd640 1 -- 172.21.15.16:0/1710336820 wait complete. 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 Processor -- start 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 -- start start 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d90102dd0 0x7f5d90198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5d90103780 0x7f5d90198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90112120 con 0x7f5d90108300 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90111fa0 con 0x7f5d90102dd0 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d968fd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901122a0 con 0x7f5d90103780 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52750/0 (socket says 172.21.15.16:52750) 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 -- 172.21.15.16:0/1393483445 learned_addr learned my addr 172.21.15.16:0/1393483445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d87fff640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5d90103780 0x7f5d90198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d90102dd0 0x7f5d90198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 -- 172.21.15.16:0/1393483445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5d90103780 msgr2=0x7f5d90198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5d90103780 0x7f5d90198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 -- 172.21.15.16:0/1393483445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d90102dd0 msgr2=0x7f5d90198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d90102dd0 0x7f5d90198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d94e9a640 1 -- 172.21.15.16:0/1393483445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9019d530 con 0x7f5d90108300 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d90102dd0 0x7f5d90198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.096+0000 7f5d87fff640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5d90103780 0x7f5d90198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.097+0000 7f5d94e9a640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9019ce50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5d88031040 tx=0x7f5d8802f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.097+0000 7f5d8dffb640 1 -- 172.21.15.16:0/1393483445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d880316a0 con 0x7f5d90108300 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.097+0000 7f5d8dffb640 1 -- 172.21.15.16:0/1393483445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d88003e50 con 0x7f5d90108300 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.097+0000 7f5d8dffb640 1 -- 172.21.15.16:0/1393483445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d880305b0 con 0x7f5d90108300 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.097+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9019d7c0 con 0x7f5d90108300 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.097+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d901a5100 con 0x7f5d90108300 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.099+0000 7f5d8dffb640 1 -- 172.21.15.16:0/1393483445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5d88004040 con 0x7f5d90108300 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.099+0000 7f5d98cd6740 1 Processor -- start 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.099+0000 7f5d98cd6740 1 -- start start 2024-09-15T22:40:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5db370 0x55a11c5dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a11c5362f0 con 0x55a11c5d1b40 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a11c4ddcf0 con 0x55a11c5d05d0 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a11c5322b0 con 0x55a11c5db370 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52752/0 (socket says 172.21.15.16:52752) 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 -- 172.21.15.16:0/3970448807 learned_addr learned my addr 172.21.15.16:0/3970448807 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d8dffb640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5d7c0a33b0 0x7f5d7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 -- 172.21.15.16:0/3970448807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5db370 msgr2=0x55a11c5dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5db370 0x55a11c5dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 -- 172.21.15.16:0/3970448807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 msgr2=0x55a11c5d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 -- 172.21.15.16:0/3970448807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a11c4c23a0 con 0x55a11c5d1b40 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d94e9a640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5d7c0a33b0 0x7f5d7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d8dffb640 1 -- 172.21.15.16:0/1393483445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5d880f0ee0 con 0x7f5d90108300 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.100+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d0090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5d78005f60 tx=0x7f5d780137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428edc5dd810d665 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d867fc640 1 -- 172.21.15.16:0/3970448807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780130d0 con 0x55a11c5d1b40 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d867fc640 1 -- 172.21.15.16:0/3970448807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d780035f0 con 0x55a11c5d1b40 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d867fc640 1 -- 172.21.15.16:0/3970448807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800f680 con 0x55a11c5d1b40 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3970448807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 msgr2=0x55a11c5d0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d94e9a640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5d7c0a33b0 0x7f5d7c0a5870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d880ee490 tx=0x7f5d880dd040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d0090 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5d78005f60 tx=0x7f5d780137d0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3970448807 shutdown_connections 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5db370 0x55a11c5dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3970448807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d0090 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3970448807 >> 172.21.15.16:0/3970448807 conn(0x55a11c599310 msgr2=0x55a11c5ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3970448807 shutdown_connections 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.101+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3970448807 wait complete. 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 -- start start 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 Processor -- start 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5db370 0x55a11c5d8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a11c5de070 con 0x55a11c5d1b40 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a11c4f1060 con 0x55a11c5db370 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55a11c5de1f0 con 0x55a11c5d05d0 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.102+0000 7f5d98cd6740 10 client.0 _ll_register_callbacks cb 0x55a11c2448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:51.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52764/0 (socket says 172.21.15.16:52764) 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 -- 172.21.15.16:0/3732413492 learned_addr learned my addr 172.21.15.16:0/3732413492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5db370 0x55a11c5d8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 -- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 msgr2=0x55a11c5dad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5dad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 -- 172.21.15.16:0/3732413492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5db370 msgr2=0x55a11c5d8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5db370 0x55a11c5d8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 -- 172.21.15.16:0/3732413492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11c2a73c0 con 0x55a11c5d1b40 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5db370 0x55a11c5d8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d7fb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5d80012480 tx=0x7f5d80012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ff71818700a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:51.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/3732413492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80005220 con 0x55a11c5d1b40 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/3732413492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5d800053c0 con 0x55a11c5d1b40 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d780035f0 con 0x55a11c5d1b40 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/3732413492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5d80025050 con 0x55a11c5d1b40 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d98cd6740 10 client.15957 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.103+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a11c5e9b00 con 0x55a11c5d1b40 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/3732413492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d780035f0 con 0x55a11c5d1b40 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/3732413492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5d80019030 con 0x55a11c5d1b40 2024-09-15T22:40:51.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d98cd6740 10 client.15957 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d98cd6740 10 client.15957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d98cd6740 20 client.15957 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c5eb060 0x55a11c5ed480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d98cd6740 4 client.15957 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a11c543860 con 0x55a11c5eb060 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.104+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c5eb060 0x55a11c5ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.105+0000 7f5d87fff640 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c5eb060 0x55a11c5ed480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbd16c2734e4ebe3 server_cookie=58caee8f02bd8403 in_seq=0 out_seq=0 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.105+0000 7f5d6f7fe640 10 client.15957 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.105+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/3732413492 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a11c543860 con 0x55a11c5eb060 2024-09-15T22:40:51.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.105+0000 7f5d6f7fe640 10 client.15957 handle_command_reply: tid=0 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.105+0000 7f5d98cd6740 1 client.15957 shutdown 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.105+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 msgr2=0x55a11c5d7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d7fb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5d80012480 tx=0x7f5d80012940 comp rx=0 tx=0).stop 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 shutdown_connections 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c5eb060 0x55a11c5ed480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5db370 0x55a11c5d8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5d1b40 0x55a11c5d7fb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/3732413492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5dad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d6f7fe640 0 client.15957 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 >> 172.21.15.16:0/3732413492 conn(0x55a11c599310 msgr2=0x55a11c5d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 shutdown_connections 2024-09-15T22:40:51.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 1 -- 172.21.15.16:0/3732413492 wait complete. 2024-09-15T22:40:51.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.106+0000 7f5d98cd6740 20 client.15957 trim_cache size 0 max 16384 2024-09-15T22:40:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 Processor -- start 2024-09-15T22:40:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 -- start start 2024-09-15T22:40:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5e7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d1b40 0x55a11c5f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c5f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901b1760 con 0x55a11c5db370 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901af030 con 0x55a11c5d05d0 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901ace80 con 0x55a11c5d1b40 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c5f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c5f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52776/0 (socket says 172.21.15.16:52776) 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d87fff640 1 -- 172.21.15.16:0/45610421 learned_addr learned my addr 172.21.15.16:0/45610421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.119+0000 7f5d87fff640 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d1b40 0x55a11c5f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d87fff640 1 -- 172.21.15.16:0/45610421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d1b40 msgr2=0x55a11c5f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d87fff640 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d1b40 0x55a11c5f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d87fff640 1 -- 172.21.15.16:0/45610421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 msgr2=0x55a11c5e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d87fff640 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d87fff640 1 -- 172.21.15.16:0/45610421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a11c5cd960 con 0x55a11c5db370 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d87fff640 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c5f3290 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009920 tx=0x7f5d78003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1329fdbc072de9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d85ffb640 1 -- 172.21.15.16:0/45610421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78025040 con 0x55a11c5db370 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d85ffb640 1 -- 172.21.15.16:0/45610421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d78003a30 con 0x55a11c5db370 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.120+0000 7f5d85ffb640 1 -- 172.21.15.16:0/45610421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7802ba20 con 0x55a11c5db370 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 -- 172.21.15.16:0/45610421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 msgr2=0x55a11c5f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c5f3290 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009920 tx=0x7f5d78003d00 comp rx=0 tx=0).stop 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 -- 172.21.15.16:0/45610421 shutdown_connections 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c5f3290 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d1b40 0x55a11c5f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/45610421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d05d0 0x55a11c5e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 -- 172.21.15.16:0/45610421 >> 172.21.15.16:0/45610421 conn(0x55a11c599310 msgr2=0x55a11c5d70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 -- 172.21.15.16:0/45610421 shutdown_connections 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 -- 172.21.15.16:0/45610421 wait complete. 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 -- start start 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.121+0000 7f5d98cd6740 1 Processor -- start 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5df940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d1b40 0x55a11c5dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d98cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c654d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901ad4f0 con 0x55a11c5db370 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901ae960 con 0x55a11c5d1b40 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d98cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901adbc0 con 0x55a11c5d05d0 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d98cd6740 10 client.0 _ll_register_callbacks cb 0x7f5d8000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c654d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c654d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52782/0 (socket says 172.21.15.16:52782) 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 -- 172.21.15.16:0/642318563 learned_addr learned my addr 172.21.15.16:0/642318563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d1b40 0x55a11c5dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 -- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 msgr2=0x55a11c5df940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5df940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 -- 172.21.15.16:0/642318563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d1b40 msgr2=0x55a11c5dfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d1b40 0x55a11c5dfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 -- 172.21.15.16:0/642318563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11c5d3120 con 0x55a11c5db370 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d1b40 0x55a11c5dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d87fff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c654d20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006010 tx=0x7f5d78001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf0a018157df09e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.122+0000 7f5d6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/642318563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7802b360 con 0x55a11c5db370 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/642318563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5d780050b0 con 0x55a11c5db370 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d78003a30 con 0x55a11c5db370 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/642318563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5d780052b0 con 0x55a11c5db370 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 10 client.15963 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a11c656360 con 0x55a11c5db370 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/642318563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d78003a30 con 0x55a11c5db370 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/642318563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5d78036070 con 0x55a11c5db370 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 10 client.15963 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 10 client.15963 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 20 client.15963 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c6567d0 0x55a11c658bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 4 client.15963 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.123+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a11c53dd70 con 0x55a11c6567d0 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c6567d0 0x55a11c658bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d6f7fe640 10 client.15963 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d8ffff640 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c6567d0 0x55a11c658bc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c82b912427e6628 server_cookie=890c41d9e33fef23 in_seq=0 out_seq=0 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d6f7fe640 1 -- 172.21.15.16:0/642318563 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a11c53dd70 con 0x55a11c6567d0 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d6f7fe640 10 client.15963 handle_command_reply: tid=0 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 client.15963 shutdown 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 msgr2=0x55a11c654d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c654d20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5d78006010 tx=0x7f5d78001e70 comp rx=0 tx=0).stop 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 shutdown_connections 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55a11c6567d0 0x55a11c658bc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a11c5db370 0x55a11c654d20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d6f7fe640 0 client.15963 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a11c5d1b40 0x55a11c5dfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 --2- 172.21.15.16:0/642318563 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55a11c5d05d0 0x55a11c5df940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.124+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 >> 172.21.15.16:0/642318563 conn(0x55a11c599310 msgr2=0x55a11c5cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 shutdown_connections 2024-09-15T22:40:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d98cd6740 1 -- 172.21.15.16:0/642318563 wait complete. 2024-09-15T22:40:51.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d98cd6740 20 client.15963 trim_cache size 0 max 16384 2024-09-15T22:40:51.133 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5d7c0a33b0 msgr2=0x7f5d7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5d7c0a33b0 0x7f5d7c0a5870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d880ee490 tx=0x7f5d880dd040 comp rx=0 tx=0).stop 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 msgr2=0x7f5d9019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9019ce50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5d88031040 tx=0x7f5d8802f420 comp rx=0 tx=0).stop 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 shutdown_connections 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d90108300 0x7f5d9019ce50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5d7c0a33b0 0x7f5d7c0a5870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5d90103780 0x7f5d90198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 --2- 172.21.15.16:0/1393483445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d90102dd0 0x7f5d90198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 >> 172.21.15.16:0/1393483445 conn(0x7f5d900fc7d0 msgr2=0x7f5d90106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 shutdown_connections 2024-09-15T22:40:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:51.125+0000 7f5d968fd640 1 -- 172.21.15.16:0/1393483445 wait complete. 2024-09-15T22:40:51.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:40:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 -- 172.21.15.16:0/2297627582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c101de0 msgr2=0x7f283c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 --2- 172.21.15.16:0/2297627582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c101de0 0x7f283c1021c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f282c009900 tx=0x7f282c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 -- 172.21.15.16:0/2297627582 shutdown_connections 2024-09-15T22:40:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 --2- 172.21.15.16:0/2297627582 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f283c106f10 0x7f283c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 --2- 172.21.15.16:0/2297627582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f283c102790 0x7f283c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 --2- 172.21.15.16:0/2297627582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c101de0 0x7f283c1021c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 -- 172.21.15.16:0/2297627582 >> 172.21.15.16:0/2297627582 conn(0x7f283c0fd8c0 msgr2=0x7f283c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 -- 172.21.15.16:0/2297627582 shutdown_connections 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.518+0000 7f2842127640 1 -- 172.21.15.16:0/2297627582 wait complete. 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 Processor -- start 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 -- start start 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f283c101de0 0x7f283c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f283c102790 0x7f283c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c106f10 0x7f283c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f283c111f70 con 0x7f283c106f10 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f283c111df0 con 0x7f283c101de0 2024-09-15T22:40:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f2842127640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1120f0 con 0x7f283c102790 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f283bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c106f10 0x7f283c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f283bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c106f10 0x7f283c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60494/0 (socket says 172.21.15.16:60494) 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f283bfff640 1 -- 172.21.15.16:0/3871549276 learned_addr learned my addr 172.21.15.16:0/3871549276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f283affd640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f283c102790 0x7f283c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.519+0000 7f283b7fe640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f283c101de0 0x7f283c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f283bfff640 1 -- 172.21.15.16:0/3871549276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f283c102790 msgr2=0x7f283c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f283bfff640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f283c102790 0x7f283c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f283bfff640 1 -- 172.21.15.16:0/3871549276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f283c101de0 msgr2=0x7f283c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f283bfff640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f283c101de0 0x7f283c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f283bfff640 1 -- 172.21.15.16:0/3871549276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c19d2b0 con 0x7f283c106f10 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f283bfff640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c106f10 0x7f283c19cbd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f283400cce0 tx=0x7f283400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f2838ff9640 1 -- 172.21.15.16:0/3871549276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283400f450 con 0x7f283c106f10 2024-09-15T22:40:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f2838ff9640 1 -- 172.21.15.16:0/3871549276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f283400fad0 con 0x7f283c106f10 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c19d5a0 con 0x7f283c106f10 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f2838ff9640 1 -- 172.21.15.16:0/3871549276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834018b60 con 0x7f283c106f10 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.520+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c1a4e80 con 0x7f283c106f10 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.522+0000 7f2838ff9640 1 -- 172.21.15.16:0/3871549276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f283400fc80 con 0x7f283c106f10 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 Processor -- start 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 -- start start 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb9b40 0x562176eb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176eb8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 0x562176ec5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562176e1e2f0 con 0x562176eb9b40 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562176dc5cf0 con 0x562176eb85d0 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2844500740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562176e1a2b0 con 0x562176ec3370 2024-09-15T22:40:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb9b40 0x562176eb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb9b40 0x562176eb8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60498/0 (socket says 172.21.15.16:60498) 2024-09-15T22:40:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 -- 172.21.15.16:0/2083443048 learned_addr learned my addr 172.21.15.16:0/2083443048 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2838ff9640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28240a33b0 0x7f28240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:52 smithi016 ceph-mon[31492]: pgmap v76: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283b7fe640 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176eb8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283bfff640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28240a33b0 0x7f28240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 -- 172.21.15.16:0/2083443048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 msgr2=0x562176ec5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 0x562176ec5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 -- 172.21.15.16:0/2083443048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 msgr2=0x562176eb8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176eb8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283affd640 1 -- 172.21.15.16:0/2083443048 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562176daa3a0 con 0x562176eb9b40 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f283b7fe640 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176eb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.523+0000 7f2838ff9640 1 -- 172.21.15.16:0/3871549276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f28340d04a0 con 0x7f283c106f10 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.524+0000 7f283affd640 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb9b40 0x562176eb8090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f283000ad10 tx=0x7f28300117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d9293a66e2a7b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.524+0000 7f2821ffb640 1 -- 172.21.15.16:0/2083443048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2830012450 con 0x562176eb9b40 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.524+0000 7f2821ffb640 1 -- 172.21.15.16:0/2083443048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2830012ad0 con 0x562176eb9b40 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.524+0000 7f2821ffb640 1 -- 172.21.15.16:0/2083443048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283000fb90 con 0x562176eb9b40 2024-09-15T22:40:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 -- 172.21.15.16:0/2083443048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb9b40 msgr2=0x562176eb8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f283bfff640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28240a33b0 0x7f28240a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f283409c3b0 tx=0x7f283406e1a0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb9b40 0x562176eb8090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f283000ad10 tx=0x7f28300117b0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 -- 172.21.15.16:0/2083443048 shutdown_connections 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 0x562176ec5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176eb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 --2- 172.21.15.16:0/2083443048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb9b40 0x562176eb8090 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 -- 172.21.15.16:0/2083443048 >> 172.21.15.16:0/2083443048 conn(0x562176e81310 msgr2=0x562176eb4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 -- 172.21.15.16:0/2083443048 shutdown_connections 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 -- 172.21.15.16:0/2083443048 wait complete. 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 -- start start 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.525+0000 7f2844500740 1 Processor -- start 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176ec2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb9b40 0x562176ec00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ec07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562176ec6070 con 0x562176ec3370 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562176dd9060 con 0x562176eb85d0 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562176ec61f0 con 0x562176eb9b40 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 10 client.0 _ll_register_callbacks cb 0x562176b2c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ec07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ec07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60514/0 (socket says 172.21.15.16:60514) 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 -- 172.21.15.16:0/579401445 learned_addr learned my addr 172.21.15.16:0/579401445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283b7fe640 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb9b40 0x562176ec00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176ec2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 -- 172.21.15.16:0/579401445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb9b40 msgr2=0x562176ec00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb9b40 0x562176ec00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 -- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 msgr2=0x562176ec2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176ec2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 -- 172.21.15.16:0/579401445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562176b8f3c0 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f283affd640 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ec07e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f283000af70 tx=0x7f283000f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6234d4096e70ef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f281affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f281affd640 1 -- 172.21.15.16:0/579401445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283001e070 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f281affd640 1 -- 172.21.15.16:0/579401445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f2830012760 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.526+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2830012ad0 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f281affd640 1 -- 172.21.15.16:0/579401445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2830030050 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 10 client.15975 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562176ed1ba0 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f281affd640 1 -- 172.21.15.16:0/579401445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2830012ad0 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f281affd640 1 -- 172.21.15.16:0/579401445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f283001a840 con 0x562176ec3370 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 10 client.15975 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 10 client.15975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 20 client.15975 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176ed2f50 0x562176ed5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 4 client.15975 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562176e2b860 con 0x562176ed2f50 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.527+0000 7f283b7fe640 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176ed2f50 0x562176ed5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.528+0000 7f283b7fe640 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176ed2f50 0x562176ed5340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3cd8eaa896f31ff server_cookie=4157a69cbd99885c in_seq=0 out_seq=0 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.528+0000 7f281affd640 10 client.15975 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f281affd640 1 -- 172.21.15.16:0/579401445 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562176e2b860 con 0x562176ed2f50 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f281affd640 10 client.15975 handle_command_reply: tid=0 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 client.15975 shutdown 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 msgr2=0x562176ec07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ec07e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f283000af70 tx=0x7f283000f770 comp rx=0 tx=0).stop 2024-09-15T22:40:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 shutdown_connections 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176ed2f50 0x562176ed5340 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ec07e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb9b40 0x562176ec00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f281affd640 0 client.15975 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 --2- 172.21.15.16:0/579401445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb85d0 0x562176ec2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 >> 172.21.15.16:0/579401445 conn(0x562176e81310 msgr2=0x562176eba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 shutdown_connections 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 1 -- 172.21.15.16:0/579401445 wait complete. 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.529+0000 7f2844500740 20 client.15975 trim_cache size 0 max 16384 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 Processor -- start 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 -- start start 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb85d0 0x562176ebf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ebb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1b1670 con 0x562176ec3370 2024-09-15T22:40:52.755 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:40:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1aef40 con 0x562176eb9b40 2024-09-15T22:40:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f2844500740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1acd90 con 0x562176eb85d0 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ebb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ebb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60518/0 (socket says 172.21.15.16:60518) 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f283affd640 1 -- 172.21.15.16:0/39260695 learned_addr learned my addr 172.21.15.16:0/39260695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.542+0000 7f283b7fe640 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283affd640 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb85d0 0x562176ebf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283affd640 1 -- 172.21.15.16:0/39260695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb85d0 msgr2=0x562176ebf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283affd640 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb85d0 0x562176ebf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283affd640 1 -- 172.21.15.16:0/39260695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 msgr2=0x562176ecf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283affd640 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283affd640 1 -- 172.21.15.16:0/39260695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562176ecf850 con 0x562176ec3370 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283b7fe640 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f283affd640 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ebb120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f283000a790 tx=0x7f283000afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c7853be6dc4763 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f28217fa640 1 -- 172.21.15.16:0/39260695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283001e070 con 0x562176ec3370 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f28217fa640 1 -- 172.21.15.16:0/39260695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2830011cb0 con 0x562176ec3370 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f28217fa640 1 -- 172.21.15.16:0/39260695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283000fba0 con 0x562176ec3370 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f2844500740 1 -- 172.21.15.16:0/39260695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 msgr2=0x562176ebb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.543+0000 7f2844500740 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ebb120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f283000a790 tx=0x7f283000afa0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- 172.21.15.16:0/39260695 shutdown_connections 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176ec3370 0x562176ebb120 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 --2- 172.21.15.16:0/39260695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176eb85d0 0x562176ebf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- 172.21.15.16:0/39260695 >> 172.21.15.16:0/39260695 conn(0x562176e81310 msgr2=0x562176eb3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- 172.21.15.16:0/39260695 shutdown_connections 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- 172.21.15.16:0/39260695 wait complete. 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- start start 2024-09-15T22:40:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 Processor -- start 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb85d0 0x562176ecb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 0x562176ebc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1ad400 con 0x562176eb85d0 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1ae870 con 0x562176eb9b40 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f283c1adad0 con 0x562176ec3370 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f2844500740 10 client.0 _ll_register_callbacks cb 0x7f283000ac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.544+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb85d0 0x562176ecb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb85d0 0x562176ecb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60530/0 (socket says 172.21.15.16:60530) 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 -- 172.21.15.16:0/3256876762 learned_addr learned my addr 172.21.15.16:0/3256876762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283b7fe640 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 0x562176ebc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 -- 172.21.15.16:0/3256876762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 msgr2=0x562176ebc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 0x562176ebc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 -- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 msgr2=0x562176ecbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 -- 172.21.15.16:0/3256876762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562176ebd160 con 0x562176eb85d0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f283affd640 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb85d0 0x562176ecb710 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f283000b010 tx=0x7f2830005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c87c2d4b81f969 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f281affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f281affd640 1 -- 172.21.15.16:0/3256876762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283001e070 con 0x562176eb85d0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f281affd640 1 -- 172.21.15.16:0/3256876762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f283001a400 con 0x562176eb85d0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2830011cb0 con 0x562176eb85d0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f281affd640 1 -- 172.21.15.16:0/3256876762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2830005460 con 0x562176eb85d0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f2844500740 10 client.15981 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562176f3d570 con 0x562176eb85d0 2024-09-15T22:40:52.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.545+0000 7f281affd640 1 -- 172.21.15.16:0/3256876762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2830011cb0 con 0x562176eb85d0 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f281affd640 1 -- 172.21.15.16:0/3256876762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2830004660 con 0x562176eb85d0 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f2844500740 10 client.15981 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f2844500740 10 client.15981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f2844500740 20 client.15981 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f2844500740 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176f3db20 0x562176f3ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f2844500740 4 client.15981 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562176e25d70 con 0x562176f3db20 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f283b7fe640 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176f3db20 0x562176f3ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f281affd640 10 client.15981 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.546+0000 7f283b7fe640 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176f3db20 0x562176f3ff10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9e731bacb4a810f server_cookie=43ebbadc42c3e3dc in_seq=0 out_seq=0 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f281affd640 1 -- 172.21.15.16:0/3256876762 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562176e25d70 con 0x562176f3db20 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f281affd640 10 client.15981 handle_command_reply: tid=0 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 client.15981 shutdown 2024-09-15T22:40:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb85d0 msgr2=0x562176ecb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb85d0 0x562176ecb710 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f283000b010 tx=0x7f2830005b10 comp rx=0 tx=0).stop 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 shutdown_connections 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562176f3db20 0x562176f3ff10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562176ec3370 0x562176ebc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f281affd640 0 client.15981 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562176eb9b40 0x562176ecbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 --2- 172.21.15.16:0/3256876762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562176eb85d0 0x562176ecb710 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 >> 172.21.15.16:0/3256876762 conn(0x562176e81310 msgr2=0x562176ec1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 shutdown_connections 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 1 -- 172.21.15.16:0/3256876762 wait complete. 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.547+0000 7f2844500740 20 client.15981 trim_cache size 0 max 16384 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28240a33b0 msgr2=0x7f28240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28240a33b0 0x7f28240a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f283409c3b0 tx=0x7f283406e1a0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c106f10 msgr2=0x7f283c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c106f10 0x7f283c19cbd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f283400cce0 tx=0x7f283400a620 comp rx=0 tx=0).stop 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 shutdown_connections 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c106f10 0x7f283c19cbd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28240a33b0 0x7f28240a5870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f283c102790 0x7f283c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 --2- 172.21.15.16:0/3871549276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f283c101de0 0x7f283c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 >> 172.21.15.16:0/3871549276 conn(0x7f283c0fd8c0 msgr2=0x7f283c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 shutdown_connections 2024-09-15T22:40:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:52.548+0000 7f2842127640 1 -- 172.21.15.16:0/3871549276 wait complete. 2024-09-15T22:40:52.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:52 smithi059 ceph-mon[34872]: pgmap v76: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:53.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:52 smithi130 ceph-mon[35742]: pgmap v76: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:53.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:40:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:54.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.278+0000 7fa24bb23640 1 -- 172.21.15.16:0/1412968832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244100a80 msgr2=0x7fa244100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 --2- 172.21.15.16:0/1412968832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244100a80 0x7fa244100ee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa240009930 tx=0x7fa24002a420 comp rx=0 tx=0).stop 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 -- 172.21.15.16:0/1412968832 shutdown_connections 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 --2- 172.21.15.16:0/1412968832 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa244101420 0x7fa24410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 --2- 172.21.15.16:0/1412968832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244100a80 0x7fa244100ee0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 --2- 172.21.15.16:0/1412968832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244106a80 0x7fa244106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 -- 172.21.15.16:0/1412968832 >> 172.21.15.16:0/1412968832 conn(0x7fa2440fc7a0 msgr2=0x7fa2440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 -- 172.21.15.16:0/1412968832 shutdown_connections 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 -- 172.21.15.16:0/1412968832 wait complete. 2024-09-15T22:40:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 Processor -- start 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.279+0000 7fa24bb23640 1 -- start start 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa24bb23640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa244100a80 0x7fa2441973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa24bb23640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244101420 0x7fa2441978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa24bb23640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244106a80 0x7fa24419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa24bb23640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa244110e80 con 0x7fa244101420 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa24bb23640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa244110d00 con 0x7fa244106a80 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa24bb23640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa244111000 con 0x7fa244100a80 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244101420 0x7fa2441978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244101420 0x7fa2441978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60552/0 (socket says 172.21.15.16:60552) 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 -- 172.21.15.16:0/4244340938 learned_addr learned my addr 172.21.15.16:0/4244340938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2498bf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa244100a80 0x7fa2441973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244106a80 0x7fa24419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 -- 172.21.15.16:0/4244340938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa244100a80 msgr2=0x7fa2441973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa244100a80 0x7fa2441973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 -- 172.21.15.16:0/4244340938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244106a80 msgr2=0x7fa24419bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244106a80 0x7fa24419bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 -- 172.21.15.16:0/4244340938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24419c360 con 0x7fa244101420 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.280+0000 7fa2490be640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244101420 0x7fa2441978f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa240009230 tx=0x7fa240004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.281+0000 7fa23affd640 1 -- 172.21.15.16:0/4244340938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24003f030 con 0x7fa244101420 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.281+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24419c5f0 con 0x7fa244101420 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.281+0000 7fa23affd640 1 -- 172.21.15.16:0/4244340938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240003ad0 con 0x7fa244101420 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.281+0000 7fa23affd640 1 -- 172.21.15.16:0/4244340938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240035070 con 0x7fa244101420 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.281+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24406a790 con 0x7fa244101420 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.283+0000 7fa23affd640 1 -- 172.21.15.16:0/4244340938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa240042030 con 0x7fa244101420 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.283+0000 7fa24defc740 1 Processor -- start 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.283+0000 7fa24defc740 1 -- start start 2024-09-15T22:40:54.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24defc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644cb40 0x55672644b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24defc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672644ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24defc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726458760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24defc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5567263b12f0 con 0x556726456370 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24defc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556726358cf0 con 0x55672644b5d0 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24defc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5567263ad2b0 con 0x55672644cb40 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa2490be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726458760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644cb40 0x55672644b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644cb40 0x55672644b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:51306/0 (socket says 172.21.15.16:51306) 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3789829679 learned_addr learned my addr 172.21.15.16:0/3789829679 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa2498bf640 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672644ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa23affd640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2300a33b0 0x7fa2300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3789829679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 msgr2=0x55672644ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672644ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3789829679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 msgr2=0x556726458760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726458760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3789829679 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55672633d3a0 con 0x55672644cb40 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa2490be640 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726458760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa23affd640 1 -- 172.21.15.16:0/4244340938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa2400f03f0 con 0x7fa244101420 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2300a33b0 0x7fa2300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.284+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644cb40 0x55672644b090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa23c003b10 tx=0x7fa23c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4846e5fbac2f7def server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.285+0000 7fa22ffff640 1 -- 172.21.15.16:0/3789829679 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c024070 con 0x55672644cb40 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.285+0000 7fa22ffff640 1 -- 172.21.15.16:0/3789829679 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c010ce0 con 0x55672644cb40 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.285+0000 7fa22ffff640 1 -- 172.21.15.16:0/3789829679 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c01ed30 con 0x55672644cb40 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.285+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2300a33b0 0x7fa2300a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa23c021000 tx=0x7fa23c01e000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.285+0000 7fa24defc740 1 -- 172.21.15.16:0/3789829679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644cb40 msgr2=0x55672644b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.285+0000 7fa24defc740 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644cb40 0x55672644b090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa23c003b10 tx=0x7fa23c009f90 comp rx=0 tx=0).stop 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 -- 172.21.15.16:0/3789829679 shutdown_connections 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726458760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672644ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 --2- 172.21.15.16:0/3789829679 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644cb40 0x55672644b090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 -- 172.21.15.16:0/3789829679 >> 172.21.15.16:0/3789829679 conn(0x556726414310 msgr2=0x556726447c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 -- 172.21.15.16:0/3789829679 shutdown_connections 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 -- 172.21.15.16:0/3789829679 wait complete. 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 -- start start 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.286+0000 7fa24defc740 1 Processor -- start 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644b5d0 0x556726455d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726453020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726453610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556726459070 con 0x556726456370 2024-09-15T22:40:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55672636c060 con 0x55672644cb40 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5567264591f0 con 0x55672644b5d0 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 10 client.0 _ll_register_callbacks cb 0x5567260bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726453610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa2498bf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644b5d0 0x556726455d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726453610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60578/0 (socket says 172.21.15.16:60578) 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3431314843 learned_addr learned my addr 172.21.15.16:0/3431314843 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa2490be640 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726453020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3431314843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644b5d0 msgr2=0x556726455d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644b5d0 0x556726455d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 msgr2=0x556726453020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726453020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 -- 172.21.15.16:0/3431314843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567261223c0 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa2498bf640 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644b5d0 0x556726455d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726453610 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa23c004810 tx=0x7fa23c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42920707f81733 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa22d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa22d7fa640 1 -- 172.21.15.16:0/3431314843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c024040 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa22d7fa640 1 -- 172.21.15.16:0/3431314843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa23c01e680 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.287+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa23c010ce0 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa22d7fa640 1 -- 172.21.15.16:0/3431314843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa23c039050 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 10 client.15993 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556726464b10 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa22d7fa640 1 -- 172.21.15.16:0/3431314843 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa23c010ce0 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa22d7fa640 1 -- 172.21.15.16:0/3431314843 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa23c011500 con 0x556726456370 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 10 client.15993 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 20 client.15993 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556726465e60 0x556726468250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 4 client.15993 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567263be860 con 0x556726465e60 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.288+0000 7fa2498bf640 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556726465e60 0x556726468250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.289+0000 7fa2498bf640 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556726465e60 0x556726468250 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0a835dd2458e29 server_cookie=cdca93d9a63a6cf3 in_seq=0 out_seq=0 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.289+0000 7fa22d7fa640 10 client.15993 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa22d7fa640 1 -- 172.21.15.16:0/3431314843 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567263be860 con 0x556726465e60 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa22d7fa640 10 client.15993 handle_command_reply: tid=0 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 client.15993 shutdown 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 msgr2=0x556726453610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726453610 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa23c004810 tx=0x7fa23c009da0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 shutdown_connections 2024-09-15T22:40:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556726465e60 0x556726468250 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556726456370 0x556726453610 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726453020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 --2- 172.21.15.16:0/3431314843 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55672644b5d0 0x556726455d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 >> 172.21.15.16:0/3431314843 conn(0x556726414310 msgr2=0x55672644d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa22d7fa640 0 client.15993 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 shutdown_connections 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 1 -- 172.21.15.16:0/3431314843 wait complete. 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.290+0000 7fa24defc740 20 client.15993 trim_cache size 0 max 16384 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.303+0000 7fa24defc740 1 Processor -- start 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.303+0000 7fa24defc740 1 -- start start 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24defc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644b5d0 0x556726447e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24defc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726462310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24defc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x55672646d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24defc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441b1820 con 0x55672644b5d0 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24defc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441af0f0 con 0x55672644cb40 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24defc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441acf40 con 0x556726456370 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644b5d0 0x556726447e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644b5d0 0x556726447e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60586/0 (socket says 172.21.15.16:60586) 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 -- 172.21.15.16:0/447487490 learned_addr learned my addr 172.21.15.16:0/447487490 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x55672646d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2490be640 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726462310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 -- 172.21.15.16:0/447487490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 msgr2=0x55672646d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x55672646d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 -- 172.21.15.16:0/447487490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 msgr2=0x556726462310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726462310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 -- 172.21.15.16:0/447487490 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556726451570 con 0x55672644b5d0 2024-09-15T22:40:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x55672646d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa2498bf640 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644b5d0 0x556726447e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa234012550 tx=0x7fa234005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d420bf3014326c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.304+0000 7fa22f7fe640 1 -- 172.21.15.16:0/447487490 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234004030 con 0x55672644b5d0 2024-09-15T22:40:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa22f7fe640 1 -- 172.21.15.16:0/447487490 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2340041d0 con 0x55672644b5d0 2024-09-15T22:40:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa22f7fe640 1 -- 172.21.15.16:0/447487490 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2340208c0 con 0x55672644b5d0 2024-09-15T22:40:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 -- 172.21.15.16:0/447487490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644b5d0 msgr2=0x556726447e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644b5d0 0x556726447e10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa234012550 tx=0x7fa234005f90 comp rx=0 tx=0).stop 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 -- 172.21.15.16:0/447487490 shutdown_connections 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x55672646d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644cb40 0x556726462310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 --2- 172.21.15.16:0/447487490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644b5d0 0x556726447e10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 -- 172.21.15.16:0/447487490 >> 172.21.15.16:0/447487490 conn(0x556726414310 msgr2=0x556726447700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 -- 172.21.15.16:0/447487490 shutdown_connections 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.305+0000 7fa24defc740 1 -- 172.21.15.16:0/447487490 wait complete. 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 -- start start 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 Processor -- start 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672645e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644cb40 0x55672645eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x5567264c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441ad5b0 con 0x55672644cb40 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441aea20 con 0x55672644b5d0 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2441adc80 con 0x556726456370 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa24defc740 10 client.0 _ll_register_callbacks cb 0x7fa23c022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa2490be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644cb40 0x55672645eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa2490be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644cb40 0x55672645eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60600/0 (socket says 172.21.15.16:60600) 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa2498bf640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672645e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.306+0000 7fa2490be640 1 -- 172.21.15.16:0/1129715510 learned_addr learned my addr 172.21.15.16:0/1129715510 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa24a0c0640 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x5567264c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa2490be640 1 -- 172.21.15.16:0/1129715510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 msgr2=0x5567264c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa2490be640 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x5567264c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa2490be640 1 -- 172.21.15.16:0/1129715510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 msgr2=0x55672645e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa2490be640 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672645e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa2490be640 1 -- 172.21.15.16:0/1129715510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567264c96a0 con 0x55672644cb40 2024-09-15T22:40:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa2490be640 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644cb40 0x55672645eac0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2400b8df0 tx=0x7fa2400f6550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356da61d8db3d721 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa22d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa22d7fa640 1 -- 172.21.15.16:0/1129715510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240049040 con 0x55672644cb40 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2340041d0 con 0x55672644cb40 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa22d7fa640 1 -- 172.21.15.16:0/1129715510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa2400eda60 con 0x55672644cb40 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa22d7fa640 1 -- 172.21.15.16:0/1129715510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2400bbf60 con 0x55672644cb40 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa24defc740 10 client.15999 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55672645bff0 con 0x55672644cb40 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.307+0000 7fa22d7fa640 1 -- 172.21.15.16:0/1129715510 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2340041d0 con 0x55672644cb40 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa22d7fa640 1 -- 172.21.15.16:0/1129715510 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2400d51f0 con 0x55672644cb40 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa24defc740 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa24defc740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa24defc740 20 client.15999 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa24defc740 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5567264d4f10 0x5567264d7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa24defc740 4 client.15999 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567263b8d70 con 0x5567264d4f10 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa2498bf640 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5567264d4f10 0x5567264d7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa22d7fa640 10 client.15999 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa2498bf640 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5567264d4f10 0x5567264d7330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2059e7eb5baaef69 server_cookie=b9d92067d45d6f2 in_seq=0 out_seq=0 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa22d7fa640 1 -- 172.21.15.16:0/1129715510 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567263b8d70 con 0x5567264d4f10 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa22d7fa640 10 client.15999 handle_command_reply: tid=0 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.308+0000 7fa24defc740 1 client.15999 shutdown 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644cb40 msgr2=0x55672645eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644cb40 0x55672645eac0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa2400b8df0 tx=0x7fa2400f6550 comp rx=0 tx=0).stop 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 shutdown_connections 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5567264d4f10 0x5567264d7330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556726456370 0x5567264c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55672644cb40 0x55672645eac0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 --2- 172.21.15.16:0/1129715510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55672644b5d0 0x55672645e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 >> 172.21.15.16:0/1129715510 conn(0x556726414310 msgr2=0x5567264518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa22d7fa640 0 client.15999 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 shutdown_connections 2024-09-15T22:40:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 1 -- 172.21.15.16:0/1129715510 wait complete. 2024-09-15T22:40:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24defc740 20 client.15999 trim_cache size 0 max 16384 2024-09-15T22:40:54.424 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:40:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2300a33b0 msgr2=0x7fa2300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24bb23640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2300a33b0 0x7fa2300a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa23c021000 tx=0x7fa23c01e000 comp rx=0 tx=0).stop 2024-09-15T22:40:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244101420 msgr2=0x7fa2441978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.309+0000 7fa24bb23640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244101420 0x7fa2441978f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa240009230 tx=0x7fa240004270 comp rx=0 tx=0).stop 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 shutdown_connections 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa244106a80 0x7fa24419bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa244101420 0x7fa2441978f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2300a33b0 0x7fa2300a5870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 --2- 172.21.15.16:0/4244340938 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa244100a80 0x7fa2441973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 >> 172.21.15.16:0/4244340938 conn(0x7fa2440fc7a0 msgr2=0x7fa2440fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 shutdown_connections 2024-09-15T22:40:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:54.310+0000 7fa24bb23640 1 -- 172.21.15.16:0/4244340938 wait complete. 2024-09-15T22:40:54.900 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:54 smithi016 ceph-mon[31492]: pgmap v77: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:54.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:54 smithi059 ceph-mon[34872]: pgmap v77: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:55.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:54 smithi130 ceph-mon[35742]: pgmap v77: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:55.931 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:40:55.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:40:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 -- 172.21.15.16:0/451548432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 msgr2=0x7f6034101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 --2- 172.21.15.16:0/451548432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f6034101200 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6030009900 tx=0x7f603002a480 comp rx=0 tx=0).stop 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 -- 172.21.15.16:0/451548432 shutdown_connections 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 --2- 172.21.15.16:0/451548432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6034102ef0 0x7f603410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 --2- 172.21.15.16:0/451548432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6034102570 0x7f60341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 --2- 172.21.15.16:0/451548432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034100e20 0x7f6034101200 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 -- 172.21.15.16:0/451548432 >> 172.21.15.16:0/451548432 conn(0x7f60340fc7d0 msgr2=0x7f60340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 -- 172.21.15.16:0/451548432 shutdown_connections 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.680+0000 7f603b855640 1 -- 172.21.15.16:0/451548432 wait complete. 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 Processor -- start 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 -- start start 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6034100e20 0x7f60341985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102570 0x7f6034198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6034102ef0 0x7f603419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6034110fa0 con 0x7f6034102570 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6034110e20 con 0x7f6034102ef0 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f603b855640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6034111120 con 0x7f6034100e20 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102570 0x7f6034198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102570 0x7f6034198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60622/0 (socket says 172.21.15.16:60622) 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 -- 172.21.15.16:0/1963637834 learned_addr learned my addr 172.21.15.16:0/1963637834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f60395f1640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6034100e20 0x7f60341985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6039df2640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6034102ef0 0x7f603419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 -- 172.21.15.16:0/1963637834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6034100e20 msgr2=0x7f60341985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6034100e20 0x7f60341985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 -- 172.21.15.16:0/1963637834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6034102ef0 msgr2=0x7f603419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6034102ef0 0x7f603419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 -- 172.21.15.16:0/1963637834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603419d570 con 0x7f6034102570 2024-09-15T22:40:56.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.681+0000 7f6038df0640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102570 0x7f6034198b00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f602400b4a0 tx=0x7f602400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.682+0000 7f602a7fc640 1 -- 172.21.15.16:0/1963637834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024014070 con 0x7f6034102570 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.682+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603419d860 con 0x7f6034102570 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.682+0000 7f602a7fc640 1 -- 172.21.15.16:0/1963637834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6024010af0 con 0x7f6034102570 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.682+0000 7f602a7fc640 1 -- 172.21.15.16:0/1963637834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024017d10 con 0x7f6034102570 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.682+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a5140 con 0x7f6034102570 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.684+0000 7f602a7fc640 1 -- 172.21.15.16:0/1963637834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6024017070 con 0x7f6034102570 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.684+0000 7f603dc2e740 1 Processor -- start 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.684+0000 7f603dc2e740 1 -- start start 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.684+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a34f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a35480 0x559249a3f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a3fd40 0x559249a42130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f603dc2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55924999a280 con 0x559249a35480 2024-09-15T22:40:56.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f603dc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559249941b10 con 0x559249a3fd40 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f603dc2e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559249996240 con 0x559249a34b60 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a35480 0x559249a3f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f6039df2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a34f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a35480 0x559249a3f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60638/0 (socket says 172.21.15.16:60638) 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f602a7fc640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f60200a33b0 0x7f60200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 -- 172.21.15.16:0/2764196022 learned_addr learned my addr 172.21.15.16:0/2764196022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f6039df2640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f60200a33b0 0x7f60200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f602a7fc640 1 -- 172.21.15.16:0/1963637834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f60240cf340 con 0x7f6034102570 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f6038df0640 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a3fd40 0x559249a42130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 -- 172.21.15.16:0/2764196022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 msgr2=0x559249a34f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a34f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 -- 172.21.15.16:0/2764196022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a3fd40 msgr2=0x559249a42130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a3fd40 0x559249a42130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 -- 172.21.15.16:0/2764196022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924994e1f0 con 0x559249a35480 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f6039df2640 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a34f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f60395f1640 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a35480 0x559249a3f800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6030009900 tx=0x7f6030004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cfb5a4b0017939 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.685+0000 7f6039df2640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f60200a33b0 0x7f60200a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f602c005fd0 tx=0x7f602c004ab0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f601f7fe640 1 -- 172.21.15.16:0/2764196022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030035030 con 0x559249a35480 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f601f7fe640 1 -- 172.21.15.16:0/2764196022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030040070 con 0x559249a35480 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f601f7fe640 1 -- 172.21.15.16:0/2764196022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030031ab0 con 0x559249a35480 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 -- 172.21.15.16:0/2764196022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a35480 msgr2=0x559249a3f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a35480 0x559249a3f800 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6030009900 tx=0x7f6030004170 comp rx=0 tx=0).stop 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 -- 172.21.15.16:0/2764196022 shutdown_connections 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a3fd40 0x559249a42130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a35480 0x559249a3f800 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 --2- 172.21.15.16:0/2764196022 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a34f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 -- 172.21.15.16:0/2764196022 >> 172.21.15.16:0/2764196022 conn(0x559249a310f0 msgr2=0x559249a314d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 -- 172.21.15.16:0/2764196022 shutdown_connections 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.686+0000 7f603dc2e740 1 -- 172.21.15.16:0/2764196022 wait complete. 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 -- start start 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 Processor -- start 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a3cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a3d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559249a42910 con 0x559249a3fd40 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559249954f10 con 0x559249a35480 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559249a42a90 con 0x559249a34b60 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f603dc2e740 10 client.0 _ll_register_callbacks cb 0x55924991da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f6039df2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a3d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f6039df2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a3d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60648/0 (socket says 172.21.15.16:60648) 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f6039df2640 1 -- 172.21.15.16:0/504797493 learned_addr learned my addr 172.21.15.16:0/504797493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.687+0000 7f60395f1640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a3cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6038df0640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6039df2640 1 -- 172.21.15.16:0/504797493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 msgr2=0x559249a3cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6039df2640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a3cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6039df2640 1 -- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 msgr2=0x559249a3d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6039df2640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a3d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6039df2640 1 -- 172.21.15.16:0/504797493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924970b6a0 con 0x559249a3fd40 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f60395f1640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a3cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6038df0640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a3d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f6039df2640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a3d620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f602c012b90 tx=0x7f602c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ba202792433075 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f601cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f601cff9640 1 -- 172.21.15.16:0/504797493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c024070 con 0x559249a3fd40 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f601cff9640 1 -- 172.21.15.16:0/504797493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f602c028400 con 0x559249a3fd40 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6030040070 con 0x559249a3fd40 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f601cff9640 1 -- 172.21.15.16:0/504797493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f602c01a070 con 0x559249a3fd40 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f603dc2e740 10 client.16011 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.688+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559249a4e500 con 0x559249a3fd40 2024-09-15T22:40:56.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f601cff9640 1 -- 172.21.15.16:0/504797493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030040070 con 0x559249a3fd40 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f601cff9640 1 -- 172.21.15.16:0/504797493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f602c00f040 con 0x559249a3fd40 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f603dc2e740 10 client.16011 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f603dc2e740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f603dc2e740 20 client.16011 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f603dc2e740 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249a4f9c0 0x559249a51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f603dc2e740 4 client.16011 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592499a77f0 con 0x559249a4f9c0 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.689+0000 7f60395f1640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249a4f9c0 0x559249a51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f60395f1640 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249a4f9c0 0x559249a51e10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab377334de1957e server_cookie=82755466401ce8d8 in_seq=0 out_seq=0 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f601cff9640 10 client.16011 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f601cff9640 1 -- 172.21.15.16:0/504797493 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592499a77f0 con 0x559249a4f9c0 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f601cff9640 10 client.16011 handle_command_reply: tid=0 2024-09-15T22:40:56.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 client.16011 shutdown 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 msgr2=0x559249a3d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a3d620 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f602c012b90 tx=0x7f602c004a40 comp rx=0 tx=0).stop 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 shutdown_connections 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249a4f9c0 0x559249a51e10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a3d620 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a3d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 --2- 172.21.15.16:0/504797493 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a34b60 0x559249a3cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f601cff9640 0 client.16011 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.690+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 >> 172.21.15.16:0/504797493 conn(0x559249a310f0 msgr2=0x559249a30270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.691+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 shutdown_connections 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.691+0000 7f603dc2e740 1 -- 172.21.15.16:0/504797493 wait complete. 2024-09-15T22:40:56.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.691+0000 7f603dc2e740 20 client.16011 trim_cache size 0 max 16384 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 Processor -- start 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 -- start start 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a34b60 0x559249a576e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a57c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a3fd40 0x559249a365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60341b18b0 con 0x559249a34b60 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60341af180 con 0x559249a35480 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f603dc2e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f60341acfd0 con 0x559249a3fd40 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f60395f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a34b60 0x559249a576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f60395f1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a34b60 0x559249a576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60654/0 (socket says 172.21.15.16:60654) 2024-09-15T22:40:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f60395f1640 1 -- 172.21.15.16:0/1429354642 learned_addr learned my addr 172.21.15.16:0/1429354642 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f6038df0640 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a57c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f60395f1640 1 -- 172.21.15.16:0/1429354642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a3fd40 msgr2=0x559249a365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f60395f1640 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a3fd40 0x559249a365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f60395f1640 1 -- 172.21.15.16:0/1429354642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 msgr2=0x559249a57c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.708+0000 7f60395f1640 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a57c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f60395f1640 1 -- 172.21.15.16:0/1429354642 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559249a348e0 con 0x559249a34b60 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f6038df0640 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a57c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f60395f1640 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a34b60 0x559249a576e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x559249a31f50 tx=0x7f603003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddaf2656b54c496 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:56.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f601effd640 1 -- 172.21.15.16:0/1429354642 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030055070 con 0x559249a34b60 2024-09-15T22:40:56.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f601effd640 1 -- 172.21.15.16:0/1429354642 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030040070 con 0x559249a34b60 2024-09-15T22:40:56.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f601effd640 1 -- 172.21.15.16:0/1429354642 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030035040 con 0x559249a34b60 2024-09-15T22:40:56.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f603dc2e740 1 -- 172.21.15.16:0/1429354642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a34b60 msgr2=0x559249a576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.709+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a34b60 0x559249a576e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x559249a31f50 tx=0x7f603003e040 comp rx=0 tx=0).stop 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- 172.21.15.16:0/1429354642 shutdown_connections 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a3fd40 0x559249a365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a35480 0x559249a57c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1429354642 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a34b60 0x559249a576e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- 172.21.15.16:0/1429354642 >> 172.21.15.16:0/1429354642 conn(0x559249a310f0 msgr2=0x559249a3bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- 172.21.15.16:0/1429354642 shutdown_connections 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- 172.21.15.16:0/1429354642 wait complete. 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- start start 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 Processor -- start 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a34b60 0x559249a43bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a35480 0x559249a44110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a44650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f60341ad640 con 0x559249a3fd40 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f60341aeab0 con 0x559249a34b60 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f60341add10 con 0x559249a35480 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.710+0000 7f603dc2e740 10 client.0 _ll_register_callbacks cb 0x7f602c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a44650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a44650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60670/0 (socket says 172.21.15.16:60670) 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 -- 172.21.15.16:0/1095515414 learned_addr learned my addr 172.21.15.16:0/1095515414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f60395f1640 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a34b60 0x559249a43bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6038df0640 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a35480 0x559249a44110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 -- 172.21.15.16:0/1095515414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a35480 msgr2=0x559249a44110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a35480 0x559249a44110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 -- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a34b60 msgr2=0x559249a43bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:56.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a34b60 0x559249a43bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 -- 172.21.15.16:0/1095515414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559249ab9850 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f6039df2640 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a44650 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f602c009c60 tx=0x7f602c011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2312e40e4e29bd52 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f601cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f601cff9640 1 -- 172.21.15.16:0/1095515414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c024070 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6030040070 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.711+0000 7f601cff9640 1 -- 172.21.15.16:0/1095515414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f602c014030 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f601cff9640 1 -- 172.21.15.16:0/1095515414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f602c009840 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 10 client.16017 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559249abaae0 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f601cff9640 1 -- 172.21.15.16:0/1095515414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030040070 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f601cff9640 1 -- 172.21.15.16:0/1095515414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f602c03d030 con 0x559249a3fd40 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 10 client.16017 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:56.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 20 client.16017 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:56.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249abb1f0 0x559249abd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:56.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 4 client.16017 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:40:56.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5592499a1d00 con 0x559249abb1f0 2024-09-15T22:40:56.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.712+0000 7f60395f1640 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249abb1f0 0x559249abd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:56.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.713+0000 7f60395f1640 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249abb1f0 0x559249abd610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76ade0fb48a308d server_cookie=b42922a1a09f6d2c in_seq=0 out_seq=0 2024-09-15T22:40:56.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.713+0000 7f601cff9640 10 client.16017 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f601cff9640 1 -- 172.21.15.16:0/1095515414 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5592499a1d00 con 0x559249abb1f0 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f601cff9640 10 client.16017 handle_command_reply: tid=0 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 client.16017 shutdown 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 msgr2=0x559249a44650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a44650 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f602c009c60 tx=0x7f602c011fd0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 shutdown_connections 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559249abb1f0 0x559249abd610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f601cff9640 0 client.16017 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559249a3fd40 0x559249a44650 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559249a35480 0x559249a44110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 --2- 172.21.15.16:0/1095515414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559249a34b60 0x559249a43bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.714+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 >> 172.21.15.16:0/1095515414 conn(0x559249a310f0 msgr2=0x559249a2fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 shutdown_connections 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603dc2e740 1 -- 172.21.15.16:0/1095515414 wait complete. 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603dc2e740 20 client.16017 trim_cache size 0 max 16384 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "d37bc892-ad9b-4427-a6b0-0980736c3d59", 2024-09-15T22:40:56.720 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:40:56.721 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f60200a33b0 msgr2=0x7f60200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f60200a33b0 0x7f60200a5870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f602c005fd0 tx=0x7f602c004ab0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102570 msgr2=0x7f6034198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102570 0x7f6034198b00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f602400b4a0 tx=0x7f602400b960 comp rx=0 tx=0).stop 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 shutdown_connections 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6034102ef0 0x7f603419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6034102570 0x7f6034198b00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f60200a33b0 0x7f60200a5870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 --2- 172.21.15.16:0/1963637834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6034100e20 0x7f60341985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 >> 172.21.15.16:0/1963637834 conn(0x7f60340fc7d0 msgr2=0x7f603406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 shutdown_connections 2024-09-15T22:40:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:56.715+0000 7f603b855640 1 -- 172.21.15.16:0/1963637834 wait complete. 2024-09-15T22:40:56.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:56 smithi059 ceph-mon[34872]: pgmap v78: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:57.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:56 smithi130 ceph-mon[35742]: pgmap v78: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:57.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:56 smithi016 ceph-mon[31492]: pgmap v78: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:40:57.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:40:57.696 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:57.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:57 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:40:57.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:57 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:57.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:57 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:40:57.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:57 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:57.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:57 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:40:57.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:57 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:57.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:57 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:40:57.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:57 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:58.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:57 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:40:58.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:57 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:58.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:57 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:40:58.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:57 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:40:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 -- 172.21.15.16:0/183400782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58102570 msgr2=0x7f1c581029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/183400782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58102570 0x7f1c581029b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009900 tx=0x7f1c4802a480 comp rx=0 tx=0).stop 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 -- 172.21.15.16:0/183400782 shutdown_connections 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/183400782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c58102ef0 0x7f1c5810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/183400782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58102570 0x7f1c581029b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/183400782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c58100e20 0x7f1c58101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 -- 172.21.15.16:0/183400782 >> 172.21.15.16:0/183400782 conn(0x7f1c580fc7d0 msgr2=0x7f1c580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 -- 172.21.15.16:0/183400782 shutdown_connections 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 -- 172.21.15.16:0/183400782 wait complete. 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 Processor -- start 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.122+0000 7f1c5f13d640 1 -- start start 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5f13d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58100e20 0x7f1c5819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5f13d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c58102570 0x7f1c5819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5f13d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c58102ef0 0x7f1c5819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5f13d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58110f80 con 0x7f1c58100e20 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5f13d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58110e00 con 0x7f1c58102570 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5f13d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c58111100 con 0x7f1c58102ef0 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58100e20 0x7f1c5819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58100e20 0x7f1c5819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60700/0 (socket says 172.21.15.16:60700) 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 -- 172.21.15.16:0/420701590 learned_addr learned my addr 172.21.15.16:0/420701590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c58102ef0 0x7f1c5819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c58102570 0x7f1c5819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 -- 172.21.15.16:0/420701590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c58102ef0 msgr2=0x7f1c5819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c58102ef0 0x7f1c5819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 -- 172.21.15.16:0/420701590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c58102570 msgr2=0x7f1c5819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c58102570 0x7f1c5819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 -- 172.21.15.16:0/420701590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5819f660 con 0x7f1c58100e20 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c58102ef0 0x7f1c5819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58100e20 0x7f1c5819a6b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c4dffb640 1 -- 172.21.15.16:0/420701590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54017070 con 0x7f1c58100e20 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c4dffb640 1 -- 172.21.15.16:0/420701590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c54009c30 con 0x7f1c58100e20 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.123+0000 7f1c4dffb640 1 -- 172.21.15.16:0/420701590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400f6b0 con 0x7f1c58100e20 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.124+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5819f950 con 0x7f1c58100e20 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.124+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c581a7230 con 0x7f1c58100e20 2024-09-15T22:40:58.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.126+0000 7f1c4dffb640 1 -- 172.21.15.16:0/420701590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1c540109d0 con 0x7f1c58100e20 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.126+0000 7f1c61516740 1 Processor -- start 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.126+0000 7f1c61516740 1 -- start start 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c61516740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13e99880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c61516740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c61516740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c61516740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eb13dff2e0 con 0x55eb13e99dc0 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c61516740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb13da6ce0 con 0x55eb13ea4680 2024-09-15T22:40:58.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c61516740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55eb13dfb2a0 con 0x55eb13e994a0 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c4dffb640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c400a33b0 0x7f1c400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5d6da640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13e99880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60712/0 (socket says 172.21.15.16:60712) 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 -- 172.21.15.16:0/729521704 learned_addr learned my addr 172.21.15.16:0/729521704 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c400a33b0 0x7f1c400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c4dffb640 1 -- 172.21.15.16:0/420701590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1c540cfdc0 con 0x7f1c58100e20 2024-09-15T22:40:58.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 -- 172.21.15.16:0/729521704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 msgr2=0x55eb13e99880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13e99880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 -- 172.21.15.16:0/729521704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 msgr2=0x55eb13ea6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 -- 172.21.15.16:0/729521704 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb13d8b440 con 0x55eb13e99dc0 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.127+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea4140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c5409ba40 tx=0x7f1c5400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4c419cfd9eae16 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c400a33b0 0x7f1c400a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000afd0 tx=0x7f1c50005e80 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c3effd640 1 -- 172.21.15.16:0/729521704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54020070 con 0x55eb13e99dc0 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c3effd640 1 -- 172.21.15.16:0/729521704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c5409c440 con 0x55eb13e99dc0 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c3effd640 1 -- 172.21.15.16:0/729521704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c540cd430 con 0x55eb13e99dc0 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 -- 172.21.15.16:0/729521704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 msgr2=0x55eb13ea4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea4140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1c5409ba40 tx=0x7f1c5400bf60 comp rx=0 tx=0).stop 2024-09-15T22:40:58.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 -- 172.21.15.16:0/729521704 shutdown_connections 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea4140 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 --2- 172.21.15.16:0/729521704 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13e99880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 -- 172.21.15.16:0/729521704 >> 172.21.15.16:0/729521704 conn(0x55eb13e95a30 msgr2=0x55eb13e95e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 -- 172.21.15.16:0/729521704 shutdown_connections 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.128+0000 7f1c61516740 1 -- 172.21.15.16:0/729521704 wait complete. 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 -- start start 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 Processor -- start 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55eb13ea71f0 con 0x55eb13e99dc0 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55eb13dba050 con 0x55eb13ea4680 2024-09-15T22:40:58.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55eb13ea7370 con 0x55eb13e994a0 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c61516740 10 client.0 _ll_register_callbacks cb 0x55eb13dfa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60714/0 (socket says 172.21.15.16:60714) 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.129+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3035590372 learned_addr learned my addr 172.21.15.16:0/3035590372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3035590372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 msgr2=0x55eb13ea3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 msgr2=0x55eb13ea17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3035590372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb13b703c0 con 0x55eb13e99dc0 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea1260 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1c48005ab0 tx=0x7f1c480052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27beeefe31a5396 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c37fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c37fff640 1 -- 172.21.15.16:0/3035590372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48035030 con 0x55eb13e99dc0 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c5409c440 con 0x55eb13e99dc0 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c37fff640 1 -- 172.21.15.16:0/3035590372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1c480048c0 con 0x55eb13e99dc0 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c37fff640 1 -- 172.21.15.16:0/3035590372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c4804c050 con 0x55eb13e99dc0 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c61516740 10 client.16029 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.130+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb13eb2ba0 con 0x55eb13e99dc0 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c37fff640 1 -- 172.21.15.16:0/3035590372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c5409c440 con 0x55eb13e99dc0 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c37fff640 1 -- 172.21.15.16:0/3035590372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1c48003a60 con 0x55eb13e99dc0 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c61516740 10 client.16029 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c61516740 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c61516740 20 client.16029 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c61516740 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13eb4060 0x55eb13eb64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c61516740 4 client.16029 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb13e0c850 con 0x55eb13eb4060 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.131+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13eb4060 0x55eb13eb64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13eb4060 0x55eb13eb64a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8ea20a88279d19 server_cookie=7695705b3fb30ed5 in_seq=0 out_seq=0 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c37fff640 10 client.16029 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c37fff640 1 -- 172.21.15.16:0/3035590372 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb13e0c850 con 0x55eb13eb4060 2024-09-15T22:40:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c37fff640 10 client.16029 handle_command_reply: tid=0 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 client.16029 shutdown 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 msgr2=0x55eb13ea1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea1260 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1c48005ab0 tx=0x7f1c480052d0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 shutdown_connections 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13eb4060 0x55eb13eb64a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13ea17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea1260 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.133+0000 7f1c37fff640 0 client.16029 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.132+0000 7f1c61516740 1 --2- 172.21.15.16:0/3035590372 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.133+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 >> 172.21.15.16:0/3035590372 conn(0x55eb13e95a30 msgr2=0x55eb13e9b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.133+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 shutdown_connections 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.133+0000 7f1c61516740 1 -- 172.21.15.16:0/3035590372 wait complete. 2024-09-15T22:40:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.133+0000 7f1c61516740 20 client.16029 trim_cache size 0 max 16384 2024-09-15T22:40:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 Processor -- start 2024-09-15T22:40:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 -- start start 2024-09-15T22:40:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e994a0 0x55eb13eb0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13e99dc0 0x55eb13ebbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13ea4680 0x55eb13ebc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581b39b0 con 0x55eb13e994a0 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581b1280 con 0x55eb13e99dc0 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c61516740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581af0d0 con 0x55eb13ea4680 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c5ced9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e994a0 0x55eb13eb0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c5ced9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e994a0 0x55eb13eb0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60722/0 (socket says 172.21.15.16:60722) 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.150+0000 7f1c5ced9640 1 -- 172.21.15.16:0/1140630681 learned_addr learned my addr 172.21.15.16:0/1140630681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13ea4680 0x55eb13ebc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13e99dc0 0x55eb13ebbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c5ced9640 1 -- 172.21.15.16:0/1140630681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13ea4680 msgr2=0x55eb13ebc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13ea4680 0x55eb13ebc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c5ced9640 1 -- 172.21.15.16:0/1140630681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13e99dc0 msgr2=0x55eb13ebbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13e99dc0 0x55eb13ebbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c5ced9640 1 -- 172.21.15.16:0/1140630681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb13ea0210 con 0x55eb13e994a0 2024-09-15T22:40:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e994a0 0x55eb13eb0450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1c540cd7b0 tx=0x7f1c54086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3474cedeefff02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/1140630681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54014070 con 0x55eb13e994a0 2024-09-15T22:40:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/1140630681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c54009de0 con 0x55eb13e994a0 2024-09-15T22:40:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.151+0000 7f1c3e7fc640 1 -- 172.21.15.16:0/1140630681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54020040 con 0x55eb13e994a0 2024-09-15T22:40:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- 172.21.15.16:0/1140630681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e994a0 msgr2=0x55eb13eb0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e994a0 0x55eb13eb0450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1c540cd7b0 tx=0x7f1c54086040 comp rx=0 tx=0).stop 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- 172.21.15.16:0/1140630681 shutdown_connections 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13ea4680 0x55eb13ebc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13e99dc0 0x55eb13ebbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 --2- 172.21.15.16:0/1140630681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e994a0 0x55eb13eb0450 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- 172.21.15.16:0/1140630681 >> 172.21.15.16:0/1140630681 conn(0x55eb13e95a30 msgr2=0x55eb13ea32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- 172.21.15.16:0/1140630681 shutdown_connections 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- 172.21.15.16:0/1140630681 wait complete. 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- start start 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 Processor -- start 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13f1dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581af740 con 0x55eb13e99dc0 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581b0bb0 con 0x55eb13ea4680 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.152+0000 7f1c61516740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c581afe10 con 0x55eb13e994a0 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c61516740 10 client.0 _ll_register_callbacks cb 0x7f1c48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60732/0 (socket says 172.21.15.16:60732) 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3626913556 learned_addr learned my addr 172.21.15.16:0/3626913556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13f1dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3626913556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 msgr2=0x55eb13ea8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 msgr2=0x55eb13f1dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13f1dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 -- 172.21.15.16:0/3626913556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb13ea4320 con 0x55eb13e99dc0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c5d6da640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13f1dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c4ffff640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea8c60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1c48006010 tx=0x7f1c48004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9996d90516aab388 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c37fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c37fff640 1 -- 172.21.15.16:0/3626913556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48035070 con 0x55eb13e99dc0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c37fff640 1 -- 172.21.15.16:0/3626913556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1c48031a30 con 0x55eb13e99dc0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.153+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c54009de0 con 0x55eb13e99dc0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c37fff640 1 -- 172.21.15.16:0/3626913556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c4805e050 con 0x55eb13e99dc0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 10 client.16035 fetch_fsmap learned FSMap version 26 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb13f1f210 con 0x55eb13e99dc0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c37fff640 1 -- 172.21.15.16:0/3626913556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c54009de0 con 0x55eb13e99dc0 2024-09-15T22:40:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c37fff640 1 -- 172.21.15.16:0/3626913556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1c48031cc0 con 0x55eb13e99dc0 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 20 client.16035 populate_metadata read hostname 'smithi016' 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13f1f9e0 0x55eb13f21e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 4 client.16035 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb13e06d60 con 0x55eb13f1f9e0 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.154+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13f1f9e0 0x55eb13f21e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c5ced9640 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13f1f9e0 0x55eb13f21e00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0dc6a826a41820 server_cookie=98f7a5b8f165faed in_seq=0 out_seq=0 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c37fff640 10 client.16035 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c37fff640 1 -- 172.21.15.16:0/3626913556 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eb13e06d60 con 0x55eb13f1f9e0 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c37fff640 10 client.16035 handle_command_reply: tid=0 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 client.16035 shutdown 2024-09-15T22:40:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 msgr2=0x55eb13ea8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea8c60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1c48006010 tx=0x7f1c48004610 comp rx=0 tx=0).stop 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 shutdown_connections 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55eb13f1f9e0 0x55eb13f21e00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55eb13ea4680 0x55eb13f1dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55eb13e99dc0 0x55eb13ea8c60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 --2- 172.21.15.16:0/3626913556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55eb13e994a0 0x55eb13ea8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 >> 172.21.15.16:0/3626913556 conn(0x55eb13e95a30 msgr2=0x55eb13e9aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c37fff640 0 client.16035 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 shutdown_connections 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 1 -- 172.21.15.16:0/3626913556 wait complete. 2024-09-15T22:40:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.155+0000 7f1c61516740 20 client.16035 trim_cache size 0 max 16384 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: "d37bc892-ad9b-4427-a6b0-0980736c3d59": { 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "d37bc892-ad9b-4427-a6b0-0980736c3d59", 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:40:58.163 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c400a33b0 msgr2=0x7f1c400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c400a33b0 0x7f1c400a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000afd0 tx=0x7f1c50005e80 comp rx=0 tx=0).stop 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58100e20 msgr2=0x7f1c5819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58100e20 0x7f1c5819a6b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).stop 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 shutdown_connections 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c400a33b0 0x7f1c400a5870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c58102ef0 0x7f1c5819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c58102570 0x7f1c5819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 --2- 172.21.15.16:0/420701590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c58100e20 0x7f1c5819a6b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 >> 172.21.15.16:0/420701590 conn(0x7f1c580fc7d0 msgr2=0x7f1c5806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 shutdown_connections 2024-09-15T22:40:58.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:40:58.156+0000 7f1c5f13d640 1 -- 172.21.15.16:0/420701590 wait complete. 2024-09-15T22:40:58.794 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:58 smithi016 ceph-mon[31492]: pgmap v79: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:58.794 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:40:58 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:40:58.823 INFO:tasks.cephfs.filesystem:scrub status for tag:d37bc892-ad9b-4427-a6b0-0980736c3d59 - {'path': '/', 'tag': 'd37bc892-ad9b-4427-a6b0-0980736c3d59', 'options': 'recursive,force'} 2024-09-15T22:40:58.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:58 smithi059 ceph-mon[34872]: pgmap v79: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:58.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:40:58 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:40:59.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:58 smithi130 ceph-mon[35742]: pgmap v79: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-09-15T22:40:59.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:40:58 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:41:00.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:00 smithi059 ceph-mon[34872]: pgmap v80: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:41:01.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:00 smithi130 ceph-mon[35742]: pgmap v80: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:41:01.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:00 smithi016 ceph-mon[31492]: pgmap v80: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:41:02.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:02 smithi059 ceph-mon[34872]: pgmap v81: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-15T22:41:03.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:02 smithi130 ceph-mon[35742]: pgmap v81: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-15T22:41:03.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:02 smithi016 ceph-mon[31492]: pgmap v81: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-15T22:41:04.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:04 smithi059 ceph-mon[34872]: pgmap v82: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:41:05.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:04 smithi130 ceph-mon[35742]: pgmap v82: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:41:05.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:04 smithi016 ceph-mon[31492]: pgmap v82: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:41:06.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:06 smithi059 ceph-mon[34872]: pgmap v83: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:41:07.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:06 smithi130 ceph-mon[35742]: pgmap v83: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:41:07.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:06 smithi016 ceph-mon[31492]: pgmap v83: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:41:08.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:08 smithi059 ceph-mon[34872]: pgmap v84: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-15T22:41:09.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:08 smithi130 ceph-mon[35742]: pgmap v84: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-15T22:41:09.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:08 smithi016 ceph-mon[31492]: pgmap v84: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-15T22:41:11.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:10 smithi130 ceph-mon[35742]: pgmap v85: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-15T22:41:11.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:10 smithi016 ceph-mon[31492]: pgmap v85: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-15T22:41:11.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:10 smithi059 ceph-mon[34872]: pgmap v85: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-15T22:41:13.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:12 smithi130 ceph-mon[35742]: pgmap v86: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:41:13.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:12 smithi016 ceph-mon[31492]: pgmap v86: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:41:13.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:12 smithi059 ceph-mon[34872]: pgmap v86: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-15T22:41:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:14 smithi130 ceph-mon[35742]: pgmap v87: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:41:15.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:14 smithi016 ceph-mon[31492]: pgmap v87: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:41:15.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:14 smithi059 ceph-mon[34872]: pgmap v87: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:41:17.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:16 smithi130 ceph-mon[35742]: pgmap v88: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:41:17.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:16 smithi016 ceph-mon[31492]: pgmap v88: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:41:17.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:16 smithi059 ceph-mon[34872]: pgmap v88: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:41:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:18 smithi130 ceph-mon[35742]: pgmap v89: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-15T22:41:19.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:18 smithi016 ceph-mon[31492]: pgmap v89: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-15T22:41:19.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:18 smithi059 ceph-mon[34872]: pgmap v89: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-15T22:41:21.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:20 smithi130 ceph-mon[35742]: pgmap v90: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:41:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:20 smithi016 ceph-mon[31492]: pgmap v90: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:41:21.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:20 smithi059 ceph-mon[34872]: pgmap v90: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:41:23.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:22 smithi130 ceph-mon[35742]: pgmap v91: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:41:23.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:22 smithi016 ceph-mon[31492]: pgmap v91: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:41:23.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:22 smithi059 ceph-mon[34872]: pgmap v91: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:41:25.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:24 smithi130 ceph-mon[35742]: pgmap v92: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:25.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:24 smithi016 ceph-mon[31492]: pgmap v92: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:25.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:24 smithi059 ceph-mon[34872]: pgmap v92: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:26.996 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:26 smithi059 ceph-mon[34872]: pgmap v93: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:27.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:26 smithi130 ceph-mon[35742]: pgmap v93: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:27.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:26 smithi016 ceph-mon[31492]: pgmap v93: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:28.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:41:29.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:28 smithi130 ceph-mon[35742]: pgmap v94: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:29.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:28 smithi016 ceph-mon[31492]: pgmap v94: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:29.098 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:29.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:28 smithi059 ceph-mon[34872]: pgmap v94: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 -- 172.21.15.16:0/2844090068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514102d40 msgr2=0x7f2514103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 --2- 172.21.15.16:0/2844090068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514102d40 0x7f2514103120 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2510009ca0 tx=0x7f251002a950 comp rx=0 tx=0).stop 2024-09-15T22:41:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 -- 172.21.15.16:0/2844090068 shutdown_connections 2024-09-15T22:41:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 --2- 172.21.15.16:0/2844090068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f251410c170 0x7f251410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 --2- 172.21.15.16:0/2844090068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514103660 0x7f251410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 --2- 172.21.15.16:0/2844090068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514102d40 0x7f2514103120 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 -- 172.21.15.16:0/2844090068 >> 172.21.15.16:0/2844090068 conn(0x7f25140fc7a0 msgr2=0x7f25140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 -- 172.21.15.16:0/2844090068 shutdown_connections 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.570+0000 7f251c794640 1 -- 172.21.15.16:0/2844090068 wait complete. 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 Processor -- start 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 -- start start 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2514102d40 0x7f251419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514103660 0x7f251419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251410c170 0x7f251419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2514111f70 con 0x7f2514103660 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2514111df0 con 0x7f251410c170 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251c794640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f25141120f0 con 0x7f2514102d40 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f2519d2f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514103660 0x7f251419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f2519d2f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514103660 0x7f251419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52120/0 (socket says 172.21.15.16:52120) 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f2519d2f640 1 -- 172.21.15.16:0/1329955726 learned_addr learned my addr 172.21.15.16:0/1329955726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.571+0000 7f251ad31640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251410c170 0x7f251419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f251a530640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2514102d40 0x7f251419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f2519d2f640 1 -- 172.21.15.16:0/1329955726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2514102d40 msgr2=0x7f251419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f2519d2f640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2514102d40 0x7f251419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f2519d2f640 1 -- 172.21.15.16:0/1329955726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251410c170 msgr2=0x7f251419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f2519d2f640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251410c170 0x7f251419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f2519d2f640 1 -- 172.21.15.16:0/1329955726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251419f600 con 0x7f2514103660 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f251a530640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2514102d40 0x7f251419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f2519d2f640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514103660 0x7f251419ab90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f250800d740 tx=0x7f250800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f25077fe640 1 -- 172.21.15.16:0/1329955726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508017070 con 0x7f2514103660 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f25077fe640 1 -- 172.21.15.16:0/1329955726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250800de50 con 0x7f2514103660 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251419f8f0 con 0x7f2514103660 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f25077fe640 1 -- 172.21.15.16:0/1329955726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250800f620 con 0x7f2514103660 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.572+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141a71d0 con 0x7f2514103660 2024-09-15T22:41:29.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.574+0000 7f25077fe640 1 -- 172.21.15.16:0/1329955726 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2508009c30 con 0x7f2514103660 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 Processor -- start 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 -- start start 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cb4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8c1a2e0 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8bc1ce0 con 0x55d3f8cb44a0 2024-09-15T22:41:29.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251eb6d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8c162a0 con 0x55d3f8cbf680 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f25077fe640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f25000a33b0 0x7f25000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251a530640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251a530640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52130/0 (socket says 172.21.15.16:52130) 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.575+0000 7f251a530640 1 -- 172.21.15.16:0/3566091031 learned_addr learned my addr 172.21.15.16:0/3566091031 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251ad31640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f25000a33b0 0x7f25000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f2519d2f640 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251ad31640 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cb4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f25077fe640 1 -- 172.21.15.16:0/1329955726 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f25080cfce0 con 0x7f2514103660 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251a530640 1 -- 172.21.15.16:0/3566091031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 msgr2=0x55d3f8cc1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251a530640 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251a530640 1 -- 172.21.15.16:0/3566091031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 msgr2=0x55d3f8cb4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251a530640 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cb4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251a530640 1 -- 172.21.15.16:0/3566091031 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f8ba6440 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f2519d2f640 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251ad31640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f25000a33b0 0x7f25000a5870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f250c0127d0 tx=0x7f250c00b9e0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:29.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f251a530640 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbf140 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2510009930 tx=0x7f25100093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24923f4168d984f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f2504ff9640 1 -- 172.21.15.16:0/3566091031 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510034040 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f2504ff9640 1 -- 172.21.15.16:0/3566091031 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251002fbc0 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.576+0000 7f2504ff9640 1 -- 172.21.15.16:0/3566091031 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25100044c0 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 -- 172.21.15.16:0/3566091031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb4dc0 msgr2=0x55d3f8cbf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbf140 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2510009930 tx=0x7f25100093f0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 -- 172.21.15.16:0/3566091031 shutdown_connections 2024-09-15T22:41:29.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbf140 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 --2- 172.21.15.16:0/3566091031 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cb4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 -- 172.21.15.16:0/3566091031 >> 172.21.15.16:0/3566091031 conn(0x55d3f8cb0a30 msgr2=0x55d3f8cb0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 -- 172.21.15.16:0/3566091031 shutdown_connections 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.577+0000 7f251eb6d740 1 -- 172.21.15.16:0/3566091031 wait complete. 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 -- start start 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 Processor -- start 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cbc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8cc2250 con 0x55d3f8cbf680 2024-09-15T22:41:29.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8bd5050 con 0x55d3f8cb44a0 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d3f8cc23d0 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251eb6d740 10 client.0 _ll_register_callbacks cb 0x55d3f8c15160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cbc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cbc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52132/0 (socket says 172.21.15.16:52132) 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 -- 172.21.15.16:0/405353137 learned_addr learned my addr 172.21.15.16:0/405353137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251a530640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f2519d2f640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 -- 172.21.15.16:0/405353137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 msgr2=0x55d3f8cbc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 -- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 msgr2=0x55d3f8cbef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251ad31640 1 -- 172.21.15.16:0/405353137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f898b3c0 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.578+0000 7f251a530640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f251ad31640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cbc760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f250c004360 tx=0x7f250c00b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8d34d578db4699 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f24fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f24fdffb640 1 -- 172.21.15.16:0/405353137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c024070 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f24fdffb640 1 -- 172.21.15.16:0/405353137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f250c010e00 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f251002fbc0 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f24fdffb640 1 -- 172.21.15.16:0/405353137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f250c032050 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f251eb6d740 10 client.16047 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3f8ccdbc0 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f24fdffb640 1 -- 172.21.15.16:0/405353137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f251002fbc0 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f24fdffb640 1 -- 172.21.15.16:0/405353137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f250c01f070 con 0x55d3f8cbf680 2024-09-15T22:41:29.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f251eb6d740 10 client.16047 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.579+0000 7f251eb6d740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.580+0000 7f251eb6d740 20 client.16047 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.580+0000 7f251eb6d740 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8ccf080 0x55d3f8cd14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.580+0000 7f251eb6d740 4 client.16047 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.580+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3f8c27850 con 0x55d3f8ccf080 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.580+0000 7f251a530640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8ccf080 0x55d3f8cd14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.580+0000 7f251a530640 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8ccf080 0x55d3f8cd14f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ab5571924122b4 server_cookie=2a9adfb4913eb6ad in_seq=0 out_seq=0 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.580+0000 7f24fdffb640 10 client.16047 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f24fdffb640 1 -- 172.21.15.16:0/405353137 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3f8c27850 con 0x55d3f8ccf080 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f24fdffb640 10 client.16047 handle_command_reply: tid=0 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 client.16047 shutdown 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 msgr2=0x55d3f8cbc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cbc760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f250c004360 tx=0x7f250c00b2e0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 shutdown_connections 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8ccf080 0x55d3f8cd14f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cbc760 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cbc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 --2- 172.21.15.16:0/405353137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f24fdffb640 0 client.16047 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 >> 172.21.15.16:0/405353137 conn(0x55d3f8cb0a30 msgr2=0x55d3f8cb6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 shutdown_connections 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 1 -- 172.21.15.16:0/405353137 wait complete. 2024-09-15T22:41:29.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.581+0000 7f251eb6d740 20 client.16047 trim_cache size 0 max 16384 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.598+0000 7f251eb6d740 1 Processor -- start 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.598+0000 7f251eb6d740 1 -- start start 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cd6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cd7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251eb6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b39c0 con 0x55d3f8cb44a0 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251eb6d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b1290 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251eb6d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f25141af0e0 con 0x55d3f8cbf680 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52148/0 (socket says 172.21.15.16:52148) 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 -- 172.21.15.16:0/1852812940 learned_addr learned my addr 172.21.15.16:0/1852812940 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251ad31640 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cd7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f2519d2f640 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cd6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 -- 172.21.15.16:0/1852812940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 msgr2=0x55d3f8cd7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cd7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 -- 172.21.15.16:0/1852812940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb4dc0 msgr2=0x55d3f8cd6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cd6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 -- 172.21.15.16:0/1852812940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f8cb0860 con 0x55d3f8cb44a0 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251ad31640 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cd7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.599+0000 7f251a530640 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbe190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2510009930 tx=0x7f251002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16e053b167e7dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f24fffff640 1 -- 172.21.15.16:0/1852812940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510034040 con 0x55d3f8cb44a0 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f24fffff640 1 -- 172.21.15.16:0/1852812940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2510009540 con 0x55d3f8cb44a0 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f24fffff640 1 -- 172.21.15.16:0/1852812940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510004450 con 0x55d3f8cb44a0 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 -- 172.21.15.16:0/1852812940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb44a0 msgr2=0x55d3f8cbe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbe190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2510009930 tx=0x7f251002a950 comp rx=0 tx=0).stop 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 -- 172.21.15.16:0/1852812940 shutdown_connections 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cbf680 0x55d3f8cd7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cd6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 --2- 172.21.15.16:0/1852812940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cbe190 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 -- 172.21.15.16:0/1852812940 >> 172.21.15.16:0/1852812940 conn(0x55d3f8cb0a30 msgr2=0x55d3f8cbda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:29.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 -- 172.21.15.16:0/1852812940 shutdown_connections 2024-09-15T22:41:29.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.600+0000 7f251eb6d740 1 -- 172.21.15.16:0/1852812940 wait complete. 2024-09-15T22:41:29.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 -- start start 2024-09-15T22:41:29.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 Processor -- start 2024-09-15T22:41:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cc7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cc51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f25141af750 con 0x55d3f8cbf680 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b0bc0 con 0x55d3f8cb44a0 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f25141afe20 con 0x55d3f8cb4dc0 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251eb6d740 10 client.0 _ll_register_callbacks cb 0x7f250c004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52152/0 (socket says 172.21.15.16:52152) 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 -- 172.21.15.16:0/443647604 learned_addr learned my addr 172.21.15.16:0/443647604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f2519d2f640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cc51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251a530640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cc7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 -- 172.21.15.16:0/443647604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 msgr2=0x55d3f8cc51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cc51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 -- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 msgr2=0x55d3f8cc7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cc7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.601+0000 7f251ad31640 1 -- 172.21.15.16:0/443647604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f8cc5f60 con 0x55d3f8cbf680 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251a530640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cc7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251ad31640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc5730 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55d3f8cb03a0 tx=0x7f250c010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85610486b35e76bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f24fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f24fdffb640 1 -- 172.21.15.16:0/443647604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c024040 con 0x55d3f8cbf680 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f24fdffb640 1 -- 172.21.15.16:0/443647604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f250c0039c0 con 0x55d3f8cbf680 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2510009540 con 0x55d3f8cbf680 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f24fdffb640 1 -- 172.21.15.16:0/443647604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f250c003bc0 con 0x55d3f8cbf680 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251eb6d740 10 client.16053 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3f8d39e50 con 0x55d3f8cbf680 2024-09-15T22:41:29.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f24fdffb640 1 -- 172.21.15.16:0/443647604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2510009540 con 0x55d3f8cbf680 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f24fdffb640 1 -- 172.21.15.16:0/443647604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f250c020730 con 0x55d3f8cbf680 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251eb6d740 10 client.16053 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251eb6d740 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251eb6d740 20 client.16053 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.602+0000 7f251eb6d740 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8d3a590 0x55d3f8d3c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f251eb6d740 4 client.16053 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3f8cbcaf0 con 0x55d3f8d3a590 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f251a530640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8d3a590 0x55d3f8d3c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f251a530640 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8d3a590 0x55d3f8d3c9b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39523eed27fa7740 server_cookie=46855d93a170d612 in_seq=0 out_seq=0 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f24fdffb640 10 client.16053 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f24fdffb640 1 -- 172.21.15.16:0/443647604 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d3f8cbcaf0 con 0x55d3f8d3a590 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f24fdffb640 10 client.16053 handle_command_reply: tid=0 2024-09-15T22:41:29.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f251eb6d740 1 client.16053 shutdown 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 msgr2=0x55d3f8cc5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.603+0000 7f251eb6d740 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc5730 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x55d3f8cb03a0 tx=0x7f250c010e00 comp rx=0 tx=0).stop 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 shutdown_connections 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d3f8d3a590 0x55d3f8d3c9b0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3f8cbf680 0x55d3f8cc5730 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3f8cb4dc0 0x55d3f8cc51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 --2- 172.21.15.16:0/443647604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3f8cb44a0 0x55d3f8cc7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f24fdffb640 0 client.16053 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 >> 172.21.15.16:0/443647604 conn(0x55d3f8cb0a30 msgr2=0x55d3f8cafb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 shutdown_connections 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 1 -- 172.21.15.16:0/443647604 wait complete. 2024-09-15T22:41:29.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251eb6d740 20 client.16053 trim_cache size 0 max 16384 2024-09-15T22:41:29.612 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:41:29.612 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:41:29.612 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:41:29.612 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:41:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f25000a33b0 msgr2=0x7f25000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f25000a33b0 0x7f25000a5870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f250c0127d0 tx=0x7f250c00b9e0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514103660 msgr2=0x7f251419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514103660 0x7f251419ab90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f250800d740 tx=0x7f250800dc00 comp rx=0 tx=0).stop 2024-09-15T22:41:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 shutdown_connections 2024-09-15T22:41:29.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f251410c170 0x7f251419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2514103660 0x7f251419ab90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f25000a33b0 0x7f25000a5870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 --2- 172.21.15.16:0/1329955726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2514102d40 0x7f251419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.604+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 >> 172.21.15.16:0/1329955726 conn(0x7f25140fc7a0 msgr2=0x7f25140fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.605+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 shutdown_connections 2024-09-15T22:41:29.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:29.605+0000 7f251c794640 1 -- 172.21.15.16:0/1329955726 wait complete. 2024-09-15T22:41:30.361 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:41:30.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:41:30.626 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:30.997 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:30 smithi016 ceph-mon[31492]: pgmap v95: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:31.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:30 smithi130 ceph-mon[35742]: pgmap v95: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 -- 172.21.15.16:0/91101247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38741068d0 msgr2=0x7f3874106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 --2- 172.21.15.16:0/91101247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38741068d0 0x7f3874106cb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3864009930 tx=0x7f386402a420 comp rx=0 tx=0).stop 2024-09-15T22:41:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 -- 172.21.15.16:0/91101247 shutdown_connections 2024-09-15T22:41:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 --2- 172.21.15.16:0/91101247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3874101270 0x7f387410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 --2- 172.21.15.16:0/91101247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38741008d0 0x7f3874100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 --2- 172.21.15.16:0/91101247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38741068d0 0x7f3874106cb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 -- 172.21.15.16:0/91101247 >> 172.21.15.16:0/91101247 conn(0x7f38740fc630 msgr2=0x7f38740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 -- 172.21.15.16:0/91101247 shutdown_connections 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.079+0000 7f387359e640 1 -- 172.21.15.16:0/91101247 wait complete. 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.080+0000 7f387359e640 1 Processor -- start 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.080+0000 7f387359e640 1 -- start start 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f387359e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38741008d0 0x7f38741982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f387359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3874101270 0x7f3874198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f387359e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38741068d0 0x7f387419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f387359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3874110cd0 con 0x7f3874101270 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f387359e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3874110b50 con 0x7f38741008d0 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f387359e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3874110e50 con 0x7f38741068d0 2024-09-15T22:41:31.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3874101270 0x7f3874198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3874101270 0x7f3874198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52172/0 (socket says 172.21.15.16:52172) 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 -- 172.21.15.16:0/2821316114 learned_addr learned my addr 172.21.15.16:0/2821316114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f387259c640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38741008d0 0x7f38741982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3872d9d640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38741068d0 0x7f387419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 -- 172.21.15.16:0/2821316114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38741068d0 msgr2=0x7f387419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38741068d0 0x7f387419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 -- 172.21.15.16:0/2821316114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38741008d0 msgr2=0x7f38741982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38741008d0 0x7f38741982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 -- 172.21.15.16:0/2821316114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387419d270 con 0x7f3874101270 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.081+0000 7f3871d9b640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3874101270 0x7f3874198800 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f385c00e9f0 tx=0x7f385c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.082+0000 7f386b7fe640 1 -- 172.21.15.16:0/2821316114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c00cd80 con 0x7f3874101270 2024-09-15T22:41:31.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.082+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387419d560 con 0x7f3874101270 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.082+0000 7f386b7fe640 1 -- 172.21.15.16:0/2821316114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f385c010910 con 0x7f3874101270 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.082+0000 7f386b7fe640 1 -- 172.21.15.16:0/2821316114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c018990 con 0x7f3874101270 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.082+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38741a4e40 con 0x7f3874101270 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.083+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3874102010 con 0x7f3874101270 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.084+0000 7f386b7fe640 1 -- 172.21.15.16:0/2821316114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f385c010ac0 con 0x7f3874101270 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.084+0000 7f386b7fe640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f38600a33b0 0x7f38600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.085+0000 7f387259c640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f38600a33b0 0x7f38600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.085+0000 7f386b7fe640 1 -- 172.21.15.16:0/2821316114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f385c0d0390 con 0x7f3874101270 2024-09-15T22:41:31.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.085+0000 7f387259c640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f38600a33b0 0x7f38600a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3864009900 tx=0x7f38640058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:31.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.089+0000 7f386b7fe640 1 -- 172.21.15.16:0/2821316114 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f385c09c3c0 con 0x7f3874101270 2024-09-15T22:41:31.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:30 smithi059 ceph-mon[34872]: pgmap v95: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.303+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38741a5010 con 0x7f3874101270 2024-09-15T22:41:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.304+0000 7f386b7fe640 1 -- 172.21.15.16:0/2821316114 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f385c0a06c0 con 0x7f3874101270 2024-09-15T22:41:31.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:41:31.306 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:41:31.306 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:41:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.307+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f38600a33b0 msgr2=0x7f38600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f38600a33b0 0x7f38600a5870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3864009900 tx=0x7f38640058d0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3874101270 msgr2=0x7f3874198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3874101270 0x7f3874198800 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f385c00e9f0 tx=0x7f385c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 shutdown_connections 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f38600a33b0 0x7f38600a5870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38741068d0 0x7f387419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3874101270 0x7f3874198800 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 --2- 172.21.15.16:0/2821316114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38741008d0 0x7f38741982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 >> 172.21.15.16:0/2821316114 conn(0x7f38740fc630 msgr2=0x7f387406a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 shutdown_connections 2024-09-15T22:41:31.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:31.308+0000 7f387359e640 1 -- 172.21.15.16:0/2821316114 wait complete. 2024-09-15T22:41:32.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:31 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2821316114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:41:32.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:31 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2821316114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:41:32.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:41:32.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:31 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2821316114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:41:32.438 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:32.822 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:32 smithi016 ceph-mon[31492]: pgmap v96: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 -- 172.21.15.16:0/1897288521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100e30 msgr2=0x7f1350104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 --2- 172.21.15.16:0/1897288521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100e30 0x7f1350104e80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1338009900 tx=0x7f133802a410 comp rx=0 tx=0).stop 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 -- 172.21.15.16:0/1897288521 shutdown_connections 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 --2- 172.21.15.16:0/1897288521 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13501055b0 0x7f135010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 --2- 172.21.15.16:0/1897288521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100e30 0x7f1350104e80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 --2- 172.21.15.16:0/1897288521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1350100480 0x7f1350100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 -- 172.21.15.16:0/1897288521 >> 172.21.15.16:0/1897288521 conn(0x7f1350077210 msgr2=0x7f13500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 -- 172.21.15.16:0/1897288521 shutdown_connections 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.905+0000 7f1355b56640 1 -- 172.21.15.16:0/1897288521 wait complete. 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 Processor -- start 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 -- start start 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100480 0x7f1350196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1350100e30 0x7f1350196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13501055b0 0x7f135019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f135010fd90 con 0x7f1350100480 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f135010fc10 con 0x7f1350100e30 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1355b56640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f135010ff10 con 0x7f13501055b0 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100480 0x7f1350196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100480 0x7f1350196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40900/0 (socket says 172.21.15.16:40900) 2024-09-15T22:41:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f134f7fe640 1 -- 172.21.15.16:0/1494503508 learned_addr learned my addr 172.21.15.16:0/1494503508 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f1347fff640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1350100e30 0x7f1350196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.906+0000 7f134ffff640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13501055b0 0x7f135019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134f7fe640 1 -- 172.21.15.16:0/1494503508 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13501055b0 msgr2=0x7f135019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13501055b0 0x7f135019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134f7fe640 1 -- 172.21.15.16:0/1494503508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1350100e30 msgr2=0x7f1350196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1350100e30 0x7f1350196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134f7fe640 1 -- 172.21.15.16:0/1494503508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135019b200 con 0x7f1350100480 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f1347fff640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1350100e30 0x7f1350196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100480 0x7f1350196250 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1340009970 tx=0x7f1340009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134d7fa640 1 -- 172.21.15.16:0/1494503508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134000d020 con 0x7f1350100480 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134d7fa640 1 -- 172.21.15.16:0/1494503508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1340007d40 con 0x7f1350100480 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135019b4f0 con 0x7f1350100480 2024-09-15T22:41:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f134d7fa640 1 -- 172.21.15.16:0/1494503508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134000ccc0 con 0x7f1350100480 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.907+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13501a2dd0 con 0x7f1350100480 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.910+0000 7f134d7fa640 1 -- 172.21.15.16:0/1494503508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f134000c460 con 0x7f1350100480 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.910+0000 7f1357f2f740 1 Processor -- start 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.910+0000 7f1357f2f740 1 -- start start 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.910+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.910+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.910+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd7eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.910+0000 7f1357f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557fbbcd32f0 con 0x557fbbd6d5d0 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f1357f2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557fbbc7acf0 con 0x557fbbd6eb40 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f1357f2f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557fbbccf2b0 con 0x557fbbd7c780 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40904/0 (socket says 172.21.15.16:40904) 2024-09-15T22:41:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 -- 172.21.15.16:0/2520279877 learned_addr learned my addr 172.21.15.16:0/2520279877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134d7fa640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f133c0a3430 0x7f133c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f1347fff640 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd7eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134ffff640 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134ffff640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f133c0a3430 0x7f133c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134d7fa640 1 -- 172.21.15.16:0/1494503508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f13400d1560 con 0x7f1350100480 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 -- 172.21.15.16:0/2520279877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 msgr2=0x557fbbd7eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd7eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 -- 172.21.15.16:0/2520279877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 msgr2=0x557fbbd6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 -- 172.21.15.16:0/2520279877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbbc5f3a0 con 0x557fbbd6d5d0 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134f7fe640 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6da10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f13400ce860 tx=0x7f134000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718cbec5d85033f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.911+0000 7f134ffff640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f133c0a3430 0x7f133c0a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1348005c40 tx=0x7f1348005bd0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1345ffb640 1 -- 172.21.15.16:0/2520279877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400d2a10 con 0x557fbbd6d5d0 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1345ffb640 1 -- 172.21.15.16:0/2520279877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134009db80 con 0x557fbbd6d5d0 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1345ffb640 1 -- 172.21.15.16:0/2520279877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13400d1180 con 0x557fbbd6d5d0 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 -- 172.21.15.16:0/2520279877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6d5d0 msgr2=0x557fbbd6da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6da10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f13400ce860 tx=0x7f134000c790 comp rx=0 tx=0).stop 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 -- 172.21.15.16:0/2520279877 shutdown_connections 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd7eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6da10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 --2- 172.21.15.16:0/2520279877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 -- 172.21.15.16:0/2520279877 >> 172.21.15.16:0/2520279877 conn(0x557fbbd36310 msgr2=0x557fbbd69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 -- 172.21.15.16:0/2520279877 shutdown_connections 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.912+0000 7f1357f2f740 1 -- 172.21.15.16:0/2520279877 wait complete. 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 -- start start 2024-09-15T22:41:32.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 Processor -- start 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd7c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd79470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd79c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557fbbd7f3c0 con 0x557fbbd6eb40 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557fbbc8e060 con 0x557fbbd6d5d0 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557fbbd7f540 con 0x557fbbd7c780 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1357f2f740 10 client.0 _ll_register_callbacks cb 0x557fbb9e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd79470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd79470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40906/0 (socket says 172.21.15.16:40906) 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f1347fff640 1 -- 172.21.15.16:0/3681500007 learned_addr learned my addr 172.21.15.16:0/3681500007 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.913+0000 7f134f7fe640 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd7c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f134ffff640 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd79c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1347fff640 1 -- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 msgr2=0x557fbbd79c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1347fff640 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd79c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1347fff640 1 -- 172.21.15.16:0/3681500007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 msgr2=0x557fbbd7c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1347fff640 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd7c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1347fff640 1 -- 172.21.15.16:0/3681500007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbba443c0 con 0x557fbbd6eb40 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1347fff640 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd79470 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1338009900 tx=0x7f13380318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405c93447a1fa645 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f132effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f132effd640 1 -- 172.21.15.16:0/3681500007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338035040 con 0x557fbbd6eb40 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f132effd640 1 -- 172.21.15.16:0/3681500007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1338004240 con 0x557fbbd6eb40 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134009db80 con 0x557fbbd6eb40 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f132effd640 1 -- 172.21.15.16:0/3681500007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1338004440 con 0x557fbbd6eb40 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1357f2f740 10 client.16071 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.914+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557fbbd8aeb0 con 0x557fbbd6eb40 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f132effd640 1 -- 172.21.15.16:0/3681500007 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134009db80 con 0x557fbbd6eb40 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f132effd640 1 -- 172.21.15.16:0/3681500007 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1338004bf0 con 0x557fbbd6eb40 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f1357f2f740 10 client.16071 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f1357f2f740 10 client.16071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f1357f2f740 20 client.16071 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f1357f2f740 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbd8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f1357f2f740 4 client.16071 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fbbce0860 con 0x557fbbd8c370 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.915+0000 7f134f7fe640 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbd8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f132effd640 10 client.16071 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:41:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f134f7fe640 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbd8e7e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffba80d65bdce73 server_cookie=348900bdcafc2a1c in_seq=0 out_seq=0 2024-09-15T22:41:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f132effd640 1 -- 172.21.15.16:0/3681500007 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fbbce0860 con 0x557fbbd8c370 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f132effd640 10 client.16071 handle_command_reply: tid=0 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f1357f2f740 1 client.16071 shutdown 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 msgr2=0x557fbbd79470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f1357f2f740 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd79470 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1338009900 tx=0x7f13380318d0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 shutdown_connections 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.916+0000 7f1357f2f740 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbd8e7e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f132effd640 0 client.16071 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f1357f2f740 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd79c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f1357f2f740 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd79470 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f1357f2f740 1 --2- 172.21.15.16:0/3681500007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd7c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 >> 172.21.15.16:0/3681500007 conn(0x557fbbd36310 msgr2=0x557fbbd6f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 shutdown_connections 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f1357f2f740 1 -- 172.21.15.16:0/3681500007 wait complete. 2024-09-15T22:41:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.917+0000 7f1357f2f740 20 client.16071 trim_cache size 0 max 16384 2024-09-15T22:41:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 Processor -- start 2024-09-15T22:41:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 -- start start 2024-09-15T22:41:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd69700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd7c780 0x557fbbd69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13501af5f0 con 0x557fbbd7c780 2024-09-15T22:41:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13501acec0 con 0x557fbbd6eb40 2024-09-15T22:41:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1357f2f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13501aad10 con 0x557fbbd6d5d0 2024-09-15T22:41:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd7c780 0x557fbbd69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd7c780 0x557fbbd69c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40920/0 (socket says 172.21.15.16:40920) 2024-09-15T22:41:32.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 -- 172.21.15.16:0/1419518543 learned_addr learned my addr 172.21.15.16:0/1419518543 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1347fff640 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd69700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 -- 172.21.15.16:0/1419518543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd6d5d0 msgr2=0x557fbbd6e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 -- 172.21.15.16:0/1419518543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 msgr2=0x557fbbd69700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd69700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134ffff640 1 -- 172.21.15.16:0/1419518543 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbbd6a960 con 0x557fbbd7c780 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f1347fff640 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd69700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.930+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f134ffff640 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd7c780 0x557fbbd69c40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f134800f9b0 tx=0x7f134800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8465dceb65807047 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f13457fa640 1 -- 172.21.15.16:0/1419518543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348024070 con 0x557fbbd7c780 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f13457fa640 1 -- 172.21.15.16:0/1419518543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134800d940 con 0x557fbbd7c780 2024-09-15T22:41:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f13457fa640 1 -- 172.21.15.16:0/1419518543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480054c0 con 0x557fbbd7c780 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f1357f2f740 1 -- 172.21.15.16:0/1419518543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd7c780 msgr2=0x557fbbd69c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd7c780 0x557fbbd69c40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f134800f9b0 tx=0x7f134800a710 comp rx=0 tx=0).stop 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f1357f2f740 1 -- 172.21.15.16:0/1419518543 shutdown_connections 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd7c780 0x557fbbd69c40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6eb40 0x557fbbd69700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1419518543 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd6d5d0 0x557fbbd6e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.931+0000 7f1357f2f740 1 -- 172.21.15.16:0/1419518543 >> 172.21.15.16:0/1419518543 conn(0x557fbbd36310 msgr2=0x557fbbd64a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 -- 172.21.15.16:0/1419518543 shutdown_connections 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 -- 172.21.15.16:0/1419518543 wait complete. 2024-09-15T22:41:32.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 -- start start 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 Processor -- start 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13501ab380 con 0x557fbbd6eb40 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13501ac7f0 con 0x557fbbd6d5d0 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13501aba50 con 0x557fbbd7c780 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1357f2f740 10 client.0 _ll_register_callbacks cb 0x7f1338005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.932+0000 7f1347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd80d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40934/0 (socket says 172.21.15.16:40934) 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 -- 172.21.15.16:0/1002584147 learned_addr learned my addr 172.21.15.16:0/1002584147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f134ffff640 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 -- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 msgr2=0x557fbbd812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd812a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 -- 172.21.15.16:0/1002584147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 msgr2=0x557fbbd80800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd80800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 -- 172.21.15.16:0/1002584147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbbd70120 con 0x557fbbd6eb40 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1347fff640 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd80d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f133802a930 tx=0x7f1338031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7785d404082fdd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f132effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f132effd640 1 -- 172.21.15.16:0/1002584147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338035070 con 0x557fbbd6eb40 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f132effd640 1 -- 172.21.15.16:0/1002584147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1338003ee0 con 0x557fbbd6eb40 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134800d940 con 0x557fbbd6eb40 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f132effd640 1 -- 172.21.15.16:0/1002584147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f133805f050 con 0x557fbbd6eb40 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1357f2f740 10 client.16077 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.933+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557fbbdf7430 con 0x557fbbd6eb40 2024-09-15T22:41:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f132effd640 1 -- 172.21.15.16:0/1002584147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134800d940 con 0x557fbbd6eb40 2024-09-15T22:41:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f132effd640 1 -- 172.21.15.16:0/1002584147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13380046c0 con 0x557fbbd6eb40 2024-09-15T22:41:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f1357f2f740 10 client.16077 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f1357f2f740 10 client.16077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:41:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f1357f2f740 20 client.16077 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbdf9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f1357f2f740 4 client.16077 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557fbbcdad70 con 0x557fbbd8c370 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.934+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbdf9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f132effd640 10 client.16077 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f134f7fe640 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbdf9c40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf508941a3879693 server_cookie=64dd1943cbd887b7 in_seq=0 out_seq=0 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f132effd640 1 -- 172.21.15.16:0/1002584147 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557fbbcdad70 con 0x557fbbd8c370 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f132effd640 10 client.16077 handle_command_reply: tid=0 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 client.16077 shutdown 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 msgr2=0x557fbbd80d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd80d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f133802a930 tx=0x7f1338031db0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 shutdown_connections 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x557fbbd8c370 0x557fbbdf9c40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557fbbd7c780 0x557fbbd812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557fbbd6eb40 0x557fbbd80d40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 --2- 172.21.15.16:0/1002584147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557fbbd6d5d0 0x557fbbd80800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 >> 172.21.15.16:0/1002584147 conn(0x557fbbd36310 msgr2=0x557fbbd69700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f132effd640 0 client.16077 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:41:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 shutdown_connections 2024-09-15T22:41:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 1 -- 172.21.15.16:0/1002584147 wait complete. 2024-09-15T22:41:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.935+0000 7f1357f2f740 20 client.16077 trim_cache size 0 max 16384 2024-09-15T22:41:32.942 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f133c0a3430 msgr2=0x7f133c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f133c0a3430 0x7f133c0a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1348005c40 tx=0x7f1348005bd0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100480 msgr2=0x7f1350196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100480 0x7f1350196250 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1340009970 tx=0x7f1340009e30 comp rx=0 tx=0).stop 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 shutdown_connections 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f133c0a3430 0x7f133c0a58f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13501055b0 0x7f135019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1350100e30 0x7f1350196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 --2- 172.21.15.16:0/1494503508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1350100480 0x7f1350196250 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 >> 172.21.15.16:0/1494503508 conn(0x7f1350077210 msgr2=0x7f1350103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 shutdown_connections 2024-09-15T22:41:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:32.936+0000 7f1355b56640 1 -- 172.21.15.16:0/1494503508 wait complete. 2024-09-15T22:41:33.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:32 smithi130 ceph-mon[35742]: pgmap v96: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:33.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:32 smithi059 ceph-mon[34872]: pgmap v96: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:33.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:41:34.067 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.524+0000 7f13f4130640 1 -- 172.21.15.16:0/49896311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 msgr2=0x7f13ec1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.524+0000 7f13f4130640 1 --2- 172.21.15.16:0/49896311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec1031d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009900 tx=0x7f13e802a410 comp rx=0 tx=0).stop 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.524+0000 7f13f4130640 1 -- 172.21.15.16:0/49896311 shutdown_connections 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.524+0000 7f13f4130640 1 --2- 172.21.15.16:0/49896311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13ec10c2f0 0x7f13ec10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.524+0000 7f13f4130640 1 --2- 172.21.15.16:0/49896311 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13ec1037a0 0x7f13ec10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.524+0000 7f13f4130640 1 --2- 172.21.15.16:0/49896311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec1031d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.524+0000 7f13f4130640 1 -- 172.21.15.16:0/49896311 >> 172.21.15.16:0/49896311 conn(0x7f13ec0fc7d0 msgr2=0x7f13ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.525+0000 7f13f4130640 1 -- 172.21.15.16:0/49896311 shutdown_connections 2024-09-15T22:41:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.525+0000 7f13f4130640 1 -- 172.21.15.16:0/49896311 wait complete. 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.525+0000 7f13f4130640 1 Processor -- start 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.525+0000 7f13f4130640 1 -- start start 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f4130640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f4130640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13ec1037a0 0x7f13ec198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f4130640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13ec10c2f0 0x7f13ec19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f4130640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1120f0 con 0x7f13ec102df0 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f4130640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec111f70 con 0x7f13ec10c2f0 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f4130640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec112270 con 0x7f13ec1037a0 2024-09-15T22:41:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40946/0 (socket says 172.21.15.16:40946) 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3421897865 learned_addr learned my addr 172.21.15.16:0/3421897865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f16cb640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13ec1037a0 0x7f13ec198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3421897865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13ec1037a0 msgr2=0x7f13ec198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13ec1037a0 0x7f13ec198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3421897865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13ec10c2f0 msgr2=0x7f13ec19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f26cd640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13ec10c2f0 0x7f13ec19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13ec10c2f0 0x7f13ec19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3421897865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec19d5c0 con 0x7f13ec102df0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f26cd640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13ec10c2f0 0x7f13ec19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.526+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec198610 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13e8006010 tx=0x7f13e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.527+0000 7f13e2ffd640 1 -- 172.21.15.16:0/3421897865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e803f030 con 0x7f13ec102df0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.527+0000 7f13e2ffd640 1 -- 172.21.15.16:0/3421897865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e8003ce0 con 0x7f13ec102df0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.527+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ec19d850 con 0x7f13ec102df0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.527+0000 7f13e2ffd640 1 -- 172.21.15.16:0/3421897865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8035070 con 0x7f13ec102df0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.527+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ec1a5190 con 0x7f13ec102df0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.529+0000 7f13e2ffd640 1 -- 172.21.15.16:0/3421897865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f13e8042030 con 0x7f13ec102df0 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.529+0000 7f13f6509740 1 Processor -- start 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.529+0000 7f13f6509740 1 -- start start 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f6509740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926640f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f6509740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f6509740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f6509740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592663752f0 con 0x55926640f5d0 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f6509740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55926631ccf0 con 0x55926641a370 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f6509740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5592663712b0 con 0x559266410b40 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40950/0 (socket says 172.21.15.16:40950) 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3401207762 learned_addr learned my addr 172.21.15.16:0/3401207762 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13e2ffd640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f13d80a33b0 0x7f13d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f16cb640 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f26cd640 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926640f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f26cd640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f13d80a33b0 0x7f13d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13e2ffd640 1 -- 172.21.15.16:0/3421897865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f13e80f0710 con 0x7f13ec102df0 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3401207762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 msgr2=0x55926640f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926640f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3401207762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 msgr2=0x55926641c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 -- 172.21.15.16:0/3401207762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592663013a0 con 0x55926640f5d0 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640fa10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13e80edaa0 tx=0x7f13e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e5567b838df56d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.530+0000 7f13f26cd640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f13d80a33b0 0x7f13d80a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f13e40094d0 tx=0x7f13e4009500 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13d7fff640 1 -- 172.21.15.16:0/3401207762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8049070 con 0x55926640f5d0 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13d7fff640 1 -- 172.21.15.16:0/3401207762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80bce50 con 0x55926640f5d0 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13d7fff640 1 -- 172.21.15.16:0/3401207762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80c53c0 con 0x55926640f5d0 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13f6509740 1 -- 172.21.15.16:0/3401207762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 msgr2=0x55926640fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13f6509740 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640fa10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13e80edaa0 tx=0x7f13e802a410 comp rx=0 tx=0).stop 2024-09-15T22:41:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13f6509740 1 -- 172.21.15.16:0/3401207762 shutdown_connections 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13f6509740 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13f6509740 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640fa10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13f6509740 1 --2- 172.21.15.16:0/3401207762 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926640f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.531+0000 7f13f6509740 1 -- 172.21.15.16:0/3401207762 >> 172.21.15.16:0/3401207762 conn(0x5592663d8310 msgr2=0x55926640bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.532+0000 7f13f6509740 1 -- 172.21.15.16:0/3401207762 shutdown_connections 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.532+0000 7f13f6509740 1 -- 172.21.15.16:0/3401207762 wait complete. 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.532+0000 7f13f6509740 1 -- start start 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.532+0000 7f13f6509740 1 Processor -- start 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266419d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x559266417020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266417610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592663752f0 con 0x55926640f5d0 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559266330060 con 0x559266410b40 2024-09-15T22:41:34.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55926641ce30 con 0x55926641a370 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 10 client.0 _ll_register_callbacks cb 0x5592660838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266419d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266419d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40958/0 (socket says 172.21.15.16:40958) 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 -- 172.21.15.16:0/1794945436 learned_addr learned my addr 172.21.15.16:0/1794945436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f16cb640 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x559266417020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f26cd640 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266417610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 -- 172.21.15.16:0/1794945436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 msgr2=0x559266417610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266417610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 -- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 msgr2=0x559266417020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x559266417020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 -- 172.21.15.16:0/1794945436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592660e63c0 con 0x55926640f5d0 2024-09-15T22:41:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266419d70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f13e80c4e90 tx=0x7f13e80c0be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60077506930bd220 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13d57fa640 1 -- 172.21.15.16:0/1794945436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8049050 con 0x55926640f5d0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13d57fa640 1 -- 172.21.15.16:0/1794945436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f13e80b8dc0 con 0x55926640f5d0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e80bce50 con 0x55926640f5d0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.533+0000 7f13d57fa640 1 -- 172.21.15.16:0/1794945436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13e80bc0b0 con 0x55926640f5d0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 10 client.16089 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559266428750 con 0x55926640f5d0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13d57fa640 1 -- 172.21.15.16:0/1794945436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80bce50 con 0x55926640f5d0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13d57fa640 1 -- 172.21.15.16:0/1794945436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13e80f0030 con 0x55926640f5d0 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 10 client.16089 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 10 client.16089 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 20 client.16089 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559266429aa0 0x55926642be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 4 client.16089 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559266382860 con 0x559266429aa0 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.534+0000 7f13f16cb640 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559266429aa0 0x55926642be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.535+0000 7f13d57fa640 10 client.16089 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.535+0000 7f13f16cb640 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559266429aa0 0x55926642be90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12bdacaa7a2f7da9 server_cookie=2e14cf1277fb056a in_seq=0 out_seq=0 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.535+0000 7f13d57fa640 1 -- 172.21.15.16:0/1794945436 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559266382860 con 0x559266429aa0 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.535+0000 7f13d57fa640 10 client.16089 handle_command_reply: tid=0 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.535+0000 7f13f6509740 1 client.16089 shutdown 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 msgr2=0x559266419d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266419d70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f13e80c4e90 tx=0x7f13e80c0be0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 shutdown_connections 2024-09-15T22:41:34.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559266429aa0 0x55926642be90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266417610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x559266417020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 --2- 172.21.15.16:0/1794945436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266419d70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13d57fa640 0 client.16089 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 >> 172.21.15.16:0/1794945436 conn(0x5592663d8310 msgr2=0x559266411990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 shutdown_connections 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 1 -- 172.21.15.16:0/1794945436 wait complete. 2024-09-15T22:41:34.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.536+0000 7f13f6509740 20 client.16089 trim_cache size 0 max 16384 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 Processor -- start 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 -- start start 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x55926640abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266412120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1b1890 con 0x55926640f5d0 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1af160 con 0x559266410b40 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f6509740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1acfb0 con 0x55926641a370 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40972/0 (socket says 172.21.15.16:40972) 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f1ecc640 1 -- 172.21.15.16:0/2629525932 learned_addr learned my addr 172.21.15.16:0/2629525932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.549+0000 7f13f16cb640 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x55926640abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13f26cd640 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266412120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13f1ecc640 1 -- 172.21.15.16:0/2629525932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 msgr2=0x559266412120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266412120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13f1ecc640 1 -- 172.21.15.16:0/2629525932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 msgr2=0x55926640abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x55926640abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13f1ecc640 1 -- 172.21.15.16:0/2629525932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926642c4c0 con 0x55926640f5d0 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640bd70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f13e8000c00 tx=0x7f13e8009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb9df1b1ee708a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13d77fe640 1 -- 172.21.15.16:0/2629525932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8049070 con 0x55926640f5d0 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13d77fe640 1 -- 172.21.15.16:0/2629525932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80bce50 con 0x55926640f5d0 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.550+0000 7f13d77fe640 1 -- 172.21.15.16:0/2629525932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80c4f30 con 0x55926640f5d0 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 -- 172.21.15.16:0/2629525932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 msgr2=0x55926640bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640bd70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f13e8000c00 tx=0x7f13e8009620 comp rx=0 tx=0).stop 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 -- 172.21.15.16:0/2629525932 shutdown_connections 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55926641a370 0x559266412120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559266410b40 0x55926640abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 --2- 172.21.15.16:0/2629525932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x55926640bd70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 -- 172.21.15.16:0/2629525932 >> 172.21.15.16:0/2629525932 conn(0x5592663d8310 msgr2=0x55926640a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 -- 172.21.15.16:0/2629525932 shutdown_connections 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 -- 172.21.15.16:0/2629525932 wait complete. 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 -- start start 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.551+0000 7f13f6509740 1 Processor -- start 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f6509740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266422550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f6509740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926641d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f6509740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f6509740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1ad620 con 0x55926640f5d0 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f6509740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1aea90 con 0x55926641a370 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f6509740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1adcf0 con 0x559266410b40 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f6509740 10 client.0 _ll_register_callbacks cb 0x7f13dc0092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:34.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266422550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266422550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40984/0 (socket says 172.21.15.16:40984) 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 -- 172.21.15.16:0/359976594 learned_addr learned my addr 172.21.15.16:0/359976594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f16cb640 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926641d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f26cd640 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 -- 172.21.15.16:0/359976594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 msgr2=0x55926641d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926641d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 -- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 msgr2=0x55926641de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 -- 172.21.15.16:0/359976594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926640c960 con 0x55926640f5d0 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13f1ecc640 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266422550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f13e80ee100 tx=0x7f13e8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962504cdd82b331a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.552+0000 7f13d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13d57fa640 1 -- 172.21.15.16:0/359976594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8049040 con 0x55926640f5d0 2024-09-15T22:41:34.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13d57fa640 1 -- 172.21.15.16:0/359976594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f13e80c5140 con 0x55926640f5d0 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e80bce50 con 0x55926640f5d0 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13d57fa640 1 -- 172.21.15.16:0/359976594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13e80b8f30 con 0x55926640f5d0 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 10 client.16095 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55926640c960 con 0x55926640f5d0 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13d57fa640 1 -- 172.21.15.16:0/359976594 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e80bce50 con 0x55926640f5d0 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13d57fa640 1 -- 172.21.15.16:0/359976594 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13e80f0040 con 0x55926640f5d0 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 10 client.16095 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 10 client.16095 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 20 client.16095 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:34.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5592664945a0 0x559266496990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 4 client.16095 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55926637cd70 con 0x5592664945a0 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.553+0000 7f13f16cb640 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5592664945a0 0x559266496990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f16cb640 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5592664945a0 0x559266496990 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1805f891ffecea7f server_cookie=c486dcca51fa8a77 in_seq=0 out_seq=0 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13d57fa640 10 client.16095 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13d57fa640 1 -- 172.21.15.16:0/359976594 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55926637cd70 con 0x5592664945a0 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13d57fa640 10 client.16095 handle_command_reply: tid=0 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 client.16095 shutdown 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 msgr2=0x559266422550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266422550 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f13e80ee100 tx=0x7f13e8005c90 comp rx=0 tx=0).stop 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 shutdown_connections 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5592664945a0 0x559266496990 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55926641a370 0x55926641de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559266410b40 0x55926641d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13d57fa640 0 client.16095 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 --2- 172.21.15.16:0/359976594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55926640f5d0 0x559266422550 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 >> 172.21.15.16:0/359976594 conn(0x5592663d8310 msgr2=0x55926640ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 shutdown_connections 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.554+0000 7f13f6509740 1 -- 172.21.15.16:0/359976594 wait complete. 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f6509740 20 client.16095 trim_cache size 0 max 16384 2024-09-15T22:41:34.564 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:41:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f13d80a33b0 msgr2=0x7f13d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f13d80a33b0 0x7f13d80a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f13e40094d0 tx=0x7f13e4009500 comp rx=0 tx=0).stop 2024-09-15T22:41:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 msgr2=0x7f13ec198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec198610 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13e8006010 tx=0x7f13e80042f0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 shutdown_connections 2024-09-15T22:41:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13ec10c2f0 0x7f13ec19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f13d80a33b0 0x7f13d80a5870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f13ec1037a0 0x7f13ec198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 --2- 172.21.15.16:0/3421897865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ec102df0 0x7f13ec198610 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 >> 172.21.15.16:0/3421897865 conn(0x7f13ec0fc7d0 msgr2=0x7f13ec0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 shutdown_connections 2024-09-15T22:41:34.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:34.555+0000 7f13f4130640 1 -- 172.21.15.16:0/3421897865 wait complete. 2024-09-15T22:41:35.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:34 smithi130 ceph-mon[35742]: pgmap v97: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:35.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:34 smithi016 ceph-mon[31492]: pgmap v97: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:35.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:34 smithi059 ceph-mon[34872]: pgmap v97: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:35.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:41:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:36.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:35 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:36.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:35 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:36.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:35 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:36.010 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:35 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:41:36.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:35 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:35 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:35 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:36.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:35 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:41:36.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 -- 172.21.15.16:0/3429018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c100a60 msgr2=0x7ffb5c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 --2- 172.21.15.16:0/3429018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c100a60 0x7ffb5c100ec0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009930 tx=0x7ffb4c02a420 comp rx=0 tx=0).stop 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 -- 172.21.15.16:0/3429018 shutdown_connections 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 --2- 172.21.15.16:0/3429018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb5c101400 0x7ffb5c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 --2- 172.21.15.16:0/3429018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c100a60 0x7ffb5c100ec0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 --2- 172.21.15.16:0/3429018 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb5c106a60 0x7ffb5c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 -- 172.21.15.16:0/3429018 >> 172.21.15.16:0/3429018 conn(0x7ffb5c0fc780 msgr2=0x7ffb5c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 -- 172.21.15.16:0/3429018 shutdown_connections 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.093+0000 7ffb634a2640 1 -- 172.21.15.16:0/3429018 wait complete. 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 Processor -- start 2024-09-15T22:41:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 -- start start 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb5c100a60 0x7ffb5c10e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb5c101400 0x7ffb5c10e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c106a60 0x7ffb5c10ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c111f80 con 0x7ffb5c106a60 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c111e00 con 0x7ffb5c100a60 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb634a2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c112100 con 0x7ffb5c101400 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c106a60 0x7ffb5c10ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c106a60 0x7ffb5c10ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40998/0 (socket says 172.21.15.16:40998) 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 -- 172.21.15.16:0/1592817257 learned_addr learned my addr 172.21.15.16:0/1592817257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb5c101400 0x7ffb5c10e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb6123e640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb5c100a60 0x7ffb5c10e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 -- 172.21.15.16:0/1592817257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb5c101400 msgr2=0x7ffb5c10e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb5c101400 0x7ffb5c10e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 -- 172.21.15.16:0/1592817257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb5c100a60 msgr2=0x7ffb5c10e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb5c100a60 0x7ffb5c10e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.094+0000 7ffb61a3f640 1 -- 172.21.15.16:0/1592817257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c1a6920 con 0x7ffb5c106a60 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.095+0000 7ffb61a3f640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c106a60 0x7ffb5c10ee40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400cce0 tx=0x7ffb5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.095+0000 7ffb527fc640 1 -- 172.21.15.16:0/1592817257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400f450 con 0x7ffb5c106a60 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.095+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5c1a6c10 con 0x7ffb5c106a60 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.095+0000 7ffb527fc640 1 -- 172.21.15.16:0/1592817257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5400fad0 con 0x7ffb5c106a60 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.095+0000 7ffb527fc640 1 -- 172.21.15.16:0/1592817257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54018c10 con 0x7ffb5c106a60 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.095+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5c1a70d0 con 0x7ffb5c106a60 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 Processor -- start 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb527fc640 1 -- 172.21.15.16:0/1592817257 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffb5400f5f0 con 0x7ffb5c106a60 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 -- start start 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b6b40 0x556e036b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b55d0 0x556e036b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e0361b2f0 con 0x556e036c0370 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e035c2cf0 con 0x556e036b6b40 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6587b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556e036172b0 con 0x556e036b55d0 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb60a3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb60a3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41014/0 (socket says 172.21.15.16:41014) 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3254356554 learned_addr learned my addr 172.21.15.16:0/3254356554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b6b40 0x556e036b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.098+0000 7ffb6123e640 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b55d0 0x556e036b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb527fc640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ffb480a33b0 0x7ffb480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3254356554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b55d0 msgr2=0x556e036b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b55d0 0x556e036b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3254356554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b6b40 msgr2=0x556e036b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b6b40 0x556e036b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3254356554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e035a73a0 con 0x556e036c0370 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb61a3f640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ffb480a33b0 0x7ffb480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb527fc640 1 -- 172.21.15.16:0/1592817257 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ffb540d0770 con 0x7ffb5c106a60 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036c2760 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005ec0 tx=0x7ffb4c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad846893e9ccb5f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb477fe640 1 -- 172.21.15.16:0/3254356554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c035070 con 0x556e036c0370 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb61a3f640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ffb480a33b0 0x7ffb480a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb540cdc10 tx=0x7ffb5401baf0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb477fe640 1 -- 172.21.15.16:0/3254356554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb4c005020 con 0x556e036c0370 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.099+0000 7ffb477fe640 1 -- 172.21.15.16:0/3254356554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c004b20 con 0x556e036c0370 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 -- 172.21.15.16:0/3254356554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 msgr2=0x556e036c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036c2760 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c005ec0 tx=0x7ffb4c02a420 comp rx=0 tx=0).stop 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 -- 172.21.15.16:0/3254356554 shutdown_connections 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036c2760 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b55d0 0x556e036b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3254356554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b6b40 0x556e036b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 -- 172.21.15.16:0/3254356554 >> 172.21.15.16:0/3254356554 conn(0x556e0367e310 msgr2=0x556e036b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 -- 172.21.15.16:0/3254356554 shutdown_connections 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 -- 172.21.15.16:0/3254356554 wait complete. 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.100+0000 7ffb6587b740 1 -- start start 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 1 Processor -- start 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036bd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556e036c3010 con 0x556e036c0370 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e035d6060 con 0x556e036b55d0 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556e036c3190 con 0x556e036b6b40 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6587b740 10 client.0 _ll_register_callbacks cb 0x556e033298c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:36.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036bd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41024/0 (socket says 172.21.15.16:41024) 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 -- 172.21.15.16:0/2552516719 learned_addr learned my addr 172.21.15.16:0/2552516719 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb6123e640 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036bd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 -- 172.21.15.16:0/2552516719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 msgr2=0x556e036bd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036bd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 -- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 msgr2=0x556e036bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.101+0000 7ffb60a3d640 1 -- 172.21.15.16:0/2552516719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0338c3c0 con 0x556e036c0370 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036bd640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c02a940 tx=0x7ffb4c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11686ad176fa43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2552516719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c035070 con 0x556e036c0370 2024-09-15T22:41:36.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2552516719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ffb4c0055e0 con 0x556e036c0370 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb4c005020 con 0x556e036c0370 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2552516719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffb4c0048a0 con 0x556e036c0370 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb6587b740 10 client.16107 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e036ceb10 con 0x556e036c0370 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2552516719 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb4c005020 con 0x556e036c0370 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2552516719 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffb4c04a040 con 0x556e036c0370 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb6587b740 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb6587b740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.102+0000 7ffb6587b740 20 client.16107 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.103+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e036cfe60 0x556e036d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.103+0000 7ffb6587b740 4 client.16107 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.103+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e03628860 con 0x556e036cfe60 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.103+0000 7ffb6123e640 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e036cfe60 0x556e036d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.103+0000 7ffb44ff9640 10 client.16107 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.103+0000 7ffb6123e640 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e036cfe60 0x556e036d2250 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a334727103bb9c43 server_cookie=12acb36ec5bdffe5 in_seq=0 out_seq=0 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2552516719 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e03628860 con 0x556e036cfe60 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb44ff9640 10 client.16107 handle_command_reply: tid=0 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 client.16107 shutdown 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 msgr2=0x556e036bd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036bd640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c02a940 tx=0x7ffb4c041040 comp rx=0 tx=0).stop 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 shutdown_connections 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e036cfe60 0x556e036d2250 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036bd640 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036bd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb44ff9640 0 client.16107 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2552516719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 >> 172.21.15.16:0/2552516719 conn(0x556e0367e310 msgr2=0x556e036b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:36.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 shutdown_connections 2024-09-15T22:41:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 1 -- 172.21.15.16:0/2552516719 wait complete. 2024-09-15T22:41:36.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.104+0000 7ffb6587b740 20 client.16107 trim_cache size 0 max 16384 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.117+0000 7ffb6587b740 1 Processor -- start 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.117+0000 7ffb6587b740 1 -- start start 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb6587b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c1b39d0 con 0x556e036c0370 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb6587b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c1b12a0 con 0x556e036b55d0 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb6587b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c1af0f0 con 0x556e036b6b40 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036ba2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41026/0 (socket says 172.21.15.16:41026) 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3867489689 learned_addr learned my addr 172.21.15.16:0/3867489689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb6123e640 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3867489689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 msgr2=0x556e036b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3867489689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 msgr2=0x556e036cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 -- 172.21.15.16:0/3867489689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e036bd810 con 0x556e036c0370 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036ba2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c04dd90 tx=0x7ffb4c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7e13386c4b4ae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.118+0000 7ffb46ffd640 1 -- 172.21.15.16:0/3867489689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c035070 con 0x556e036c0370 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb46ffd640 1 -- 172.21.15.16:0/3867489689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb4c03d450 con 0x556e036c0370 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb46ffd640 1 -- 172.21.15.16:0/3867489689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c04d7f0 con 0x556e036c0370 2024-09-15T22:41:36.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 -- 172.21.15.16:0/3867489689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 msgr2=0x556e036ba2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036ba2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c04dd90 tx=0x7ffb4c02a420 comp rx=0 tx=0).stop 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 -- 172.21.15.16:0/3867489689 shutdown_connections 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036c0370 0x556e036ba2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036b6b40 0x556e036b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 --2- 172.21.15.16:0/3867489689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 -- 172.21.15.16:0/3867489689 >> 172.21.15.16:0/3867489689 conn(0x556e0367e310 msgr2=0x556e036b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 -- 172.21.15.16:0/3867489689 shutdown_connections 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.119+0000 7ffb6587b740 1 -- 172.21.15.16:0/3867489689 wait complete. 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 -- start start 2024-09-15T22:41:36.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 Processor -- start 2024-09-15T22:41:36.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036b6b40 0x556e036d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036c0370 0x556e03738790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c1af760 con 0x556e036b6b40 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c1b0bd0 con 0x556e036b55d0 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c1afe30 con 0x556e036c0370 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6587b740 10 client.0 _ll_register_callbacks cb 0x7ffb4c032dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6123e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036b6b40 0x556e036d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6123e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036b6b40 0x556e036d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41036/0 (socket says 172.21.15.16:41036) 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb6123e640 1 -- 172.21.15.16:0/2692035255 learned_addr learned my addr 172.21.15.16:0/2692035255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:36.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036c0370 0x556e03738790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.120+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6123e640 1 -- 172.21.15.16:0/2692035255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036c0370 msgr2=0x556e03738790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6123e640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036c0370 0x556e03738790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6123e640 1 -- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 msgr2=0x556e036d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6123e640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6123e640 1 -- 172.21.15.16:0/2692035255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e03738d80 con 0x556e036b6b40 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6123e640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036b6b40 0x556e036d2d30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffb58004700 tx=0x7ffb58009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a9be5507794f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2692035255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5802c060 con 0x556e036b6b40 2024-09-15T22:41:36.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb4c03d450 con 0x556e036b6b40 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2692035255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ffb58004460 con 0x556e036b6b40 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2692035255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffb58033050 con 0x556e036b6b40 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6587b740 10 client.16113 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e0373a0d0 con 0x556e036b6b40 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.121+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2692035255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb4c03d450 con 0x556e036b6b40 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2692035255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffb5801e8b0 con 0x556e036b6b40 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb6587b740 10 client.16113 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb6587b740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb6587b740 20 client.16113 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e0373a880 0x556e0373cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb6587b740 4 client.16113 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:41:36.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e03622d70 con 0x556e0373a880 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e0373a880 0x556e0373cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb60a3d640 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e0373a880 0x556e0373cca0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4733883bf741cf server_cookie=ffc3d62fa53a525d in_seq=0 out_seq=0 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb44ff9640 10 client.16113 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb44ff9640 1 -- 172.21.15.16:0/2692035255 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e03622d70 con 0x556e0373a880 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb44ff9640 10 client.16113 handle_command_reply: tid=0 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.122+0000 7ffb6587b740 1 client.16113 shutdown 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036b6b40 msgr2=0x556e036d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036b6b40 0x556e036d2d30 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ffb58004700 tx=0x7ffb58009bc0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 shutdown_connections 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556e0373a880 0x556e0373cca0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556e036c0370 0x556e03738790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb44ff9640 0 client.16113 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556e036b6b40 0x556e036d2d30 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 --2- 172.21.15.16:0/2692035255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e036b55d0 0x556e036d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 >> 172.21.15.16:0/2692035255 conn(0x556e0367e310 msgr2=0x556e036be890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 shutdown_connections 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 1 -- 172.21.15.16:0/2692035255 wait complete. 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb6587b740 20 client.16113 trim_cache size 0 max 16384 2024-09-15T22:41:36.131 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:41:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ffb480a33b0 msgr2=0x7ffb480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb634a2640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ffb480a33b0 0x7ffb480a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ffb540cdc10 tx=0x7ffb5401baf0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c106a60 msgr2=0x7ffb5c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:36.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.123+0000 7ffb634a2640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c106a60 0x7ffb5c10ee40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400cce0 tx=0x7ffb5400a620 comp rx=0 tx=0).stop 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 shutdown_connections 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb5c106a60 0x7ffb5c10ee40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ffb480a33b0 0x7ffb480a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ffb5c101400 0x7ffb5c10e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 --2- 172.21.15.16:0/1592817257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb5c100a60 0x7ffb5c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 >> 172.21.15.16:0/1592817257 conn(0x7ffb5c0fc780 msgr2=0x7ffb5c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 shutdown_connections 2024-09-15T22:41:36.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:36.124+0000 7ffb634a2640 1 -- 172.21.15.16:0/1592817257 wait complete. 2024-09-15T22:41:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:35 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:35 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:35 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:36.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:35 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:41:37.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:36 smithi130 ceph-mon[35742]: pgmap v98: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:36 smithi016 ceph-mon[31492]: pgmap v98: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:36 smithi059 ceph-mon[34872]: pgmap v98: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:37.797 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:41:37.797 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:41:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 -- 172.21.15.16:0/441068415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 msgr2=0x7f77fc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 --2- 172.21.15.16:0/441068415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc10da40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009ca0 tx=0x7f77f402a950 comp rx=0 tx=0).stop 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 -- 172.21.15.16:0/441068415 shutdown_connections 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 --2- 172.21.15.16:0/441068415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc10da40 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 --2- 172.21.15.16:0/441068415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77fc102570 0x7f77fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 --2- 172.21.15.16:0/441068415 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77fc100e20 0x7f77fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 -- 172.21.15.16:0/441068415 >> 172.21.15.16:0/441068415 conn(0x7f77fc0fc7d0 msgr2=0x7f77fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 -- 172.21.15.16:0/441068415 shutdown_connections 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.530+0000 7f78009a2640 1 -- 172.21.15.16:0/441068415 wait complete. 2024-09-15T22:41:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 Processor -- start 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 -- start start 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77fc100e20 0x7f77fc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77fc102570 0x7f77fc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc110f80 con 0x7f77fc102ef0 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc110e00 con 0x7f77fc102570 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f78009a2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc111100 con 0x7f77fc100e20 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41044/0 (socket says 172.21.15.16:41044) 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 -- 172.21.15.16:0/103296155 learned_addr learned my addr 172.21.15.16:0/103296155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fa59c640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77fc100e20 0x7f77fc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77fc102570 0x7f77fc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 -- 172.21.15.16:0/103296155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77fc100e20 msgr2=0x7f77fc19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77fc100e20 0x7f77fc19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 -- 172.21.15.16:0/103296155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77fc102570 msgr2=0x7f77fc19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77fc102570 0x7f77fc19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.531+0000 7f77fad9d640 1 -- 172.21.15.16:0/103296155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77fc19f610 con 0x7f77fc102ef0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.532+0000 7f77fad9d640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc19ef30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f77f4004af0 tx=0x7f77f402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.532+0000 7f77eb7fe640 1 -- 172.21.15.16:0/103296155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4034040 con 0x7f77fc102ef0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.532+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77fc19f8a0 con 0x7f77fc102ef0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.532+0000 7f77eb7fe640 1 -- 172.21.15.16:0/103296155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f4030920 con 0x7f77fc102ef0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.532+0000 7f77eb7fe640 1 -- 172.21.15.16:0/103296155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f4004820 con 0x7f77fc102ef0 2024-09-15T22:41:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.532+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77fc1a71e0 con 0x7f77fc102ef0 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.534+0000 7f77eb7fe640 1 -- 172.21.15.16:0/103296155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f77f4037020 con 0x7f77fc102ef0 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.534+0000 7f7802d7b740 1 Processor -- start 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f7802d7b740 1 -- start start 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de66b60 0x55df6de66f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de71800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f7802d7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df6ddcc280 con 0x55df6de67480 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f7802d7b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df6dd73b10 con 0x55df6de66b60 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f7802d7b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55df6ddc8240 con 0x55df6de71d40 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de71800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de71800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41050/0 (socket says 172.21.15.16:41050) 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 -- 172.21.15.16:0/1444391282 learned_addr learned my addr 172.21.15.16:0/1444391282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de66b60 0x55df6de66f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77eb7fe640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77e40a33b0 0x7f77e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 -- 172.21.15.16:0/1444391282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 msgr2=0x55df6de74130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de74130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 -- 172.21.15.16:0/1444391282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de66b60 msgr2=0x55df6de66f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de66b60 0x55df6de66f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.535+0000 7f77fa59c640 1 -- 172.21.15.16:0/1444391282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6dd801f0 con 0x55df6de67480 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.536+0000 7f77eb7fe640 1 -- 172.21.15.16:0/103296155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f77f40f0eb0 con 0x7f77fc102ef0 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.536+0000 7f77fad9d640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77e40a33b0 0x7f77e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.536+0000 7f77fa59c640 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de71800 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00b760 tx=0x7f77ec00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5e023f8454892c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.536+0000 7f77e8ff9640 1 -- 172.21.15.16:0/1444391282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec003dd0 con 0x55df6de67480 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.536+0000 7f77e8ff9640 1 -- 172.21.15.16:0/1444391282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ec015070 con 0x55df6de67480 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.536+0000 7f77e8ff9640 1 -- 172.21.15.16:0/1444391282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011440 con 0x55df6de67480 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 -- 172.21.15.16:0/1444391282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 msgr2=0x55df6de71800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f77fad9d640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77e40a33b0 0x7f77e40a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77f40b9910 tx=0x7f77f40095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de71800 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f77ec00b760 tx=0x7f77ec00bc20 comp rx=0 tx=0).stop 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 -- 172.21.15.16:0/1444391282 shutdown_connections 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de74130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de71800 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1444391282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de66b60 0x55df6de66f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 -- 172.21.15.16:0/1444391282 >> 172.21.15.16:0/1444391282 conn(0x55df6de630f0 msgr2=0x55df6de634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 -- 172.21.15.16:0/1444391282 shutdown_connections 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 -- 172.21.15.16:0/1444391282 wait complete. 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 -- start start 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.537+0000 7f7802d7b740 1 Processor -- start 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de66b60 0x55df6de6ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de6f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de6f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f7802d7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df6de74910 con 0x55df6de67480 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f7802d7b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df6dd86f10 con 0x55df6de71d40 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f7802d7b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55df6de74a90 con 0x55df6de66b60 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de6f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de6f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41066/0 (socket says 172.21.15.16:41066) 2024-09-15T22:41:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f7802d7b740 10 client.0 _ll_register_callbacks cb 0x55df6dd4fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 -- 172.21.15.16:0/1598782538 learned_addr learned my addr 172.21.15.16:0/1598782538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de66b60 0x55df6de6ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de6f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 -- 172.21.15.16:0/1598782538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de66b60 msgr2=0x55df6de6ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de66b60 0x55df6de6ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 -- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 msgr2=0x55df6de6f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de6f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 -- 172.21.15.16:0/1598782538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6db3d6a0 con 0x55df6de67480 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77fa59c640 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de6f190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006010 tx=0x7f77ec003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae7ff37beac3b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.538+0000 7f77e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f77e1ffb640 1 -- 172.21.15.16:0/1598782538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ec028060 con 0x55df6de67480 2024-09-15T22:41:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77ec015070 con 0x55df6de67480 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f77e1ffb640 1 -- 172.21.15.16:0/1598782538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f77ec011850 con 0x55df6de67480 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f77e1ffb640 1 -- 172.21.15.16:0/1598782538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77ec02f050 con 0x55df6de67480 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 10 client.16125 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df6de805a0 con 0x55df6de67480 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f77e1ffb640 1 -- 172.21.15.16:0/1598782538 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ec015070 con 0x55df6de67480 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f77e1ffb640 1 -- 172.21.15.16:0/1598782538 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f77ec022030 con 0x55df6de67480 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 20 client.16125 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6de819b0 0x55df6de83da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 4 client.16125 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.539+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6ddd97f0 con 0x55df6de819b0 2024-09-15T22:41:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.540+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6de819b0 0x55df6de83da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.540+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6de819b0 0x55df6de83da0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a52c224b546d36a server_cookie=d26ce9598ee7a8b4 in_seq=0 out_seq=0 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.540+0000 7f77e1ffb640 10 client.16125 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f77e1ffb640 1 -- 172.21.15.16:0/1598782538 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df6ddd97f0 con 0x55df6de819b0 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f77e1ffb640 10 client.16125 handle_command_reply: tid=0 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 client.16125 shutdown 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 msgr2=0x55df6de6f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de6f190 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006010 tx=0x7f77ec003e00 comp rx=0 tx=0).stop 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 shutdown_connections 2024-09-15T22:41:38.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6de819b0 0x55df6de83da0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de6f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de67480 0x55df6de6f190 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 --2- 172.21.15.16:0/1598782538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de66b60 0x55df6de6ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f77e1ffb640 0 client.16125 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 >> 172.21.15.16:0/1598782538 conn(0x55df6de630f0 msgr2=0x55df6de62270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 shutdown_connections 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 1 -- 172.21.15.16:0/1598782538 wait complete. 2024-09-15T22:41:38.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.541+0000 7f7802d7b740 20 client.16125 trim_cache size 0 max 16384 2024-09-15T22:41:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.558+0000 7f7802d7b740 1 Processor -- start 2024-09-15T22:41:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f7802d7b740 1 -- start start 2024-09-15T22:41:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de70a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de67480 0x55df6de893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de89900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f7802d7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1b38c0 con 0x55df6de66b60 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f7802d7b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1b1190 con 0x55df6de67480 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f7802d7b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1aefe0 con 0x55df6de71d40 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de70a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de70a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41068/0 (socket says 172.21.15.16:41068) 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 -- 172.21.15.16:0/406367769 learned_addr learned my addr 172.21.15.16:0/406367769 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77fa59c640 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de67480 0x55df6de893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de89900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 -- 172.21.15.16:0/406367769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 msgr2=0x55df6de89900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de89900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 -- 172.21.15.16:0/406367769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de67480 msgr2=0x55df6de893c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de67480 0x55df6de893c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.559+0000 7f77f9d9b640 1 -- 172.21.15.16:0/406367769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6de71520 con 0x55df6de66b60 2024-09-15T22:41:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.560+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de70a40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55df6de84360 tx=0x7f77f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5002bb8089182b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.560+0000 7f77e3fff640 1 -- 172.21.15.16:0/406367769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011690 con 0x55df6de66b60 2024-09-15T22:41:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.560+0000 7f77e3fff640 1 -- 172.21.15.16:0/406367769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f0026040 con 0x55df6de66b60 2024-09-15T22:41:38.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.560+0000 7f77e3fff640 1 -- 172.21.15.16:0/406367769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0023c90 con 0x55df6de66b60 2024-09-15T22:41:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- 172.21.15.16:0/406367769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 msgr2=0x55df6de70a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de70a40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55df6de84360 tx=0x7f77f0004730 comp rx=0 tx=0).stop 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- 172.21.15.16:0/406367769 shutdown_connections 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de71d40 0x55df6de89900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de67480 0x55df6de893c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 --2- 172.21.15.16:0/406367769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de70a40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- 172.21.15.16:0/406367769 >> 172.21.15.16:0/406367769 conn(0x55df6de630f0 msgr2=0x55df6de70330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- 172.21.15.16:0/406367769 shutdown_connections 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- 172.21.15.16:0/406367769 wait complete. 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- start start 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 Processor -- start 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de75c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de67480 0x55df6de76140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de76680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1af650 con 0x55df6de66b60 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1b0ac0 con 0x55df6de71d40 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f77fc1afd20 con 0x55df6de67480 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.561+0000 7f7802d7b740 10 client.0 _ll_register_callbacks cb 0x7f77f0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de75c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de75c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41082/0 (socket says 172.21.15.16:41082) 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 -- 172.21.15.16:0/3106221980 learned_addr learned my addr 172.21.15.16:0/3106221980 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77fa59c640 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de67480 0x55df6de76140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 -- 172.21.15.16:0/3106221980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de67480 msgr2=0x55df6de76140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de67480 0x55df6de76140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 -- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 msgr2=0x55df6de76680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de76680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 -- 172.21.15.16:0/3106221980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6deeb560 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77f9d9b640 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de75c00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55df6dd82f80 tx=0x7f77f0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4597447185c12b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77e1ffb640 1 -- 172.21.15.16:0/3106221980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f0023d20 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f77e1ffb640 1 -- 172.21.15.16:0/3106221980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f77f0011690 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.562+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f0026040 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f77e1ffb640 1 -- 172.21.15.16:0/3106221980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77f0011890 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 10 client.16131 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df6deec7d0 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f77e1ffb640 1 -- 172.21.15.16:0/3106221980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77f0026040 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f77e1ffb640 1 -- 172.21.15.16:0/3106221980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f77f0029020 con 0x55df6de66b60 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 10 client.16131 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 20 client.16131 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6deecde0 0x55df6deef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 4 client.16131 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55df6ddd3d00 con 0x55df6deecde0 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.563+0000 7f77fa59c640 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6deecde0 0x55df6deef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.564+0000 7f77fa59c640 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6deecde0 0x55df6deef200 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec20f9ffa179c6b3 server_cookie=acaa71fa219a9e91 in_seq=0 out_seq=0 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.564+0000 7f77e1ffb640 10 client.16131 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.564+0000 7f77e1ffb640 1 -- 172.21.15.16:0/3106221980 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55df6ddd3d00 con 0x55df6deecde0 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.564+0000 7f77e1ffb640 10 client.16131 handle_command_reply: tid=0 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.564+0000 7f7802d7b740 1 client.16131 shutdown 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 msgr2=0x55df6de75c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de75c00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55df6dd82f80 tx=0x7f77f0009b30 comp rx=0 tx=0).stop 2024-09-15T22:41:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 shutdown_connections 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df6deecde0 0x55df6deef200 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f77e1ffb640 0 client.16131 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df6de71d40 0x55df6de76680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df6de67480 0x55df6de76140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 --2- 172.21.15.16:0/3106221980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df6de66b60 0x55df6de75c00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 >> 172.21.15.16:0/3106221980 conn(0x55df6de630f0 msgr2=0x55df6de62270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 shutdown_connections 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 1 -- 172.21.15.16:0/3106221980 wait complete. 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.565+0000 7f7802d7b740 20 client.16131 trim_cache size 0 max 16384 2024-09-15T22:41:38.571 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:41:38.572 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:41:38.572 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "8353af1a-eb0d-44f9-8eb7-29d538921743", 2024-09-15T22:41:38.572 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:41:38.572 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:41:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77e40a33b0 msgr2=0x7f77e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77e40a33b0 0x7f77e40a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f77f40b9910 tx=0x7f77f40095c0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 msgr2=0x7f77fc19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc19ef30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f77f4004af0 tx=0x7f77f402f5b0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 shutdown_connections 2024-09-15T22:41:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77fc102ef0 0x7f77fc19ef30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77fc102570 0x7f77fc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77e40a33b0 0x7f77e40a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 --2- 172.21.15.16:0/103296155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f77fc100e20 0x7f77fc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 >> 172.21.15.16:0/103296155 conn(0x7f77fc0fc7d0 msgr2=0x7f77fc0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 shutdown_connections 2024-09-15T22:41:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:38.566+0000 7f78009a2640 1 -- 172.21.15.16:0/103296155 wait complete. 2024-09-15T22:41:39.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:38 smithi130 ceph-mon[35742]: pgmap v99: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:39.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:38 smithi016 ceph-mon[31492]: pgmap v99: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:39.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:38 smithi059 ceph-mon[34872]: pgmap v99: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:39.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:41:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:39.893 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:39 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:41:39.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:39 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:41:39.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:39 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:41:39.894 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:39 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:41:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:39 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:41:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:39 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:41:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:39 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:41:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:39 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:41:40.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/2737651199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 msgr2=0x7fe6f010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/2737651199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f010eaf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009cd0 tx=0x7fe6e802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/2737651199 shutdown_connections 2024-09-15T22:41:40.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/2737651199 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f010eaf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/2737651199 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6f0100ca0 0x7fe6f0106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/2737651199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6f01002f0 0x7fe6f01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/2737651199 >> 172.21.15.16:0/2737651199 conn(0x7fe6f0077210 msgr2=0x7fe6f00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/2737651199 shutdown_connections 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.128+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/2737651199 wait complete. 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 Processor -- start 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 -- start start 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6f01002f0 0x7fe6f01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6f0100ca0 0x7fe6f0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f0112000 con 0x7fe6f0106f50 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f0111e80 con 0x7fe6f01002f0 2024-09-15T22:41:40.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f6f9f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f0112180 con 0x7fe6f0100ca0 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f553c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f553c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41100/0 (socket says 172.21.15.16:41100) 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f553c640 1 -- 172.21.15.16:0/4222716254 learned_addr learned my addr 172.21.15.16:0/4222716254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6f01002f0 0x7fe6f01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.129+0000 7fe6effff640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6f0100ca0 0x7fe6f0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f553c640 1 -- 172.21.15.16:0/4222716254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6f0100ca0 msgr2=0x7fe6f0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f553c640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6f0100ca0 0x7fe6f0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f553c640 1 -- 172.21.15.16:0/4222716254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6f01002f0 msgr2=0x7fe6f01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f553c640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6f01002f0 0x7fe6f01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f553c640 1 -- 172.21.15.16:0/4222716254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f019d490 con 0x7fe6f0106f50 2024-09-15T22:41:40.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6effff640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6f0100ca0 0x7fe6f0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:41:40.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f553c640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f019cdb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8005cc0 tx=0x7fe6e8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:40.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6edffb640 1 -- 172.21.15.16:0/4222716254 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8004040 con 0x7fe6f0106f50 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6edffb640 1 -- 172.21.15.16:0/4222716254 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e80046c0 con 0x7fe6f0106f50 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f019d720 con 0x7fe6f0106f50 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6edffb640 1 -- 172.21.15.16:0/4222716254 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8046480 con 0x7fe6f0106f50 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.130+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f01a5060 con 0x7fe6f0106f50 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.132+0000 7fe6edffb640 1 -- 172.21.15.16:0/4222716254 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe6e80305a0 con 0x7fe6f0106f50 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 Processor -- start 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 -- start start 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070f3a0 0x55c18070f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c18071e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180720e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c1806752e0 con 0x55c18071ea90 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c18061cce0 con 0x55c18070fcc0 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f9378740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c1806712a0 con 0x55c18070f3a0 2024-09-15T22:41:40.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6edffb640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6d80a3430 0x7fe6d80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180720e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f4d3b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c18071e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6f553c640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6d80a3430 0x7fe6d80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180720e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41114/0 (socket says 172.21.15.16:41114) 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6effff640 1 -- 172.21.15.16:0/1439684450 learned_addr learned my addr 172.21.15.16:0/1439684450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.133+0000 7fe6edffb640 1 -- 172.21.15.16:0/4222716254 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe6e802e030 con 0x7fe6f0106f50 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6effff640 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070f3a0 0x55c18070f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6f4d3b640 1 -- 172.21.15.16:0/1439684450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070f3a0 msgr2=0x55c18070f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070f3a0 0x55c18070f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6f4d3b640 1 -- 172.21.15.16:0/1439684450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 msgr2=0x55c180720e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180720e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6f4d3b640 1 -- 172.21.15.16:0/1439684450 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c180601440 con 0x55c18070fcc0 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6f553c640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6d80a3430 0x7fe6d80a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80bd740 tx=0x7fe6e80a8040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:40.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c18071e550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000bbf0 tx=0x7fe6e00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fa2e0741df0f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:40.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6d6ffd640 1 -- 172.21.15.16:0/1439684450 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0004030 con 0x55c18070fcc0 2024-09-15T22:41:40.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6d6ffd640 1 -- 172.21.15.16:0/1439684450 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e00041d0 con 0x55c18070fcc0 2024-09-15T22:41:40.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.134+0000 7fe6d6ffd640 1 -- 172.21.15.16:0/1439684450 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0011680 con 0x55c18070fcc0 2024-09-15T22:41:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 -- 172.21.15.16:0/1439684450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 msgr2=0x55c18071e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c18071e550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000bbf0 tx=0x7fe6e00107a0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 -- 172.21.15.16:0/1439684450 shutdown_connections 2024-09-15T22:41:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180720e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c18071e550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1439684450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070f3a0 0x55c18070f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 -- 172.21.15.16:0/1439684450 >> 172.21.15.16:0/1439684450 conn(0x55c18070ba30 msgr2=0x55c18070be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 -- 172.21.15.16:0/1439684450 shutdown_connections 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.135+0000 7fe6f9378740 1 -- 172.21.15.16:0/1439684450 wait complete. 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 -- start start 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 Processor -- start 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c18071e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c18071b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c18071bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c1807216c0 con 0x55c18071ea90 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c180630050 con 0x55c18070f3a0 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c180721840 con 0x55c18070fcc0 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6f9378740 10 client.0 _ll_register_callbacks cb 0x55c180670160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:40.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.136+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c18071bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c18071bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41130/0 (socket says 172.21.15.16:41130) 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 -- 172.21.15.16:0/1531192229 learned_addr learned my addr 172.21.15.16:0/1531192229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c18071b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c18071e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 -- 172.21.15.16:0/1531192229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 msgr2=0x55c18071b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c18071b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 -- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 msgr2=0x55c18071e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c18071e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 -- 172.21.15.16:0/1531192229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1803e63c0 con 0x55c18071ea90 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6effff640 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c18071bcf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400d010 tx=0x7fe6e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebd3468d4849b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6cffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6cffff640 1 -- 172.21.15.16:0/1531192229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e401e070 con 0x55c18071ea90 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6cffff640 1 -- 172.21.15.16:0/1531192229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe6e40175e0 con 0x55c18071ea90 2024-09-15T22:41:40.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e00041d0 con 0x55c18071ea90 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6cffff640 1 -- 172.21.15.16:0/1531192229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6e4027050 con 0x55c18071ea90 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6f9378740 10 client.16143 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.137+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c18072d140 con 0x55c18071ea90 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6cffff640 1 -- 172.21.15.16:0/1531192229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e00041d0 con 0x55c18071ea90 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6cffff640 1 -- 172.21.15.16:0/1531192229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe6e4021020 con 0x55c18071ea90 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6f9378740 10 client.16143 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6f9378740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6f9378740 20 client.16143 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c18072e600 0x55c1807309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6f9378740 4 client.16143 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c180682850 con 0x55c18072e600 2024-09-15T22:41:40.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.138+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c18072e600 0x55c1807309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.139+0000 7fe6cffff640 10 client.16143 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.139+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c18072e600 0x55c1807309f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f9ce2216b3c7c8 server_cookie=e01ae29f09c096f9 in_seq=0 out_seq=0 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.139+0000 7fe6cffff640 1 -- 172.21.15.16:0/1531192229 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c180682850 con 0x55c18072e600 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.139+0000 7fe6cffff640 10 client.16143 handle_command_reply: tid=0 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 client.16143 shutdown 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 msgr2=0x55c18071bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c18071bcf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400d010 tx=0x7fe6e40046c0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 shutdown_connections 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c18072e600 0x55c1807309f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c18071bcf0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c18071b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 --2- 172.21.15.16:0/1531192229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c18071e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 >> 172.21.15.16:0/1531192229 conn(0x55c18070ba30 msgr2=0x55c180711790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:40.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6cffff640 0 client.16143 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 shutdown_connections 2024-09-15T22:41:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 1 -- 172.21.15.16:0/1531192229 wait complete. 2024-09-15T22:41:40.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.140+0000 7fe6f9378740 20 client.16143 trim_cache size 0 max 16384 2024-09-15T22:41:40.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.157+0000 7fe6f9378740 1 Processor -- start 2024-09-15T22:41:40.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.157+0000 7fe6f9378740 1 -- start start 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18070f3a0 0x55c18072a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c1807362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18071ea90 0x55c180736800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6f9378740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f01af5b0 con 0x55c18070f3a0 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6f9378740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f01ace80 con 0x55c18070fcc0 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6f9378740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f01aacd0 con 0x55c18071ea90 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18070f3a0 0x55c18072a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18070f3a0 0x55c18072a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41138/0 (socket says 172.21.15.16:41138) 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 -- 172.21.15.16:0/53398169 learned_addr learned my addr 172.21.15.16:0/53398169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c1807362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 -- 172.21.15.16:0/53398169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18071ea90 msgr2=0x55c180736800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18071ea90 0x55c180736800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 -- 172.21.15.16:0/53398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 msgr2=0x55c1807362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c1807362c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 -- 172.21.15.16:0/53398169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c18071e7e0 con 0x55c18070f3a0 2024-09-15T22:41:40.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.158+0000 7fe6effff640 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18070f3a0 0x55c18072a960 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006dd0 tx=0x7fe6e400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e8907ff92eadd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:40.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6d67fc640 1 -- 172.21.15.16:0/53398169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e401e040 con 0x55c18070f3a0 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6d67fc640 1 -- 172.21.15.16:0/53398169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e400cce0 con 0x55c18070f3a0 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6d67fc640 1 -- 172.21.15.16:0/53398169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400f040 con 0x55c18070f3a0 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 -- 172.21.15.16:0/53398169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18070f3a0 msgr2=0x55c18072a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18070f3a0 0x55c18072a960 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4006dd0 tx=0x7fe6e400c9a0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 -- 172.21.15.16:0/53398169 shutdown_connections 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18071ea90 0x55c180736800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070fcc0 0x55c1807362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 --2- 172.21.15.16:0/53398169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18070f3a0 0x55c18072a960 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 -- 172.21.15.16:0/53398169 >> 172.21.15.16:0/53398169 conn(0x55c18070ba30 msgr2=0x55c180716d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 -- 172.21.15.16:0/53398169 shutdown_connections 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.159+0000 7fe6f9378740 1 -- 172.21.15.16:0/53398169 wait complete. 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 -- start start 2024-09-15T22:41:40.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 Processor -- start 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c1807270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c1807182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180718820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f01ab340 con 0x55c18071ea90 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f01ac7b0 con 0x55c18070f3a0 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f01aba10 con 0x55c18070fcc0 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.160+0000 7fe6f9378740 10 client.0 _ll_register_callbacks cb 0x7fe6e4006ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180718820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180718820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41144/0 (socket says 172.21.15.16:41144) 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 -- 172.21.15.16:0/291663957 learned_addr learned my addr 172.21.15.16:0/291663957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c1807270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c1807182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 -- 172.21.15.16:0/291663957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 msgr2=0x55c1807182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c1807182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 -- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 msgr2=0x55c1807270a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c1807270a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 -- 172.21.15.16:0/291663957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c180718ec0 con 0x55c18071ea90 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c1807182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6effff640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180718820 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4022aa0 tx=0x7fe6e4024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32eb55f2a502a720 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6cffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6cffff640 1 -- 172.21.15.16:0/291663957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400f040 con 0x55c18071ea90 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6cffff640 1 -- 172.21.15.16:0/291663957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe6e402bd60 con 0x55c18071ea90 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.161+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e400cce0 con 0x55c18071ea90 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6cffff640 1 -- 172.21.15.16:0/291663957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe6e402b3e0 con 0x55c18071ea90 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 10 client.16149 fetch_fsmap learned FSMap version 26 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c180799130 con 0x55c18071ea90 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6cffff640 1 -- 172.21.15.16:0/291663957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6e400cce0 con 0x55c18071ea90 2024-09-15T22:41:40.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6cffff640 1 -- 172.21.15.16:0/291663957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe6e4037020 con 0x55c18071ea90 2024-09-15T22:41:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 10 client.16149 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:41:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:41:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 20 client.16149 populate_metadata read hostname 'smithi016' 2024-09-15T22:41:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c1807996e0 0x55c18079bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 4 client.16149 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:41:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c18067cd60 con 0x55c1807996e0 2024-09-15T22:41:40.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.162+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c1807996e0 0x55c18079bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6cffff640 10 client.16149 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f4d3b640 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c1807996e0 0x55c18079bad0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f392cb0a8658dc84 server_cookie=e3fde2765249941b in_seq=0 out_seq=0 2024-09-15T22:41:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6cffff640 1 -- 172.21.15.16:0/291663957 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c18067cd60 con 0x55c1807996e0 2024-09-15T22:41:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6cffff640 10 client.16149 handle_command_reply: tid=0 2024-09-15T22:41:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 client.16149 shutdown 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 msgr2=0x55c180718820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180718820 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4022aa0 tx=0x7fe6e4024040 comp rx=0 tx=0).stop 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 shutdown_connections 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c1807996e0 0x55c18079bad0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c18071ea90 0x55c180718820 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c18070fcc0 0x55c1807182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 --2- 172.21.15.16:0/291663957 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c18070f3a0 0x55c1807270a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6cffff640 0 client.16149 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.163+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 >> 172.21.15.16:0/291663957 conn(0x55c18070ba30 msgr2=0x55c180716d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.164+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 shutdown_connections 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.164+0000 7fe6f9378740 1 -- 172.21.15.16:0/291663957 wait complete. 2024-09-15T22:41:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.164+0000 7fe6f9378740 20 client.16149 trim_cache size 0 max 16384 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: "8353af1a-eb0d-44f9-8eb7-29d538921743": { 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "8353af1a-eb0d-44f9-8eb7-29d538921743", 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:41:40.171 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:41:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.164+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6d80a3430 msgr2=0x7fe6d80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.164+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6d80a3430 0x7fe6d80a58f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80bd740 tx=0x7fe6e80a8040 comp rx=0 tx=0).stop 2024-09-15T22:41:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 msgr2=0x7fe6f019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f019cdb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8005cc0 tx=0x7fe6e8003970 comp rx=0 tx=0).stop 2024-09-15T22:41:40.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 shutdown_connections 2024-09-15T22:41:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6f0106f50 0x7fe6f019cdb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe6d80a3430 0x7fe6d80a58f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe6f0100ca0 0x7fe6f0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 --2- 172.21.15.16:0/4222716254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6f01002f0 0x7fe6f01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 >> 172.21.15.16:0/4222716254 conn(0x7fe6f0077210 msgr2=0x7fe6f0105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 shutdown_connections 2024-09-15T22:41:40.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:41:40.165+0000 7fe6f6f9f640 1 -- 172.21.15.16:0/4222716254 wait complete. 2024-09-15T22:41:40.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:39 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:41:40.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:39 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:41:40.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:39 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:41:40.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:39 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:41:40.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:40 smithi016 ceph-mon[31492]: pgmap v100: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:40.855 INFO:tasks.cephfs.filesystem:scrub status for tag:8353af1a-eb0d-44f9-8eb7-29d538921743 - {'path': '/', 'tag': '8353af1a-eb0d-44f9-8eb7-29d538921743', 'options': 'recursive,force'} 2024-09-15T22:41:41.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:40 smithi130 ceph-mon[35742]: pgmap v100: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:41.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:40 smithi059 ceph-mon[34872]: pgmap v100: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:43.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:42 smithi130 ceph-mon[35742]: pgmap v101: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:42 smithi016 ceph-mon[31492]: pgmap v101: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:43.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:42 smithi059 ceph-mon[34872]: pgmap v101: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:44.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:43 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:41:44.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:43 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:41:44.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:43 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:41:45.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:44 smithi130 ceph-mon[35742]: pgmap v102: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:45.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:44 smithi016 ceph-mon[31492]: pgmap v102: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:45.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:44 smithi059 ceph-mon[34872]: pgmap v102: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:47.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:46 smithi130 ceph-mon[35742]: pgmap v103: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:47.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:46 smithi016 ceph-mon[31492]: pgmap v103: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:47.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:46 smithi059 ceph-mon[34872]: pgmap v103: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:48 smithi130 ceph-mon[35742]: pgmap v104: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:49.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:48 smithi016 ceph-mon[31492]: pgmap v104: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:49.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:48 smithi059 ceph-mon[34872]: pgmap v104: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:51.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:50 smithi130 ceph-mon[35742]: pgmap v105: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:51.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:50 smithi016 ceph-mon[31492]: pgmap v105: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:50 smithi059 ceph-mon[34872]: pgmap v105: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:41:53.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:52 smithi130 ceph-mon[35742]: pgmap v106: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:53.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:52 smithi016 ceph-mon[31492]: pgmap v106: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:53.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:52 smithi059 ceph-mon[34872]: pgmap v106: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:41:55.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:54 smithi130 ceph-mon[35742]: pgmap v107: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:55.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:54 smithi016 ceph-mon[31492]: pgmap v107: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:55.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:54 smithi059 ceph-mon[34872]: pgmap v107: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:57.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:56 smithi016 ceph-mon[31492]: pgmap v108: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:57.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:56 smithi059 ceph-mon[34872]: pgmap v108: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:57.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:56 smithi130 ceph-mon[35742]: pgmap v108: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:41:59.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:41:58 smithi016 ceph-mon[31492]: pgmap v109: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:59.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:41:58 smithi059 ceph-mon[34872]: pgmap v109: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:41:59.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:41:58 smithi130 ceph-mon[35742]: pgmap v109: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:01.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:00 smithi016 ceph-mon[31492]: pgmap v110: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:01.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:00 smithi059 ceph-mon[34872]: pgmap v110: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:01.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:00 smithi130 ceph-mon[35742]: pgmap v110: 129 pgs: 129 active+clean; 21 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:03.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:02 smithi016 ceph-mon[31492]: pgmap v111: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:03.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:02 smithi059 ceph-mon[34872]: pgmap v111: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:03.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:02 smithi130 ceph-mon[35742]: pgmap v111: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:05.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:04 smithi016 ceph-mon[31492]: pgmap v112: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:42:05.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:04 smithi059 ceph-mon[34872]: pgmap v112: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:42:05.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:04 smithi130 ceph-mon[35742]: pgmap v112: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:42:07.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:06 smithi016 ceph-mon[31492]: pgmap v113: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:42:07.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:06 smithi059 ceph-mon[34872]: pgmap v113: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:42:07.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:06 smithi130 ceph-mon[35742]: pgmap v113: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:42:09.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:08 smithi016 ceph-mon[31492]: pgmap v114: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:09.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:08 smithi059 ceph-mon[34872]: pgmap v114: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:09.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:08 smithi130 ceph-mon[35742]: pgmap v114: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:10.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:42:11.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:10 smithi016 ceph-mon[31492]: pgmap v115: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:42:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:11.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:10 smithi059 ceph-mon[34872]: pgmap v115: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:42:11.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:10 smithi130 ceph-mon[35742]: pgmap v115: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:42:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 -- 172.21.15.16:0/2499507581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec101df0 msgr2=0x7f28ec1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 --2- 172.21.15.16:0/2499507581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec101df0 0x7f28ec1021d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc02a410 comp rx=0 tx=0).stop 2024-09-15T22:42:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 -- 172.21.15.16:0/2499507581 shutdown_connections 2024-09-15T22:42:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 --2- 172.21.15.16:0/2499507581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28ec106f20 0x7f28ec10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 --2- 172.21.15.16:0/2499507581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec1027a0 0x7f28ec1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 --2- 172.21.15.16:0/2499507581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec101df0 0x7f28ec1021d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 -- 172.21.15.16:0/2499507581 >> 172.21.15.16:0/2499507581 conn(0x7f28ec0fd8b0 msgr2=0x7f28ec0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 -- 172.21.15.16:0/2499507581 shutdown_connections 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.637+0000 7f28f265b640 1 -- 172.21.15.16:0/2499507581 wait complete. 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 Processor -- start 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 -- start start 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec101df0 0x7f28ec198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28ec1027a0 0x7f28ec198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec106f20 0x7f28ec19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec111f80 con 0x7f28ec106f20 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec111e00 con 0x7f28ec101df0 2024-09-15T22:42:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f265b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec112100 con 0x7f28ec1027a0 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f0bf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec106f20 0x7f28ec19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f0bf8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec106f20 0x7f28ec19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51868/0 (socket says 172.21.15.16:51868) 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f0bf8640 1 -- 172.21.15.16:0/3825088986 learned_addr learned my addr 172.21.15.16:0/3825088986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28ec1027a0 0x7f28ec198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28ebfff640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec101df0 0x7f28ec198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f0bf8640 1 -- 172.21.15.16:0/3825088986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28ec1027a0 msgr2=0x7f28ec198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f0bf8640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28ec1027a0 0x7f28ec198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f0bf8640 1 -- 172.21.15.16:0/3825088986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec101df0 msgr2=0x7f28ec198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.638+0000 7f28f0bf8640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec101df0 0x7f28ec198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.639+0000 7f28f0bf8640 1 -- 172.21.15.16:0/3825088986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ec19d3d0 con 0x7f28ec106f20 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.639+0000 7f28f0bf8640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec106f20 0x7f28ec19ccf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f28e400eaa0 tx=0x7f28e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.639+0000 7f28e97fa640 1 -- 172.21.15.16:0/3825088986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400cdb0 con 0x7f28ec106f20 2024-09-15T22:42:11.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.639+0000 7f28e97fa640 1 -- 172.21.15.16:0/3825088986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e4005020 con 0x7f28ec106f20 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.639+0000 7f28e97fa640 1 -- 172.21.15.16:0/3825088986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e40116b0 con 0x7f28ec106f20 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.639+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ec19d6c0 con 0x7f28ec106f20 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.639+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ec1a4fa0 con 0x7f28ec106f20 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.641+0000 7f28e97fa640 1 -- 172.21.15.16:0/3825088986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f28e4005560 con 0x7f28ec106f20 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.641+0000 7f28f4a34740 1 Processor -- start 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.641+0000 7f28f4a34740 1 -- start start 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a78b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a796140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a798a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28f4a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56158a6f12e0 con 0x56158a78bdc0 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28f4a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56158a698ce0 con 0x56158a796680 2024-09-15T22:42:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28f4a34740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56158a6ed2a0 con 0x56158a78b4a0 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a796140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a796140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51878/0 (socket says 172.21.15.16:51878) 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 -- 172.21.15.16:0/235576654 learned_addr learned my addr 172.21.15.16:0/235576654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a798a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28e97fa640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28d40a33b0 0x7f28d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a78b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 -- 172.21.15.16:0/235576654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 msgr2=0x56158a78b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a78b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 -- 172.21.15.16:0/235576654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 msgr2=0x56158a798a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a798a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 -- 172.21.15.16:0/235576654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56158a67d440 con 0x56158a78bdc0 2024-09-15T22:42:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a78b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28f0bf8640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28d40a33b0 0x7f28d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.642+0000 7f28ebfff640 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a796140 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eaec8676f13979a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28e97fa640 1 -- 172.21.15.16:0/3825088986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f28e40d0220 con 0x7f28ec106f20 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28d27fc640 1 -- 172.21.15.16:0/235576654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc03f070 con 0x56158a78bdc0 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28d27fc640 1 -- 172.21.15.16:0/235576654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28dc004030 con 0x56158a78bdc0 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28d27fc640 1 -- 172.21.15.16:0/235576654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc035070 con 0x56158a78bdc0 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f4a34740 1 -- 172.21.15.16:0/235576654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 msgr2=0x56158a796140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f4a34740 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a796140 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc005e80 comp rx=0 tx=0).stop 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f0bf8640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28d40a33b0 0x7f28d40a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28e4098600 tx=0x7f28e40a0040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f4a34740 1 -- 172.21.15.16:0/235576654 shutdown_connections 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f4a34740 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a798a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f4a34740 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a796140 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f4a34740 1 --2- 172.21.15.16:0/235576654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a78b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.643+0000 7f28f4a34740 1 -- 172.21.15.16:0/235576654 >> 172.21.15.16:0/235576654 conn(0x56158a787a30 msgr2=0x56158a787e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 -- 172.21.15.16:0/235576654 shutdown_connections 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 -- 172.21.15.16:0/235576654 wait complete. 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 -- start start 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 Processor -- start 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a795f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a793220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a793760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56158a799250 con 0x56158a78bdc0 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56158a6ac050 con 0x56158a796680 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56158a7993d0 con 0x56158a78b4a0 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28f4a34740 10 client.0 _ll_register_callbacks cb 0x56158a6ec160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.644+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a793220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a793220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51884/0 (socket says 172.21.15.16:51884) 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 -- 172.21.15.16:0/2135146555 learned_addr learned my addr 172.21.15.16:0/2135146555 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a795f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a793760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 -- 172.21.15.16:0/2135146555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 msgr2=0x56158a795f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a795f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 -- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 msgr2=0x56158a793760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a793760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 -- 172.21.15.16:0/2135146555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56158a4623c0 con 0x56158a78bdc0 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a793760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28ebfff640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a793220 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006010 tx=0x7f28dc0041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e14a71843a652a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28cb7fe640 1 -- 172.21.15.16:0/2135146555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc035070 con 0x56158a78bdc0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28cb7fe640 1 -- 172.21.15.16:0/2135146555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f28dc0317b0 con 0x56158a78bdc0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28dc004030 con 0x56158a78bdc0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28cb7fe640 1 -- 172.21.15.16:0/2135146555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28dc03f040 con 0x56158a78bdc0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28f4a34740 10 client.16161 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.645+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56158a7a4b30 con 0x56158a78bdc0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28cb7fe640 1 -- 172.21.15.16:0/2135146555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28dc004030 con 0x56158a78bdc0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28cb7fe640 1 -- 172.21.15.16:0/2135146555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f28dc004960 con 0x56158a78bdc0 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28f4a34740 10 client.16161 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28f4a34740 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28f4a34740 20 client.16161 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28f4a34740 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a7a5e80 0x56158a7a8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28f4a34740 4 client.16161 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56158a6fe850 con 0x56158a7a5e80 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.646+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a7a5e80 0x56158a7a8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a7a5e80 0x56158a7a8270 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a84c6f95c4d94d server_cookie=78ee07634297d18d in_seq=0 out_seq=0 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28cb7fe640 10 client.16161 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28cb7fe640 1 -- 172.21.15.16:0/2135146555 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56158a6fe850 con 0x56158a7a5e80 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28cb7fe640 10 client.16161 handle_command_reply: tid=0 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 client.16161 shutdown 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 msgr2=0x56158a793220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a793220 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006010 tx=0x7f28dc0041e0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 shutdown_connections 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a7a5e80 0x56158a7a8270 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a796680 0x56158a793760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78bdc0 0x56158a793220 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28cb7fe640 0 client.16161 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 --2- 172.21.15.16:0/2135146555 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a795f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.647+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 >> 172.21.15.16:0/2135146555 conn(0x56158a787a30 msgr2=0x56158a78d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.648+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 shutdown_connections 2024-09-15T22:42:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.648+0000 7f28f4a34740 1 -- 172.21.15.16:0/2135146555 wait complete. 2024-09-15T22:42:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.648+0000 7f28f4a34740 20 client.16161 trim_cache size 0 max 16384 2024-09-15T22:42:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 Processor -- start 2024-09-15T22:42:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 -- start start 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a791af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7ab290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a796680 0x56158a7ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1b17c0 con 0x56158a796680 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1af090 con 0x56158a78bdc0 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.665+0000 7f28f4a34740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1acee0 con 0x56158a78b4a0 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a796680 0x56158a7ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a796680 0x56158a7ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51894/0 (socket says 172.21.15.16:51894) 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 -- 172.21.15.16:0/3968742130 learned_addr learned my addr 172.21.15.16:0/3968742130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28ebfff640 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a791af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 -- 172.21.15.16:0/3968742130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 msgr2=0x56158a791af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a791af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 -- 172.21.15.16:0/3968742130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 msgr2=0x56158a7ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7ab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 -- 172.21.15.16:0/3968742130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56158a798db0 con 0x56158a796680 2024-09-15T22:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a796680 0x56158a7ab7d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f28e0000c00 tx=0x7f28e000ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f32ad6673794ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28d1ffb640 1 -- 172.21.15.16:0/3968742130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0004220 con 0x56158a796680 2024-09-15T22:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28d1ffb640 1 -- 172.21.15.16:0/3968742130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0012e00 con 0x56158a796680 2024-09-15T22:42:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.666+0000 7f28d1ffb640 1 -- 172.21.15.16:0/3968742130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e00103d0 con 0x56158a796680 2024-09-15T22:42:11.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 -- 172.21.15.16:0/3968742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a796680 msgr2=0x56158a7ab7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a796680 0x56158a7ab7d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f28e0000c00 tx=0x7f28e000ba70 comp rx=0 tx=0).stop 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 -- 172.21.15.16:0/3968742130 shutdown_connections 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a796680 0x56158a7ab7d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 --2- 172.21.15.16:0/3968742130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a78b4a0 0x56158a791af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 -- 172.21.15.16:0/3968742130 >> 172.21.15.16:0/3968742130 conn(0x56158a787a30 msgr2=0x56158a7952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 -- 172.21.15.16:0/3968742130 shutdown_connections 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 -- 172.21.15.16:0/3968742130 wait complete. 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.667+0000 7f28f4a34740 1 -- start start 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 1 Processor -- start 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78b4a0 0x56158a7ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a796680 0x56158a7a9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ad550 con 0x56158a78b4a0 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1ae9c0 con 0x56158a78bdc0 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f28ec1adc20 con 0x56158a796680 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28f4a34740 10 client.0 _ll_register_callbacks cb 0x7f28dc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78b4a0 0x56158a7ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78b4a0 0x56158a7ab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51908/0 (socket says 172.21.15.16:51908) 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 -- 172.21.15.16:0/1675871721 learned_addr learned my addr 172.21.15.16:0/1675871721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28ebfff640 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 -- 172.21.15.16:0/1675871721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a796680 msgr2=0x56158a7a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a796680 0x56158a7a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 -- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 msgr2=0x56158a7a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.668+0000 7f28eb7fe640 1 -- 172.21.15.16:0/1675871721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56158a7a9a40 con 0x56158a78b4a0 2024-09-15T22:42:11.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28eb7fe640 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78b4a0 0x56158a7ab9b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f28e0003a00 tx=0x7f28e001e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a66d16dfef5c18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28cb7fe640 1 -- 172.21.15.16:0/1675871721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e0012be0 con 0x56158a78b4a0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28e0012e00 con 0x56158a78b4a0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28cb7fe640 1 -- 172.21.15.16:0/1675871721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f28e0005040 con 0x56158a78b4a0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28cb7fe640 1 -- 172.21.15.16:0/1675871721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28e0005240 con 0x56158a78b4a0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28f4a34740 10 client.16167 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56158a8107a0 con 0x56158a78b4a0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28cb7fe640 1 -- 172.21.15.16:0/1675871721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e0012e00 con 0x56158a78b4a0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28cb7fe640 1 -- 172.21.15.16:0/1675871721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f28e0030070 con 0x56158a78b4a0 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28f4a34740 10 client.16167 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28f4a34740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28f4a34740 20 client.16167 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:11.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.669+0000 7f28f4a34740 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a810e00 0x56158a813220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28f4a34740 4 client.16167 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56158a6f8d60 con 0x56158a810e00 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28ebfff640 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a810e00 0x56158a813220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28ebfff640 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a810e00 0x56158a813220 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b717bada1183883 server_cookie=9406da99cbda61e6 in_seq=0 out_seq=0 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28cb7fe640 10 client.16167 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28cb7fe640 1 -- 172.21.15.16:0/1675871721 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56158a6f8d60 con 0x56158a810e00 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28cb7fe640 10 client.16167 handle_command_reply: tid=0 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.670+0000 7f28f4a34740 1 client.16167 shutdown 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78b4a0 msgr2=0x56158a7ab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78b4a0 0x56158a7ab9b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f28e0003a00 tx=0x7f28e001e420 comp rx=0 tx=0).stop 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 shutdown_connections 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56158a810e00 0x56158a813220 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56158a796680 0x56158a7a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56158a78bdc0 0x56158a7a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 --2- 172.21.15.16:0/1675871721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56158a78b4a0 0x56158a7ab9b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 >> 172.21.15.16:0/1675871721 conn(0x56158a787a30 msgr2=0x56158a786350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28cb7fe640 0 client.16167 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 shutdown_connections 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 1 -- 172.21.15.16:0/1675871721 wait complete. 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f4a34740 20 client.16167 trim_cache size 0 max 16384 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:42:11.678 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:42:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28d40a33b0 msgr2=0x7f28d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28d40a33b0 0x7f28d40a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28e4098600 tx=0x7f28e40a0040 comp rx=0 tx=0).stop 2024-09-15T22:42:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec106f20 msgr2=0x7f28ec19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec106f20 0x7f28ec19ccf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f28e400eaa0 tx=0x7f28e400ef60 comp rx=0 tx=0).stop 2024-09-15T22:42:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 shutdown_connections 2024-09-15T22:42:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28ec106f20 0x7f28ec19ccf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f28d40a33b0 0x7f28d40a5870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f28ec1027a0 0x7f28ec198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 --2- 172.21.15.16:0/3825088986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28ec101df0 0x7f28ec198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:11.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.671+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 >> 172.21.15.16:0/3825088986 conn(0x7f28ec0fd8b0 msgr2=0x7f28ec0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:11.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.672+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 shutdown_connections 2024-09-15T22:42:11.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:11.672+0000 7f28f265b640 1 -- 172.21.15.16:0/3825088986 wait complete. 2024-09-15T22:42:12.474 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:42:12.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:42:12.737 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:13.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:12 smithi016 ceph-mon[31492]: pgmap v116: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:13.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:12 smithi059 ceph-mon[34872]: pgmap v116: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:13.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:12 smithi130 ceph-mon[35742]: pgmap v116: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:42:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 -- 172.21.15.16:0/3140177728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4100a80 msgr2=0x7f40a4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/3140177728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4100a80 0x7f40a4100ee0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4094009930 tx=0x7f409402a420 comp rx=0 tx=0).stop 2024-09-15T22:42:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 -- 172.21.15.16:0/3140177728 shutdown_connections 2024-09-15T22:42:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/3140177728 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40a4101420 0x7f40a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/3140177728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4100a80 0x7f40a4100ee0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/3140177728 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a4106a80 0x7f40a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 -- 172.21.15.16:0/3140177728 >> 172.21.15.16:0/3140177728 conn(0x7f40a40fc7a0 msgr2=0x7f40a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 -- 172.21.15.16:0/3140177728 shutdown_connections 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.217+0000 7f40aaeb2640 1 -- 172.21.15.16:0/3140177728 wait complete. 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 Processor -- start 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 -- start start 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40a4100a80 0x7f40a4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a4101420 0x7f40a4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4106a80 0x7f40a419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4110e80 con 0x7f40a4106a80 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4110d00 con 0x7f40a4101420 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40aaeb2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4111000 con 0x7f40a4100a80 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4106a80 0x7f40a419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:13.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4106a80 0x7f40a419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51928/0 (socket says 172.21.15.16:51928) 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 -- 172.21.15.16:0/1868879507 learned_addr learned my addr 172.21.15.16:0/1868879507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a8c4e640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40a4100a80 0x7f40a4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a3fff640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a4101420 0x7f40a4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 -- 172.21.15.16:0/1868879507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40a4100a80 msgr2=0x7f40a4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40a4100a80 0x7f40a4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 -- 172.21.15.16:0/1868879507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a4101420 msgr2=0x7f40a4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a4101420 0x7f40a4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a944f640 1 -- 172.21.15.16:0/1868879507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a419d3d0 con 0x7f40a4106a80 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.218+0000 7f40a3fff640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a4101420 0x7f40a4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.219+0000 7f40a944f640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4106a80 0x7f40a419ccf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f409c00eaa0 tx=0x7f409c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.219+0000 7f40a1ffb640 1 -- 172.21.15.16:0/1868879507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c00cdb0 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.219+0000 7f40a1ffb640 1 -- 172.21.15.16:0/1868879507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c005020 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.219+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a419d6c0 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.219+0000 7f40a1ffb640 1 -- 172.21.15.16:0/1868879507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0116b0 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.219+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a41a4fa0 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.220+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a4100ee0 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.221+0000 7f40a1ffb640 1 -- 172.21.15.16:0/1868879507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f409c005560 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.222+0000 7f40a1ffb640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40900a33b0 0x7f40900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.222+0000 7f40a8c4e640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40900a33b0 0x7f40900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.222+0000 7f40a1ffb640 1 -- 172.21.15.16:0/1868879507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f409c0d0400 con 0x7f40a4106a80 2024-09-15T22:42:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.222+0000 7f40a8c4e640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40900a33b0 0x7f40900a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f408c005fd0 tx=0x7f408c005850 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.226+0000 7f40a1ffb640 1 -- 172.21.15.16:0/1868879507 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f409c016030 con 0x7f40a4106a80 2024-09-15T22:42:13.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.439+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40a4109a70 con 0x7f40a4106a80 2024-09-15T22:42:13.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.440+0000 7f40a1ffb640 1 -- 172.21.15.16:0/1868879507 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f409c09ca20 con 0x7f40a4106a80 2024-09-15T22:42:13.442 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:42:13.442 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:42:13.443 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:42:13.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40900a33b0 msgr2=0x7f40900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:13.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40900a33b0 0x7f40900a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f408c005fd0 tx=0x7f408c005850 comp rx=0 tx=0).stop 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4106a80 msgr2=0x7f40a419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4106a80 0x7f40a419ccf0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f409c00eaa0 tx=0x7f409c00ef60 comp rx=0 tx=0).stop 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 shutdown_connections 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40a4106a80 0x7f40a419ccf0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40a4101420 0x7f40a4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f40900a33b0 0x7f40900a5870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 --2- 172.21.15.16:0/1868879507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f40a4100a80 0x7f40a4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 >> 172.21.15.16:0/1868879507 conn(0x7f40a40fc7a0 msgr2=0x7f40a406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 shutdown_connections 2024-09-15T22:42:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:13.442+0000 7f40aaeb2640 1 -- 172.21.15.16:0/1868879507 wait complete. 2024-09-15T22:42:14.050 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:13 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1868879507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:42:14.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:42:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:13 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1868879507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:42:14.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:13 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1868879507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:42:14.344 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.947+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/3296825659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4100a80 msgr2=0x7f0eb4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.947+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/3296825659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4100a80 0x7f0eb4100ee0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009930 tx=0x7f0ea802a420 comp rx=0 tx=0).stop 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.947+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/3296825659 shutdown_connections 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.947+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/3296825659 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0eb4101420 0x7f0eb410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.947+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/3296825659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4100a80 0x7f0eb4100ee0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.947+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/3296825659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb4106a80 0x7f0eb4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.948+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/3296825659 >> 172.21.15.16:0/3296825659 conn(0x7f0eb40fc7a0 msgr2=0x7f0eb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.948+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/3296825659 shutdown_connections 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.948+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/3296825659 wait complete. 2024-09-15T22:42:14.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.948+0000 7f0eb9ed8640 1 Processor -- start 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.948+0000 7f0eb9ed8640 1 -- start start 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb9ed8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb4100a80 0x7f0eb4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb9ed8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4101420 0x7f0eb4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb9ed8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0eb4106a80 0x7f0eb419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb9ed8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4110e80 con 0x7f0eb4101420 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb9ed8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4110d00 con 0x7f0eb4100a80 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb9ed8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4111000 con 0x7f0eb4106a80 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4101420 0x7f0eb4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4101420 0x7f0eb4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51946/0 (socket says 172.21.15.16:51946) 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/2196299785 learned_addr learned my addr 172.21.15.16:0/2196299785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0eb4106a80 0x7f0eb419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb4100a80 0x7f0eb4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/2196299785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0eb4106a80 msgr2=0x7f0eb419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0eb4106a80 0x7f0eb419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/2196299785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb4100a80 msgr2=0x7f0eb4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb4100a80 0x7f0eb4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/2196299785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb419d3c0 con 0x7f0eb4101420 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb4100a80 0x7f0eb4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4101420 0x7f0eb4198950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009a50 tx=0x7f0ea8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb0ff9640 1 -- 172.21.15.16:0/2196299785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea803f030 con 0x7f0eb4101420 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb0ff9640 1 -- 172.21.15.16:0/2196299785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8003ce0 con 0x7f0eb4101420 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb419d650 con 0x7f0eb4101420 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.949+0000 7f0eb0ff9640 1 -- 172.21.15.16:0/2196299785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8035070 con 0x7f0eb4101420 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.950+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb41a4f90 con 0x7f0eb4101420 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.952+0000 7f0eb0ff9640 1 -- 172.21.15.16:0/2196299785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0ea8042030 con 0x7f0eb4101420 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.952+0000 7f0ebc2b1740 1 Processor -- start 2024-09-15T22:42:14.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.952+0000 7f0ebc2b1740 1 -- start start 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e39a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e44370 0x562fe3e46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562fe3d9f2f0 con 0x562fe3e395d0 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562fe3d46cf0 con 0x562fe3e3ab40 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562fe3d9b2b0 con 0x562fe3e44370 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb0ff9640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0e9c0a3430 0x7f0e9c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e39a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e39a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51956/0 (socket says 172.21.15.16:51956) 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 -- 172.21.15.16:0/856369627 learned_addr learned my addr 172.21.15.16:0/856369627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0e9c0a3430 0x7f0e9c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e44370 0x562fe3e46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb0ff9640 1 -- 172.21.15.16:0/2196299785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0ea80f0640 con 0x7f0eb4101420 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 -- 172.21.15.16:0/856369627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e44370 msgr2=0x562fe3e46760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e44370 0x562fe3e46760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 -- 172.21.15.16:0/856369627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 msgr2=0x562fe3e39090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e39090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 -- 172.21.15.16:0/856369627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe3d2b3a0 con 0x562fe3e395d0 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e39a10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400bb50 tx=0x7f0ea400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9a80dfaa9c4f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0e9c0a3430 0x7f0e9c0a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0077f0 tx=0x7f0eac007780 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.953+0000 7f0e99ffb640 1 -- 172.21.15.16:0/856369627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4023070 con 0x562fe3e395d0 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0e99ffb640 1 -- 172.21.15.16:0/856369627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea400be50 con 0x562fe3e395d0 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0e99ffb640 1 -- 172.21.15.16:0/856369627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea401c540 con 0x562fe3e395d0 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/856369627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 msgr2=0x562fe3e39a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e39a10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400bb50 tx=0x7f0ea400d710 comp rx=0 tx=0).stop 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/856369627 shutdown_connections 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e44370 0x562fe3e46760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e39a10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/856369627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/856369627 >> 172.21.15.16:0/856369627 conn(0x562fe3e02310 msgr2=0x562fe3e35c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/856369627 shutdown_connections 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.954+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/856369627 wait complete. 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 -- start start 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 Processor -- start 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e43d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e3ab40 0x562fe3e40fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 0x562fe3e415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562fe3e47070 con 0x562fe3e3ab40 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562fe3d5a060 con 0x562fe3e44370 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562fe3e471f0 con 0x562fe3e395d0 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0ebc2b1740 10 client.0 _ll_register_callbacks cb 0x562fe3aad8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:14.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e3ab40 0x562fe3e40fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e3ab40 0x562fe3e40fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51958/0 (socket says 172.21.15.16:51958) 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/749259016 learned_addr learned my addr 172.21.15.16:0/749259016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e43d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 0x562fe3e415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 msgr2=0x562fe3e43d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e43d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/749259016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 msgr2=0x562fe3e415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.955+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 0x562fe3e415d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0eb2ffd640 1 -- 172.21.15.16:0/749259016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe3b103c0 con 0x562fe3e3ab40 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e3ab40 0x562fe3e40fe0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80bc260 tx=0x7f0ea80b8330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd062cae8ca9eacb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0e92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0e92ffd640 1 -- 172.21.15.16:0/749259016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8049070 con 0x562fe3e3ab40 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0e92ffd640 1 -- 172.21.15.16:0/749259016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0ea803b460 con 0x562fe3e3ab40 2024-09-15T22:42:14.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea400be50 con 0x562fe3e3ab40 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0e92ffd640 1 -- 172.21.15.16:0/749259016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0ea80bcc60 con 0x562fe3e3ab40 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 10 client.16185 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fe3e52a40 con 0x562fe3e3ab40 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0e92ffd640 1 -- 172.21.15.16:0/749259016 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea400be50 con 0x562fe3e3ab40 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0e92ffd640 1 -- 172.21.15.16:0/749259016 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0ea80edc50 con 0x562fe3e3ab40 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 10 client.16185 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 10 client.16185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 20 client.16185 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3e53fd0 0x562fe3e56420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 4 client.16185 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.956+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe3dac860 con 0x562fe3e53fd0 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.957+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3e53fd0 0x562fe3e56420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.957+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3e53fd0 0x562fe3e56420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75fc06b7924b98cf server_cookie=a9a60393fdd47ffd in_seq=0 out_seq=0 2024-09-15T22:42:14.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.957+0000 7f0e92ffd640 10 client.16185 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0e92ffd640 1 -- 172.21.15.16:0/749259016 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fe3dac860 con 0x562fe3e53fd0 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0e92ffd640 10 client.16185 handle_command_reply: tid=0 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 client.16185 shutdown 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e3ab40 msgr2=0x562fe3e40fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e3ab40 0x562fe3e40fe0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80bc260 tx=0x7f0ea80b8330 comp rx=0 tx=0).stop 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 shutdown_connections 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3e53fd0 0x562fe3e56420 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 0x562fe3e415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e3ab40 0x562fe3e40fe0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/749259016 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e43d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 >> 172.21.15.16:0/749259016 conn(0x562fe3e02310 msgr2=0x562fe3e3b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0e92ffd640 0 client.16185 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 shutdown_connections 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/749259016 wait complete. 2024-09-15T22:42:14.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.958+0000 7f0ebc2b1740 20 client.16185 trim_cache size 0 max 16384 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.971+0000 7f0ebc2b1740 1 Processor -- start 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.971+0000 7f0ebc2b1740 1 -- start start 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e35e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e3ab40 0x562fe3e50310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 0x562fe3e408a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41af570 con 0x562fe3e395d0 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ace40 con 0x562fe3e44370 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41aac90 con 0x562fe3e3ab40 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e35e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e35e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51974/0 (socket says 172.21.15.16:51974) 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 -- 172.21.15.16:0/3561776919 learned_addr learned my addr 172.21.15.16:0/3561776919 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e3ab40 0x562fe3e50310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 -- 172.21.15.16:0/3561776919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e3ab40 msgr2=0x562fe3e50310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e3ab40 0x562fe3e50310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 -- 172.21.15.16:0/3561776919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 msgr2=0x562fe3e408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 0x562fe3e408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 -- 172.21.15.16:0/3561776919 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe3e3ff90 con 0x562fe3e395d0 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e35e10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0ea401eef0 tx=0x7f0ea4005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc5a925b6b425cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0e997fa640 1 -- 172.21.15.16:0/3561776919 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4039070 con 0x562fe3e395d0 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0e997fa640 1 -- 172.21.15.16:0/3561776919 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea400b700 con 0x562fe3e395d0 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.972+0000 7f0e997fa640 1 -- 172.21.15.16:0/3561776919 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4023040 con 0x562fe3e395d0 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/3561776919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 msgr2=0x562fe3e35e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e35e10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0ea401eef0 tx=0x7f0ea4005e50 comp rx=0 tx=0).stop 2024-09-15T22:42:14.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/3561776919 shutdown_connections 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e44370 0x562fe3e408a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e3ab40 0x562fe3e50310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/3561776919 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e395d0 0x562fe3e35e10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/3561776919 >> 172.21.15.16:0/3561776919 conn(0x562fe3e02310 msgr2=0x562fe3e35700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/3561776919 shutdown_connections 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.973+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/3561776919 wait complete. 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 -- start start 2024-09-15T22:42:14.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 Processor -- start 2024-09-15T22:42:14.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e4cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e47e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e44370 0x562fe3e483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ab300 con 0x562fe3e44370 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ac770 con 0x562fe3e3ab40 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ab9d0 con 0x562fe3e395d0 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0ebc2b1740 10 client.0 _ll_register_callbacks cb 0x7f0ea8004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e44370 0x562fe3e483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e44370 0x562fe3e483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51984/0 (socket says 172.21.15.16:51984) 2024-09-15T22:42:14.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0eb3fff640 1 -- 172.21.15.16:0/383520213 learned_addr learned my addr 172.21.15.16:0/383520213 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e4cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.974+0000 7f0eb2ffd640 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e47e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0eb3fff640 1 -- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 msgr2=0x562fe3e4cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e4cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0eb3fff640 1 -- 172.21.15.16:0/383520213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 msgr2=0x562fe3e47e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e47e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0eb3fff640 1 -- 172.21.15.16:0/383520213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe3e48ba0 con 0x562fe3e44370 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0eb3fff640 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e44370 0x562fe3e483a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007310 tx=0x7f0eac007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c647f90426e68a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0e92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0e92ffd640 1 -- 172.21.15.16:0/383520213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac023460 con 0x562fe3e44370 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0e92ffd640 1 -- 172.21.15.16:0/383520213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0eac023600 con 0x562fe3e44370 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea400b700 con 0x562fe3e44370 2024-09-15T22:42:14.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0e92ffd640 1 -- 172.21.15.16:0/383520213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0eac031050 con 0x562fe3e44370 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0ebc2b1740 10 client.16191 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562fe3ebec40 con 0x562fe3e44370 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.975+0000 7f0e92ffd640 1 -- 172.21.15.16:0/383520213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea400b700 con 0x562fe3e44370 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0e92ffd640 1 -- 172.21.15.16:0/383520213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0eac0118b0 con 0x562fe3e44370 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0ebc2b1740 10 client.16191 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0ebc2b1740 10 client.16191 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0ebc2b1740 20 client.16191 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3ebf410 0x562fe3ec1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0ebc2b1740 4 client.16191 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fe3da6d70 con 0x562fe3ebf410 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3ebf410 0x562fe3ec1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0eb37fe640 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3ebf410 0x562fe3ec1830 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44cb45bae771e492 server_cookie=e9fb1ef3a3714f07 in_seq=0 out_seq=0 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.976+0000 7f0e92ffd640 10 client.16191 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0e92ffd640 1 -- 172.21.15.16:0/383520213 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fe3da6d70 con 0x562fe3ebf410 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0e92ffd640 10 client.16191 handle_command_reply: tid=0 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 client.16191 shutdown 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e44370 msgr2=0x562fe3e483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e44370 0x562fe3e483a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007310 tx=0x7f0eac007de0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 shutdown_connections 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562fe3ebf410 0x562fe3ec1830 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fe3e44370 0x562fe3e483a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562fe3e3ab40 0x562fe3e47e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 --2- 172.21.15.16:0/383520213 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562fe3e395d0 0x562fe3e4cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 >> 172.21.15.16:0/383520213 conn(0x562fe3e02310 msgr2=0x562fe3e3b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0e92ffd640 0 client.16191 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 shutdown_connections 2024-09-15T22:42:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 1 -- 172.21.15.16:0/383520213 wait complete. 2024-09-15T22:42:14.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.977+0000 7f0ebc2b1740 20 client.16191 trim_cache size 0 max 16384 2024-09-15T22:42:14.984 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:42:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0e9c0a3430 msgr2=0x7f0e9c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0e9c0a3430 0x7f0e9c0a58f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0077f0 tx=0x7f0eac007780 comp rx=0 tx=0).stop 2024-09-15T22:42:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4101420 msgr2=0x7f0eb4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4101420 0x7f0eb4198950 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009a50 tx=0x7f0ea8004270 comp rx=0 tx=0).stop 2024-09-15T22:42:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 shutdown_connections 2024-09-15T22:42:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0e9c0a3430 0x7f0e9c0a58f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0eb4106a80 0x7f0eb419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0eb4101420 0x7f0eb4198950 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 --2- 172.21.15.16:0/2196299785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb4100a80 0x7f0eb4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 >> 172.21.15.16:0/2196299785 conn(0x7f0eb40fc7a0 msgr2=0x7f0eb4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 shutdown_connections 2024-09-15T22:42:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:14.978+0000 7f0eb9ed8640 1 -- 172.21.15.16:0/2196299785 wait complete. 2024-09-15T22:42:15.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:14 smithi016 ceph-mon[31492]: pgmap v117: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:42:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:14 smithi059 ceph-mon[34872]: pgmap v117: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:42:15.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:14 smithi130 ceph-mon[35742]: pgmap v117: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:42:15.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:42:16.148 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 -- 172.21.15.16:0/679438797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc101de0 msgr2=0x7f7dfc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 --2- 172.21.15.16:0/679438797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc101de0 0x7f7dfc1021c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009900 tx=0x7f7dec02a410 comp rx=0 tx=0).stop 2024-09-15T22:42:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 -- 172.21.15.16:0/679438797 shutdown_connections 2024-09-15T22:42:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 --2- 172.21.15.16:0/679438797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dfc106f10 0x7f7dfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 --2- 172.21.15.16:0/679438797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dfc102790 0x7f7dfc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 --2- 172.21.15.16:0/679438797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc101de0 0x7f7dfc1021c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 -- 172.21.15.16:0/679438797 >> 172.21.15.16:0/679438797 conn(0x7f7dfc0fd8c0 msgr2=0x7f7dfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 -- 172.21.15.16:0/679438797 shutdown_connections 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.581+0000 7f7e00d85640 1 -- 172.21.15.16:0/679438797 wait complete. 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 Processor -- start 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 -- start start 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dfc101de0 0x7f7dfc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc102790 0x7f7dfc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dfc106f10 0x7f7dfc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc111f70 con 0x7f7dfc102790 2024-09-15T22:42:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc111df0 con 0x7f7dfc106f10 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7e00d85640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1120f0 con 0x7f7dfc101de0 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.582+0000 7f7df9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc102790 0x7f7dfc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc102790 0x7f7dfc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52002/0 (socket says 172.21.15.16:52002) 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2783399916 learned_addr learned my addr 172.21.15.16:0/2783399916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dfc101de0 0x7f7dfc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dfc106f10 0x7f7dfc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2783399916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dfc101de0 msgr2=0x7f7dfc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dfc101de0 0x7f7dfc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2783399916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dfc106f10 msgr2=0x7f7dfc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dfc106f10 0x7f7dfc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2783399916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc19d480 con 0x7f7dfc102790 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc102790 0x7f7dfc198a10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c890 tx=0x7f7df000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:16.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7deb7fe640 1 -- 172.21.15.16:0/2783399916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0004220 con 0x7f7dfc102790 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7deb7fe640 1 -- 172.21.15.16:0/2783399916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df00043c0 con 0x7f7dfc102790 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc19d770 con 0x7f7dfc102790 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7deb7fe640 1 -- 172.21.15.16:0/2783399916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df00049e0 con 0x7f7dfc102790 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.583+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a5050 con 0x7f7dfc102790 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7deb7fe640 1 -- 172.21.15.16:0/2783399916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7df00079b0 con 0x7f7dfc102790 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 Processor -- start 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 -- start start 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fd2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93fdf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd93f382f0 con 0x55bd93fdd370 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bd93edfcf0 con 0x55bd93fd3b40 2024-09-15T22:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7e0315e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bd93f342b0 con 0x55bd93fd25d0 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7df9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93fdf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7deb7fe640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7de40a33b0 0x7f7de40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7dfa59c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fd2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7df9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93fdf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52004/0 (socket says 172.21.15.16:52004) 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.586+0000 7f7df9d9b640 1 -- 172.21.15.16:0/3646726204 learned_addr learned my addr 172.21.15.16:0/3646726204 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7de40a33b0 0x7f7de40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7deb7fe640 1 -- 172.21.15.16:0/2783399916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7df00d0030 con 0x7f7dfc102790 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7df9d9b640 1 -- 172.21.15.16:0/3646726204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 msgr2=0x55bd93fd2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fd2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7df9d9b640 1 -- 172.21.15.16:0/3646726204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 msgr2=0x55bd93fd2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fd2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7df9d9b640 1 -- 172.21.15.16:0/3646726204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd93ec43a0 con 0x55bd93fdd370 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fd2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93fdf760 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7df00058c0 tx=0x7f7df0098800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e313407e7da8987 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7de40a33b0 0x7f7de40a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7df400a780 tx=0x7f7df400a4d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7de8ff9640 1 -- 172.21.15.16:0/3646726204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df001d070 con 0x55bd93fdd370 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7de8ff9640 1 -- 172.21.15.16:0/3646726204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df009c060 con 0x55bd93fdd370 2024-09-15T22:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.587+0000 7f7de8ff9640 1 -- 172.21.15.16:0/3646726204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df00cd6a0 con 0x55bd93fdd370 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 -- 172.21.15.16:0/3646726204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 msgr2=0x55bd93fdf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93fdf760 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7df00058c0 tx=0x7f7df0098800 comp rx=0 tx=0).stop 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 -- 172.21.15.16:0/3646726204 shutdown_connections 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93fdf760 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fd2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 --2- 172.21.15.16:0/3646726204 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fd2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 -- 172.21.15.16:0/3646726204 >> 172.21.15.16:0/3646726204 conn(0x55bd93f9b310 msgr2=0x55bd93fcec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 -- 172.21.15.16:0/3646726204 shutdown_connections 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.588+0000 7f7e0315e740 1 -- 172.21.15.16:0/3646726204 wait complete. 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 -- start start 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 Processor -- start 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fdce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd3b40 0x55bd93fda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fda840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bd93fe0010 con 0x55bd93fd3b40 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bd93ef3060 con 0x55bd93fdd370 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55bd93fe0190 con 0x55bd93fd25d0 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7e0315e740 10 client.0 _ll_register_callbacks cb 0x55bd93c468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd3b40 0x55bd93fda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd3b40 0x55bd93fda0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52016/0 (socket says 172.21.15.16:52016) 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2139775551 learned_addr learned my addr 172.21.15.16:0/2139775551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fdce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fda840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2139775551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 msgr2=0x55bd93fdce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fdce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 msgr2=0x55bd93fda840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fda840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7df9d9b640 1 -- 172.21.15.16:0/2139775551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd93ca93c0 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.589+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fda840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd3b40 0x55bd93fda0b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7df00cd4d0 tx=0x7f7df009a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0f3f919df39ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7de1ffb640 1 -- 172.21.15.16:0/2139775551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df001d040 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7de1ffb640 1 -- 172.21.15.16:0/2139775551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7df0096dd0 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df009c060 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7de1ffb640 1 -- 172.21.15.16:0/2139775551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7df009b9c0 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 10 client.16203 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd93febb60 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7de1ffb640 1 -- 172.21.15.16:0/2139775551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df009c060 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7de1ffb640 1 -- 172.21.15.16:0/2139775551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7df009af20 con 0x55bd93fd3b40 2024-09-15T22:42:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 10 client.16203 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 10 client.16203 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 20 client.16203 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd93fef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 4 client.16203 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.590+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd93f45860 con 0x55bd93feceb0 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.591+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd93fef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.591+0000 7f7de1ffb640 10 client.16203 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.591+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd93fef2a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c163e7b70676f6c2 server_cookie=ff24673de15e69c5 in_seq=0 out_seq=0 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7de1ffb640 1 -- 172.21.15.16:0/2139775551 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd93f45860 con 0x55bd93feceb0 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7de1ffb640 10 client.16203 handle_command_reply: tid=0 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 client.16203 shutdown 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd3b40 msgr2=0x55bd93fda0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd3b40 0x55bd93fda0b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7df00cd4d0 tx=0x7f7df009a570 comp rx=0 tx=0).stop 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 shutdown_connections 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd93fef2a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7de1ffb640 0 client.16203 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fda840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd3b40 0x55bd93fda0b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 --2- 172.21.15.16:0/2139775551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fdce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 >> 172.21.15.16:0/2139775551 conn(0x55bd93f9b310 msgr2=0x55bd93fd4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 shutdown_connections 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 1 -- 172.21.15.16:0/2139775551 wait complete. 2024-09-15T22:42:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.592+0000 7f7e0315e740 20 client.16203 trim_cache size 0 max 16384 2024-09-15T22:42:16.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 Processor -- start 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 -- start start 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fe8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93ff22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1b1860 con 0x55bd93fdd370 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1af130 con 0x55bd93fd3b40 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7e0315e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1acf80 con 0x55bd93fd25d0 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93ff22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7dfad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93ff22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52028/0 (socket says 172.21.15.16:52028) 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7dfad9d640 1 -- 172.21.15.16:0/225008669 learned_addr learned my addr 172.21.15.16:0/225008669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.606+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fe8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7dfad9d640 1 -- 172.21.15.16:0/225008669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 msgr2=0x55bd93fced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7dfad9d640 1 -- 172.21.15.16:0/225008669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 msgr2=0x55bd93fe8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fe8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7dfad9d640 1 -- 172.21.15.16:0/225008669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd93fd3760 con 0x55bd93fdd370 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93ff22c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7df400a820 tx=0x7f7df400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeb599148e120cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7de3fff640 1 -- 172.21.15.16:0/225008669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4004d20 con 0x55bd93fdd370 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7de3fff640 1 -- 172.21.15.16:0/225008669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df4003970 con 0x55bd93fdd370 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.607+0000 7f7de3fff640 1 -- 172.21.15.16:0/225008669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4004310 con 0x55bd93fdd370 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 -- 172.21.15.16:0/225008669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 msgr2=0x55bd93ff22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93ff22c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7df400a820 tx=0x7f7df400a890 comp rx=0 tx=0).stop 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 -- 172.21.15.16:0/225008669 shutdown_connections 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fdd370 0x55bd93ff22c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fd3b40 0x55bd93fe8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 --2- 172.21.15.16:0/225008669 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd25d0 0x55bd93fced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 -- 172.21.15.16:0/225008669 >> 172.21.15.16:0/225008669 conn(0x55bd93f9b310 msgr2=0x55bd93fe8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:16.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 -- 172.21.15.16:0/225008669 shutdown_connections 2024-09-15T22:42:16.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 -- 172.21.15.16:0/225008669 wait complete. 2024-09-15T22:42:16.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 -- start start 2024-09-15T22:42:16.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.608+0000 7f7e0315e740 1 Processor -- start 2024-09-15T22:42:16.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd25d0 0x55bd94054010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd3b40 0x55bd94054550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7e0315e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7e0315e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1ad5f0 con 0x55bd93fd25d0 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7e0315e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1aea60 con 0x55bd93fdd370 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7e0315e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc1adcc0 con 0x55bd93fd3b40 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7e0315e740 10 client.0 _ll_register_callbacks cb 0x7f7df400adf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd25d0 0x55bd94054010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd25d0 0x55bd94054010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52040/0 (socket says 172.21.15.16:52040) 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 -- 172.21.15.16:0/553934989 learned_addr learned my addr 172.21.15.16:0/553934989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:16.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd3b40 0x55bd94054550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 -- 172.21.15.16:0/553934989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd3b40 msgr2=0x55bd94054550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd3b40 0x55bd94054550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 -- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 msgr2=0x55bd93fef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 -- 172.21.15.16:0/553934989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd93ff00c0 con 0x55bd93fd25d0 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfad9d640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7dfa59c640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd25d0 0x55bd94054010 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7dec02a930 tx=0x7f7dec031b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9783b4f431add323 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.609+0000 7f7de1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7de1ffb640 1 -- 172.21.15.16:0/553934989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec035040 con 0x55bd93fd25d0 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7de1ffb640 1 -- 172.21.15.16:0/553934989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7dec004300 con 0x55bd93fd25d0 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df4003970 con 0x55bd93fd25d0 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7de1ffb640 1 -- 172.21.15.16:0/553934989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7dec058050 con 0x55bd93fd25d0 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 10 client.16209 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd9405b7d0 con 0x55bd93fd25d0 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7de1ffb640 1 -- 172.21.15.16:0/553934989 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df4003970 con 0x55bd93fd25d0 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7de1ffb640 1 -- 172.21.15.16:0/553934989 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7dec04f6b0 con 0x55bd93fd25d0 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 10 client.16209 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 10 client.16209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 20 client.16209 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd9405df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 4 client.16209 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.610+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd93f3fd70 con 0x55bd93feceb0 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.611+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd9405df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.611+0000 7f7df9d9b640 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd9405df10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dba597b13a9875a server_cookie=9d46ce3bb202be37 in_seq=0 out_seq=0 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.611+0000 7f7de1ffb640 10 client.16209 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.611+0000 7f7de1ffb640 1 -- 172.21.15.16:0/553934989 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd93f3fd70 con 0x55bd93feceb0 2024-09-15T22:42:16.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.611+0000 7f7de1ffb640 10 client.16209 handle_command_reply: tid=0 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.611+0000 7f7e0315e740 1 client.16209 shutdown 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd25d0 msgr2=0x55bd94054010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd25d0 0x55bd94054010 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7dec02a930 tx=0x7f7dec031b80 comp rx=0 tx=0).stop 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 shutdown_connections 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55bd93feceb0 0x55bd9405df10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bd93fdd370 0x55bd93fef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55bd93fd3b40 0x55bd94054550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 --2- 172.21.15.16:0/553934989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bd93fd25d0 0x55bd94054010 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 >> 172.21.15.16:0/553934989 conn(0x55bd93f9b310 msgr2=0x55bd93fe8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7de1ffb640 0 client.16209 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 shutdown_connections 2024-09-15T22:42:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 1 -- 172.21.15.16:0/553934989 wait complete. 2024-09-15T22:42:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e0315e740 20 client.16209 trim_cache size 0 max 16384 2024-09-15T22:42:16.625 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7de40a33b0 msgr2=0x7f7de40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7de40a33b0 0x7f7de40a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7df400a780 tx=0x7f7df400a4d0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc102790 msgr2=0x7f7dfc198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc102790 0x7f7dfc198a10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7df000c890 tx=0x7f7df000cd50 comp rx=0 tx=0).stop 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 shutdown_connections 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7dfc106f10 0x7f7dfc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dfc102790 0x7f7dfc198a10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7de40a33b0 0x7f7de40a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 --2- 172.21.15.16:0/2783399916 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7dfc101de0 0x7f7dfc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.612+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 >> 172.21.15.16:0/2783399916 conn(0x7f7dfc0fd8c0 msgr2=0x7f7dfc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.613+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 shutdown_connections 2024-09-15T22:42:16.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:16.613+0000 7f7e00d85640 1 -- 172.21.15.16:0/2783399916 wait complete. 2024-09-15T22:42:17.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:16 smithi016 ceph-mon[31492]: pgmap v118: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:42:17.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:16 smithi059 ceph-mon[34872]: pgmap v118: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:42:17.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:16 smithi130 ceph-mon[35742]: pgmap v118: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:42:17.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:42:17.706 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 -- 172.21.15.16:0/3729541928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a01027a0 msgr2=0x7f24a01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 --2- 172.21.15.16:0/3729541928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a01027a0 0x7f24a01067f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2488009900 tx=0x7f248802a410 comp rx=0 tx=0).stop 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 -- 172.21.15.16:0/3729541928 shutdown_connections 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 --2- 172.21.15.16:0/3729541928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a0106f20 0x7f24a010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 --2- 172.21.15.16:0/3729541928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a01027a0 0x7f24a01067f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 --2- 172.21.15.16:0/3729541928 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24a0101df0 0x7f24a01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 -- 172.21.15.16:0/3729541928 >> 172.21.15.16:0/3729541928 conn(0x7f24a00fd8b0 msgr2=0x7f24a00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 -- 172.21.15.16:0/3729541928 shutdown_connections 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.175+0000 7f24a645c640 1 -- 172.21.15.16:0/3729541928 wait complete. 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 Processor -- start 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 -- start start 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a0101df0 0x7f24a01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a01027a0 0x7f24a0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24a0106f20 0x7f24a019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0111f80 con 0x7f24a0101df0 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0111e00 con 0x7f24a01027a0 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a645c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0112100 con 0x7f24a0106f20 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a0101df0 0x7f24a01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a0101df0 0x7f24a01984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52050/0 (socket says 172.21.15.16:52050) 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 -- 172.21.15.16:0/859441172 learned_addr learned my addr 172.21.15.16:0/859441172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f2497fff640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a01027a0 0x7f24a0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f24a49f9640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24a0106f20 0x7f24a019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 -- 172.21.15.16:0/859441172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24a0106f20 msgr2=0x7f24a019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24a0106f20 0x7f24a019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 -- 172.21.15.16:0/859441172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a01027a0 msgr2=0x7f24a0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a01027a0 0x7f24a0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f249ffff640 1 -- 172.21.15.16:0/859441172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a019d470 con 0x7f24a0101df0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.176+0000 7f2497fff640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a01027a0 0x7f24a0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.177+0000 7f249ffff640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a0101df0 0x7f24a01984c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f249000d770 tx=0x7f249000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.177+0000 7f249dffb640 1 -- 172.21.15.16:0/859441172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490017070 con 0x7f24a0101df0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.177+0000 7f249dffb640 1 -- 172.21.15.16:0/859441172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2490005500 con 0x7f24a0101df0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.177+0000 7f249dffb640 1 -- 172.21.15.16:0/859441172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490004e00 con 0x7f24a0101df0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.177+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a019d760 con 0x7f24a0101df0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.177+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a01a5040 con 0x7f24a0101df0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.179+0000 7f24a8835740 1 Processor -- start 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.179+0000 7f249dffb640 1 -- 172.21.15.16:0/859441172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2490004fa0 con 0x7f24a0101df0 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.179+0000 7f24a8835740 1 -- start start 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a8835740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521acd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a8835740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521acda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a8835740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ada760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a8835740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557521a332f0 con 0x557521acd5d0 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a8835740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5575219dacf0 con 0x557521ad8370 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a8835740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557521a2f2b0 con 0x557521aceb40 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521acda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521acda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52052/0 (socket says 172.21.15.16:52052) 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 -- 172.21.15.16:0/2345518494 learned_addr learned my addr 172.21.15.16:0/2345518494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a49f9640 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521acd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f2497fff640 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ada760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 -- 172.21.15.16:0/2345518494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 msgr2=0x557521acd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521acd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 -- 172.21.15.16:0/2345518494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 msgr2=0x557521ada760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ada760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 -- 172.21.15.16:0/2345518494 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575219bf3a0 con 0x557521acd5d0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f2497fff640 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ada760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249dffb640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f248c0a3430 0x7f248c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249ffff640 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521acda10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f24900cce60 tx=0x7f24900d8640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df21327a7c77258 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f24a49f9640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f248c0a3430 0x7f248c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.180+0000 7f249dffb640 1 -- 172.21.15.16:0/859441172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f24900cfb00 con 0x7f24a0101df0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24967fc640 1 -- 172.21.15.16:0/2345518494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490097d20 con 0x557521acd5d0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24967fc640 1 -- 172.21.15.16:0/2345518494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24900d8e40 con 0x557521acd5d0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24967fc640 1 -- 172.21.15.16:0/2345518494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249009b3e0 con 0x557521acd5d0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 -- 172.21.15.16:0/2345518494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 msgr2=0x557521acda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521acda10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f24900cce60 tx=0x7f24900d8640 comp rx=0 tx=0).stop 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a49f9640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f248c0a3430 0x7f248c0a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2498005fd0 tx=0x7f24980096e0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 -- 172.21.15.16:0/2345518494 shutdown_connections 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ada760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521acda10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 --2- 172.21.15.16:0/2345518494 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521acd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 -- 172.21.15.16:0/2345518494 >> 172.21.15.16:0/2345518494 conn(0x557521a96310 msgr2=0x557521ac9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 -- 172.21.15.16:0/2345518494 shutdown_connections 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.181+0000 7f24a8835740 1 -- 172.21.15.16:0/2345518494 wait complete. 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 -- start start 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 Processor -- start 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521acd5d0 0x557521ad7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557521adb070 con 0x557521ad8370 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5575219ee060 con 0x557521acd5d0 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557521adb1f0 con 0x557521aceb40 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a8835740 10 client.0 _ll_register_callbacks cb 0x5575217418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a49f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a49f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52068/0 (socket says 172.21.15.16:52068) 2024-09-15T22:42:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f24a49f9640 1 -- 172.21.15.16:0/132329158 learned_addr learned my addr 172.21.15.16:0/132329158 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.182+0000 7f249ffff640 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521acd5d0 0x557521ad7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f2497fff640 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a49f9640 1 -- 172.21.15.16:0/132329158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 msgr2=0x557521ad5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a49f9640 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a49f9640 1 -- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521acd5d0 msgr2=0x557521ad7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a49f9640 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521acd5d0 0x557521ad7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a49f9640 1 -- 172.21.15.16:0/132329158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575217a43c0 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a49f9640 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad5650 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f249801d850 tx=0x7f2498004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f28d2e165b8c87c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f247f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f247f7fe640 1 -- 172.21.15.16:0/132329158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249801e820 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f247f7fe640 1 -- 172.21.15.16:0/132329158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f249801e9c0 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557521a40860 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f247f7fe640 1 -- 172.21.15.16:0/132329158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f249801ebc0 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a8835740 10 client.16221 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.183+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557521ae6bf0 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f247f7fe640 1 -- 172.21.15.16:0/132329158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557521a40860 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f247f7fe640 1 -- 172.21.15.16:0/132329158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f249800f040 con 0x557521ad8370 2024-09-15T22:42:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f24a8835740 10 client.16221 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f24a8835740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f24a8835740 20 client.16221 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f24a8835740 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521ae8000 0x557521aea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f24a8835740 4 client.16221 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557521a3ad70 con 0x557521ae8000 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.184+0000 7f249ffff640 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521ae8000 0x557521aea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f247f7fe640 10 client.16221 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f249ffff640 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521ae8000 0x557521aea3f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b471deb06de176cd server_cookie=5b212f2652b9b8b4 in_seq=0 out_seq=0 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f247f7fe640 1 -- 172.21.15.16:0/132329158 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557521a3ad70 con 0x557521ae8000 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f247f7fe640 10 client.16221 handle_command_reply: tid=0 2024-09-15T22:42:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f24a8835740 1 client.16221 shutdown 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 msgr2=0x557521ad5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f24a8835740 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad5650 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f249801d850 tx=0x7f2498004a40 comp rx=0 tx=0).stop 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 shutdown_connections 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.185+0000 7f24a8835740 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521ae8000 0x557521aea3f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f24a8835740 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad5650 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f247f7fe640 0 client.16221 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f24a8835740 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f24a8835740 1 --2- 172.21.15.16:0/132329158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521acd5d0 0x557521ad7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 >> 172.21.15.16:0/132329158 conn(0x557521a96310 msgr2=0x557521acf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 shutdown_connections 2024-09-15T22:42:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f24a8835740 1 -- 172.21.15.16:0/132329158 wait complete. 2024-09-15T22:42:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.186+0000 7f24a8835740 20 client.16221 trim_cache size 0 max 16384 2024-09-15T22:42:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 Processor -- start 2024-09-15T22:42:18.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 -- start start 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521acd5d0 0x557521ad6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521aceb40 0x557521ad0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01b1770 con 0x557521ad8370 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01af040 con 0x557521aceb40 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a8835740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01ace90 con 0x557521acd5d0 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52070/0 (socket says 172.21.15.16:52070) 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 -- 172.21.15.16:0/1870896308 learned_addr learned my addr 172.21.15.16:0/1870896308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f249ffff640 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521acd5d0 0x557521ad6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f2497fff640 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521aceb40 0x557521ad0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 -- 172.21.15.16:0/1870896308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521acd5d0 msgr2=0x557521ad6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521acd5d0 0x557521ad6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 -- 172.21.15.16:0/1870896308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521aceb40 msgr2=0x557521ad0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521aceb40 0x557521ad0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 -- 172.21.15.16:0/1870896308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557521ad8100 con 0x557521ad8370 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f249ffff640 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521acd5d0 0x557521ad6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.199+0000 7f24a49f9640 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad0660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2498000f80 tx=0x7f249801d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb338fcf7f8b0ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f2495ffb640 1 -- 172.21.15.16:0/1870896308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249803c020 con 0x557521ad8370 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f2495ffb640 1 -- 172.21.15.16:0/1870896308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498009c60 con 0x557521ad8370 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f2495ffb640 1 -- 172.21.15.16:0/1870896308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24980296e0 con 0x557521ad8370 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 -- 172.21.15.16:0/1870896308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 msgr2=0x557521ad0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad0660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2498000f80 tx=0x7f249801d450 comp rx=0 tx=0).stop 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 -- 172.21.15.16:0/1870896308 shutdown_connections 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521ad8370 0x557521ad0660 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521aceb40 0x557521ad0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 --2- 172.21.15.16:0/1870896308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521acd5d0 0x557521ad6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 -- 172.21.15.16:0/1870896308 >> 172.21.15.16:0/1870896308 conn(0x557521a96310 msgr2=0x557521ad68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 -- 172.21.15.16:0/1870896308 shutdown_connections 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.200+0000 7f24a8835740 1 -- 172.21.15.16:0/1870896308 wait complete. 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 -- start start 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 Processor -- start 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521ae0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ad2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01ad500 con 0x557521acd5d0 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01ae970 con 0x557521ad8370 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f24a01adbd0 con 0x557521aceb40 2024-09-15T22:42:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f24a8835740 10 client.0 _ll_register_callbacks cb 0x7f249801d500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f249ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521ae0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f249ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521ae0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52074/0 (socket says 172.21.15.16:52074) 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.201+0000 7f249ffff640 1 -- 172.21.15.16:0/2231333319 learned_addr learned my addr 172.21.15.16:0/2231333319 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f24a49f9640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ad2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f2497fff640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f249ffff640 1 -- 172.21.15.16:0/2231333319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 msgr2=0x557521ad1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f249ffff640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f249ffff640 1 -- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 msgr2=0x557521ad2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f249ffff640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ad2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f249ffff640 1 -- 172.21.15.16:0/2231333319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557521ad2b40 con 0x557521acd5d0 2024-09-15T22:42:18.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f2497fff640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f249ffff640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521ae0c10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2490099d30 tx=0x7f2490009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec53306ce1db63d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f247f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f247f7fe640 1 -- 172.21.15.16:0/2231333319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490020040 con 0x557521acd5d0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2498009c60 con 0x557521acd5d0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f247f7fe640 1 -- 172.21.15.16:0/2231333319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f2490009cc0 con 0x557521acd5d0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f247f7fe640 1 -- 172.21.15.16:0/2231333319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f249009b5c0 con 0x557521acd5d0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f24a8835740 10 client.16227 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557521b5b4e0 con 0x557521acd5d0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.202+0000 7f247f7fe640 1 -- 172.21.15.16:0/2231333319 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498009c60 con 0x557521acd5d0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f247f7fe640 1 -- 172.21.15.16:0/2231333319 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f24900cd170 con 0x557521acd5d0 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f24a8835740 10 client.16227 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f24a8835740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f24a8835740 20 client.16227 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f24a8835740 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521b5b710 0x557521b5db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f24a8835740 4 client.16227 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575219b7170 con 0x557521b5b710 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f2497fff640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521b5b710 0x557521b5db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f2497fff640 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521b5b710 0x557521b5db00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fd577190fa6ec97 server_cookie=759f573089aec66a in_seq=0 out_seq=0 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.203+0000 7f247f7fe640 10 client.16227 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f247f7fe640 1 -- 172.21.15.16:0/2231333319 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575219b7170 con 0x557521b5b710 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f247f7fe640 10 client.16227 handle_command_reply: tid=0 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 client.16227 shutdown 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 msgr2=0x557521ae0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521ae0c10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2490099d30 tx=0x7f2490009c30 comp rx=0 tx=0).stop 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 shutdown_connections 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557521b5b710 0x557521b5db00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557521ad8370 0x557521ad2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557521aceb40 0x557521ad1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 --2- 172.21.15.16:0/2231333319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557521acd5d0 0x557521ae0c10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 >> 172.21.15.16:0/2231333319 conn(0x557521a96310 msgr2=0x557521ad70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f247f7fe640 0 client.16227 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 shutdown_connections 2024-09-15T22:42:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 1 -- 172.21.15.16:0/2231333319 wait complete. 2024-09-15T22:42:18.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.204+0000 7f24a8835740 20 client.16227 trim_cache size 0 max 16384 2024-09-15T22:42:18.216 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:42:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f248c0a3430 msgr2=0x7f248c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f248c0a3430 0x7f248c0a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2498005fd0 tx=0x7f24980096e0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a0101df0 msgr2=0x7f24a01984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:18.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a0101df0 0x7f24a01984c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f249000d770 tx=0x7f249000dc30 comp rx=0 tx=0).stop 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 shutdown_connections 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f248c0a3430 0x7f248c0a58f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f24a0106f20 0x7f24a019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a01027a0 0x7f24a0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 --2- 172.21.15.16:0/859441172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24a0101df0 0x7f24a01984c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 >> 172.21.15.16:0/859441172 conn(0x7f24a00fd8b0 msgr2=0x7f24a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 shutdown_connections 2024-09-15T22:42:18.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:18.205+0000 7f24a645c640 1 -- 172.21.15.16:0/859441172 wait complete. 2024-09-15T22:42:19.007 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:18 smithi016 ceph-mon[31492]: pgmap v119: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:42:19.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:18 smithi059 ceph-mon[34872]: pgmap v119: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:42:19.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:18 smithi130 ceph-mon[35742]: pgmap v119: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:42:20.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:42:20.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:42:20.310 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.820+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3372182307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 msgr2=0x7fe2bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.820+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3372182307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc1029b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009900 tx=0x7fe2b002a480 comp rx=0 tx=0).stop 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3372182307 shutdown_connections 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3372182307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2bc102ef0 0x7fe2bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3372182307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc1029b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3372182307 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc100e20 0x7fe2bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3372182307 >> 172.21.15.16:0/3372182307 conn(0x7fe2bc0fc7d0 msgr2=0x7fe2bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3372182307 shutdown_connections 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3372182307 wait complete. 2024-09-15T22:42:20.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 Processor -- start 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.821+0000 7fe2c18a0640 1 -- start start 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2c18a0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2bc100e20 0x7fe2bc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2c18a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2c18a0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc102ef0 0x7fe2bc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2c18a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc110f80 con 0x7fe2bc102570 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2c18a0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc110e00 con 0x7fe2bc100e20 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2c18a0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc111100 con 0x7fe2bc102ef0 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52088/0 (socket says 172.21.15.16:52088) 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3250993015 learned_addr learned my addr 172.21.15.16:0/3250993015 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2bb7fe640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc102ef0 0x7fe2bc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2baffd640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2bc100e20 0x7fe2bc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3250993015 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc102ef0 msgr2=0x7fe2bc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc102ef0 0x7fe2bc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3250993015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2bc100e20 msgr2=0x7fe2bc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2bc100e20 0x7fe2bc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3250993015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc1a6a10 con 0x7fe2bc102570 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc0fd1c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0006010 tx=0x7fe2b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.822+0000 7fe2c089e640 1 -- 172.21.15.16:0/3250993015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b003f070 con 0x7fe2bc102570 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.823+0000 7fe2c089e640 1 -- 172.21.15.16:0/3250993015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0031c30 con 0x7fe2bc102570 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.823+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc1a6ca0 con 0x7fe2bc102570 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.823+0000 7fe2c089e640 1 -- 172.21.15.16:0/3250993015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0035030 con 0x7fe2bc102570 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.823+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1a7180 con 0x7fe2bc102570 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.825+0000 7fe2c089e640 1 -- 172.21.15.16:0/3250993015 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe2b0042070 con 0x7fe2bc102570 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.825+0000 7fe2c3c79740 1 Processor -- start 2024-09-15T22:42:20.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.825+0000 7fe2c3c79740 1 -- start start 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.825+0000 7fe2c089e640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe2a40a3430 0x7fe2a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.825+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.825+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb1980d280 con 0x55fb198a7b60 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fb197b4b10 con 0x55fb198b2d40 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55fb19809240 con 0x55fb198a8480 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2bb7fe640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe2a40a3430 0x7fe2a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198a7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52096/0 (socket says 172.21.15.16:52096) 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 -- 172.21.15.16:0/1054193191 learned_addr learned my addr 172.21.15.16:0/1054193191 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2c089e640 1 -- 172.21.15.16:0/3250993015 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe2b00f07b0 con 0x7fe2bc102570 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 -- 172.21.15.16:0/1054193191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 msgr2=0x55fb198b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 -- 172.21.15.16:0/1054193191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 msgr2=0x55fb198b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 -- 172.21.15.16:0/1054193191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb197c11f0 con 0x55fb198a7b60 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2bb7fe640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe2a40a3430 0x7fe2a40a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005fd0 tx=0x7fe2b4009a60 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2baffd640 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198a7f40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00bcd0 tx=0x7fe2ac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c01e40e30667f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:20.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2a1ffb640 1 -- 172.21.15.16:0/1054193191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac005020 con 0x55fb198a7b60 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.826+0000 7fe2a1ffb640 1 -- 172.21.15.16:0/1054193191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac0047a0 con 0x55fb198a7b60 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2a1ffb640 1 -- 172.21.15.16:0/1054193191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0117a0 con 0x55fb198a7b60 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 -- 172.21.15.16:0/1054193191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 msgr2=0x55fb198a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198a7f40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00bcd0 tx=0x7fe2ac0107a0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 -- 172.21.15.16:0/1054193191 shutdown_connections 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/1054193191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198a7f40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 -- 172.21.15.16:0/1054193191 >> 172.21.15.16:0/1054193191 conn(0x55fb198a40f0 msgr2=0x55fb198a44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 -- 172.21.15.16:0/1054193191 shutdown_connections 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.827+0000 7fe2c3c79740 1 -- 172.21.15.16:0/1054193191 wait complete. 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 -- start start 2024-09-15T22:42:20.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 Processor -- start 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fb198b5910 con 0x55fb198a7b60 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fb197c7f10 con 0x55fb198b2d40 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55fb198b5a90 con 0x55fb198a8480 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2c3c79740 10 client.0 _ll_register_callbacks cb 0x55fb19790a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52106/0 (socket says 172.21.15.16:52106) 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2baffd640 1 -- 172.21.15.16:0/446103812 learned_addr learned my addr 172.21.15.16:0/446103812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.828+0000 7fe2bb7fe640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2baffd640 1 -- 172.21.15.16:0/446103812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 msgr2=0x55fb198b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2baffd640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2baffd640 1 -- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 msgr2=0x55fb198b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2baffd640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2baffd640 1 -- 172.21.15.16:0/446103812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb1957e6a0 con 0x55fb198a7b60 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2baffd640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198afc00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00b670 tx=0x7fe2ac004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47b13c826b6dcb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe29affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe29affd640 1 -- 172.21.15.16:0/446103812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac004c80 con 0x55fb198a7b60 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe29affd640 1 -- 172.21.15.16:0/446103812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe2ac004f90 con 0x55fb198a7b60 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2ac0047a0 con 0x55fb198a7b60 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe29affd640 1 -- 172.21.15.16:0/446103812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2ac005190 con 0x55fb198a7b60 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2c3c79740 10 client.16239 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb198c14f0 con 0x55fb198a7b60 2024-09-15T22:42:20.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.829+0000 7fe29affd640 1 -- 172.21.15.16:0/446103812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2ac0047a0 con 0x55fb198a7b60 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe29affd640 1 -- 172.21.15.16:0/446103812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe2ac01b430 con 0x55fb198a7b60 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2c3c79740 10 client.16239 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2c3c79740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2c3c79740 20 client.16239 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb198c29b0 0x55fb198c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2c3c79740 4 client.16239 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb1981a7f0 con 0x55fb198c29b0 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb198c29b0 0x55fb198c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe29affd640 10 client.16239 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.830+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb198c29b0 0x55fb198c4da0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda9a36ea1addad3 server_cookie=ddad12d324ad1f83 in_seq=0 out_seq=0 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe29affd640 1 -- 172.21.15.16:0/446103812 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb1981a7f0 con 0x55fb198c29b0 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe29affd640 10 client.16239 handle_command_reply: tid=0 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 client.16239 shutdown 2024-09-15T22:42:20.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 msgr2=0x55fb198afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198afc00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00b670 tx=0x7fe2ac004950 comp rx=0 tx=0).stop 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 shutdown_connections 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb198c29b0 0x55fb198c4da0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a8480 0x55fb198b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/446103812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a7b60 0x55fb198afc00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 >> 172.21.15.16:0/446103812 conn(0x55fb198a40f0 msgr2=0x55fb198a3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe29affd640 0 client.16239 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.831+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 shutdown_connections 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.832+0000 7fe2c3c79740 1 -- 172.21.15.16:0/446103812 wait complete. 2024-09-15T22:42:20.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.832+0000 7fe2c3c79740 20 client.16239 trim_cache size 0 max 16384 2024-09-15T22:42:20.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.849+0000 7fe2c3c79740 1 Processor -- start 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.849+0000 7fe2c3c79740 1 -- start start 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.849+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.849+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198ca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.849+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b39f0 con 0x55fb198a8480 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b12c0 con 0x55fb198b2d40 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1af110 con 0x55fb198a7b60 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52108/0 (socket says 172.21.15.16:52108) 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/2190901524 learned_addr learned my addr 172.21.15.16:0/2190901524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:20.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2baffd640 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2bb7fe640 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/2190901524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 msgr2=0x55fb198b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/2190901524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 msgr2=0x55fb198caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/2190901524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb198b5470 con 0x55fb198a8480 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198ca560 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00bc770 tx=0x7fe2b00d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f0e3b6c82a3712 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2a17fa640 1 -- 172.21.15.16:0/2190901524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0049040 con 0x55fb198a8480 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2a17fa640 1 -- 172.21.15.16:0/2190901524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0031de0 con 0x55fb198a8480 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.850+0000 7fe2a17fa640 1 -- 172.21.15.16:0/2190901524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00ede20 con 0x55fb198a8480 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 -- 172.21.15.16:0/2190901524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 msgr2=0x55fb198ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198ca560 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00bc770 tx=0x7fe2b00d0040 comp rx=0 tx=0).stop 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 -- 172.21.15.16:0/2190901524 shutdown_connections 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb198caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198ca560 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/2190901524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 -- 172.21.15.16:0/2190901524 >> 172.21.15.16:0/2190901524 conn(0x55fb198a40f0 msgr2=0x55fb198b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 -- 172.21.15.16:0/2190901524 shutdown_connections 2024-09-15T22:42:20.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 -- 172.21.15.16:0/2190901524 wait complete. 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.851+0000 7fe2c3c79740 1 -- start start 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 1 Processor -- start 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb1992b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1af780 con 0x55fb198a8480 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b0bf0 con 0x55fb198b2d40 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1afe50 con 0x55fb198a7b60 2024-09-15T22:42:20.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2c3c79740 10 client.0 _ll_register_callbacks cb 0x7fe2b0005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2baffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2ba7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198b62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52122/0 (socket says 172.21.15.16:52122) 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3495997690 learned_addr learned my addr 172.21.15.16:0/3495997690 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.852+0000 7fe2bb7fe640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb1992b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3495997690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 msgr2=0x55fb198b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 msgr2=0x55fb1992b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb1992b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2ba7fc640 1 -- 172.21.15.16:0/3495997690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb198b6bb0 con 0x55fb198a8480 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2bb7fe640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb1992b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:20.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2baffd640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2ba7fc640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198b62e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b00d5b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a84691dec17412 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe29affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe29affd640 1 -- 172.21.15.16:0/3495997690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0049050 con 0x55fb198a8480 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe29affd640 1 -- 172.21.15.16:0/3495997690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe2b00b8e50 con 0x55fb198a8480 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b0031de0 con 0x55fb198a8480 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe29affd640 1 -- 172.21.15.16:0/3495997690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe2b00d5930 con 0x55fb198a8480 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2c3c79740 10 client.16245 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fb1992cba0 con 0x55fb198a8480 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe29affd640 1 -- 172.21.15.16:0/3495997690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b0031de0 con 0x55fb198a8480 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.853+0000 7fe29affd640 1 -- 172.21.15.16:0/3495997690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe2b00ef040 con 0x55fb198a8480 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2c3c79740 10 client.16245 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2c3c79740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:20.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2c3c79740 20 client.16245 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb1992d2e0 0x55fb1992f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2c3c79740 4 client.16245 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fb19814d00 con 0x55fb1992d2e0 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2baffd640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb1992d2e0 0x55fb1992f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe2baffd640 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb1992d2e0 0x55fb1992f700 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355d7b84433b3bd7 server_cookie=a1d670ff8411f7 in_seq=0 out_seq=0 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.854+0000 7fe29affd640 10 client.16245 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe29affd640 1 -- 172.21.15.16:0/3495997690 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fb19814d00 con 0x55fb1992d2e0 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe29affd640 10 client.16245 handle_command_reply: tid=0 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 client.16245 shutdown 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 msgr2=0x55fb198b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198b62e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0000c00 tx=0x7fe2b00d5b10 comp rx=0 tx=0).stop 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 shutdown_connections 2024-09-15T22:42:20.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55fb1992d2e0 0x55fb1992f700 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fb198b2d40 0x55fb1992b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fb198a8480 0x55fb198b62e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 --2- 172.21.15.16:0/3495997690 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fb198a7b60 0x55fb198b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 >> 172.21.15.16:0/3495997690 conn(0x55fb198a40f0 msgr2=0x55fb198a3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe29affd640 0 client.16245 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 shutdown_connections 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.856+0000 7fe2c3c79740 1 -- 172.21.15.16:0/3495997690 wait complete. 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c3c79740 20 client.16245 trim_cache size 0 max 16384 2024-09-15T22:42:20.867 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:42:20.868 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:42:20.868 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4441e06e-47f9-4ef2-87a8-fa02deca4852", 2024-09-15T22:42:20.868 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:42:20.868 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe2a40a3430 msgr2=0x7fe2a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe2a40a3430 0x7fe2a40a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4005fd0 tx=0x7fe2b4009a60 comp rx=0 tx=0).stop 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 msgr2=0x7fe2bc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc0fd1c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0006010 tx=0x7fe2b0003c10 comp rx=0 tx=0).stop 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 shutdown_connections 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe2a40a3430 0x7fe2a40a58f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe2bc102ef0 0x7fe2bc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe2bc102570 0x7fe2bc0fd1c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 --2- 172.21.15.16:0/3250993015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2bc100e20 0x7fe2bc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.857+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 >> 172.21.15.16:0/3250993015 conn(0x7fe2bc0fc7d0 msgr2=0x7fe2bc06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.858+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 shutdown_connections 2024-09-15T22:42:20.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:20.858+0000 7fe2c18a0640 1 -- 172.21.15.16:0/3250993015 wait complete. 2024-09-15T22:42:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:20 smithi016 ceph-mon[31492]: pgmap v120: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:42:21.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:20 smithi059 ceph-mon[34872]: pgmap v120: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:42:21.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:20 smithi130 ceph-mon[35742]: pgmap v120: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:42:21.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:42:21.909 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:22.030 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:21 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:42:22.030 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:21 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:42:22.030 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:21 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:42:22.031 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:21 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:42:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:21 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:42:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:21 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:42:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:21 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:42:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:21 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:42:22.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:21 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:42:22.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:21 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:42:22.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:21 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:42:22.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:21 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:42:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.394+0000 7f1807ea1640 1 -- 172.21.15.16:0/3968599198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800100c10 msgr2=0x7f1800110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.394+0000 7f1807ea1640 1 --2- 172.21.15.16:0/3968599198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800100c10 0x7f1800110840 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009ca0 tx=0x7f17f802a950 comp rx=0 tx=0).stop 2024-09-15T22:42:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- 172.21.15.16:0/3968599198 shutdown_connections 2024-09-15T22:42:22.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 --2- 172.21.15.16:0/3968599198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800100c10 0x7f1800110840 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 --2- 172.21.15.16:0/3968599198 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1800102ea0 0x7f18001005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 --2- 172.21.15.16:0/3968599198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800102ac0 0x7f18001000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- 172.21.15.16:0/3968599198 >> 172.21.15.16:0/3968599198 conn(0x7f1800077210 msgr2=0x7f18000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- 172.21.15.16:0/3968599198 shutdown_connections 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- 172.21.15.16:0/3968599198 wait complete. 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 Processor -- start 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- start start 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1800100c10 0x7f180019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800102ac0 0x7f180019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800102ea0 0x7f180019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1800114160 con 0x7f1800102ac0 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1800113fe0 con 0x7f1800102ea0 2024-09-15T22:42:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.395+0000 7f1807ea1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18001142e0 con 0x7f1800100c10 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800102ac0 0x7f180019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f1805c3d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1800100c10 0x7f180019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800102ac0 0x7f180019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50674/0 (socket says 172.21.15.16:50674) 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 -- 172.21.15.16:0/264359111 learned_addr learned my addr 172.21.15.16:0/264359111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180643e640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800102ea0 0x7f180019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 -- 172.21.15.16:0/264359111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1800100c10 msgr2=0x7f180019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1800100c10 0x7f180019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 -- 172.21.15.16:0/264359111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800102ea0 msgr2=0x7f180019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800102ea0 0x7f180019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 -- 172.21.15.16:0/264359111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180019f470 con 0x7f1800102ac0 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f180543c640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800102ac0 0x7f180019aa00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c920 tx=0x7f17f000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f17f6ffd640 1 -- 172.21.15.16:0/264359111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0007bb0 con 0x7f1800102ac0 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f17f6ffd640 1 -- 172.21.15.16:0/264359111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f0007d50 con 0x7f1800102ac0 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180019f760 con 0x7f1800102ac0 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f17f6ffd640 1 -- 172.21.15.16:0/264359111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f00187d0 con 0x7f1800102ac0 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.396+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001a7040 con 0x7f1800102ac0 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f17f6ffd640 1 -- 172.21.15.16:0/264359111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f17f000f450 con 0x7f1800102ac0 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 Processor -- start 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 -- start start 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf714a0 0x560a8bf71880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf7c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf7ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a8bed72e0 con 0x560a8bf714a0 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560a8be7ece0 con 0x560a8bf7c680 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180a27a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560a8bed32a0 con 0x560a8bf71dc0 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180643e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf714a0 0x560a8bf71880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180643e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf714a0 0x560a8bf71880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50680/0 (socket says 172.21.15.16:50680) 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.399+0000 7f180643e640 1 -- 172.21.15.16:0/1551690600 learned_addr learned my addr 172.21.15.16:0/1551690600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180543c640 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf7ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f1805c3d640 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf7c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180643e640 1 -- 172.21.15.16:0/1551690600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 msgr2=0x560a8bf7c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180643e640 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf7c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180643e640 1 -- 172.21.15.16:0/1551690600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 msgr2=0x560a8bf7ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180643e640 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf7ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180643e640 1 -- 172.21.15.16:0/1551690600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a8be63440 con 0x560a8bf714a0 2024-09-15T22:42:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180543c640 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf7ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f17f6ffd640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17ec0a3430 0x7f17ec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f1805c3d640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17ec0a3430 0x7f17ec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f180643e640 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf714a0 0x560a8bf71880 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009930 tx=0x7f17f8009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752391f08ff8016b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f17f6ffd640 1 -- 172.21.15.16:0/264359111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f17f00d00b0 con 0x7f1800102ac0 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f17ebfff640 1 -- 172.21.15.16:0/1551690600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8034040 con 0x560a8bf714a0 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f17ebfff640 1 -- 172.21.15.16:0/1551690600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f8003970 con 0x560a8bf714a0 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.400+0000 7f17ebfff640 1 -- 172.21.15.16:0/1551690600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8004610 con 0x560a8bf714a0 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 -- 172.21.15.16:0/1551690600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf714a0 msgr2=0x560a8bf71880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf714a0 0x560a8bf71880 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009930 tx=0x7f17f8009960 comp rx=0 tx=0).stop 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f1805c3d640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17ec0a3430 0x7f17ec0a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x560a8bf6e890 tx=0x7f17fc00a2d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 -- 172.21.15.16:0/1551690600 shutdown_connections 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf7ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf7c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 --2- 172.21.15.16:0/1551690600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf714a0 0x560a8bf71880 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 -- 172.21.15.16:0/1551690600 >> 172.21.15.16:0/1551690600 conn(0x560a8bf6da30 msgr2=0x560a8bf6de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 -- 172.21.15.16:0/1551690600 shutdown_connections 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 -- 172.21.15.16:0/1551690600 wait complete. 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.401+0000 7f180a27a740 1 -- start start 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 1 Processor -- start 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf714a0 0x560a8bf7bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf79210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf79750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a8bf7f250 con 0x560a8bf71dc0 2024-09-15T22:42:22.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560a8be92050 con 0x560a8bf7c680 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560a8bf7f3d0 con 0x560a8bf714a0 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180a27a740 10 client.0 _ll_register_callbacks cb 0x560a8bed2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180543c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf79210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180543c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf79210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50692/0 (socket says 172.21.15.16:50692) 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180543c640 1 -- 172.21.15.16:0/962815459 learned_addr learned my addr 172.21.15.16:0/962815459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f1805c3d640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf714a0 0x560a8bf7bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180643e640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf79750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180543c640 1 -- 172.21.15.16:0/962815459 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf714a0 msgr2=0x560a8bf7bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180543c640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf714a0 0x560a8bf7bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180543c640 1 -- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 msgr2=0x560a8bf79750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.402+0000 7f180543c640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf79750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180543c640 1 -- 172.21.15.16:0/962815459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a8bc483c0 con 0x560a8bf71dc0 2024-09-15T22:42:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180643e640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf79750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180543c640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf79210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17f009bfc0 tx=0x7f17f0098350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94893ded2a5e9169 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f17e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f17e97fa640 1 -- 172.21.15.16:0/962815459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f001f300 con 0x560a8bf71dc0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f17e97fa640 1 -- 172.21.15.16:0/962815459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f17f000fa30 con 0x560a8bf71dc0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f8003970 con 0x560a8bf71dc0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f17e97fa640 1 -- 172.21.15.16:0/962815459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17f009c730 con 0x560a8bf71dc0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 10 client.16257 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a8bf8ac70 con 0x560a8bf71dc0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f17e97fa640 1 -- 172.21.15.16:0/962815459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f8003970 con 0x560a8bf71dc0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f17e97fa640 1 -- 172.21.15.16:0/962815459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17f00a4810 con 0x560a8bf71dc0 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 10 client.16257 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 20 client.16257 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bf8bfc0 0x560a8bf8e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 4 client.16257 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.403+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a8bee4850 con 0x560a8bf8bfc0 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.404+0000 7f1805c3d640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bf8bfc0 0x560a8bf8e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.404+0000 7f17e97fa640 10 client.16257 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.404+0000 7f1805c3d640 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bf8bfc0 0x560a8bf8e3b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f91775a9055055 server_cookie=a0dfd14088c9305a in_seq=0 out_seq=0 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f17e97fa640 1 -- 172.21.15.16:0/962815459 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a8bee4850 con 0x560a8bf8bfc0 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f17e97fa640 10 client.16257 handle_command_reply: tid=0 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 client.16257 shutdown 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 msgr2=0x560a8bf79210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf79210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17f009bfc0 tx=0x7f17f0098350 comp rx=0 tx=0).stop 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 shutdown_connections 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bf8bfc0 0x560a8bf8e3b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf7c680 0x560a8bf79750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf79210 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f17e97fa640 0 client.16257 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 --2- 172.21.15.16:0/962815459 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf714a0 0x560a8bf7bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 >> 172.21.15.16:0/962815459 conn(0x560a8bf6da30 msgr2=0x560a8bf6cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 shutdown_connections 2024-09-15T22:42:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 1 -- 172.21.15.16:0/962815459 wait complete. 2024-09-15T22:42:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.405+0000 7f180a27a740 20 client.16257 trim_cache size 0 max 16384 2024-09-15T22:42:22.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 Processor -- start 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 -- start start 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf7b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf93c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf7c680 0x560a8bf94180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b38e0 con 0x560a8bf7c680 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b11b0 con 0x560a8bf714a0 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180a27a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18001af000 con 0x560a8bf71dc0 2024-09-15T22:42:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf7c680 0x560a8bf94180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf7c680 0x560a8bf94180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50694/0 (socket says 172.21.15.16:50694) 2024-09-15T22:42:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 -- 172.21.15.16:0/3866319390 learned_addr learned my addr 172.21.15.16:0/3866319390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 -- 172.21.15.16:0/3866319390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 msgr2=0x560a8bf93c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180543c640 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf93c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f1805c3d640 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf7b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf93c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 -- 172.21.15.16:0/3866319390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 msgr2=0x560a8bf7b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf7b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.423+0000 7f180643e640 1 -- 172.21.15.16:0/3866319390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a8bf712d0 con 0x560a8bf7c680 2024-09-15T22:42:22.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180643e640 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf7c680 0x560a8bf94180 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17f80095c0 tx=0x7f17f802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b01a2a6c75964 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f17eb7fe640 1 -- 172.21.15.16:0/3866319390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f804a070 con 0x560a8bf7c680 2024-09-15T22:42:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f17eb7fe640 1 -- 172.21.15.16:0/3866319390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f8003970 con 0x560a8bf7c680 2024-09-15T22:42:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f17eb7fe640 1 -- 172.21.15.16:0/3866319390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8034050 con 0x560a8bf7c680 2024-09-15T22:42:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180a27a740 1 -- 172.21.15.16:0/3866319390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf7c680 msgr2=0x560a8bf94180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180a27a740 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf7c680 0x560a8bf94180 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17f80095c0 tx=0x7f17f802a950 comp rx=0 tx=0).stop 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180a27a740 1 -- 172.21.15.16:0/3866319390 shutdown_connections 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180a27a740 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf7c680 0x560a8bf94180 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180a27a740 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf71dc0 0x560a8bf93c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180a27a740 1 --2- 172.21.15.16:0/3866319390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf7b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.424+0000 7f180a27a740 1 -- 172.21.15.16:0/3866319390 >> 172.21.15.16:0/3866319390 conn(0x560a8bf6da30 msgr2=0x560a8bf7aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 -- 172.21.15.16:0/3866319390 shutdown_connections 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 -- 172.21.15.16:0/3866319390 wait complete. 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 -- start start 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 Processor -- start 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf84bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf81f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf7c680 0x560a8bf82480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18001af670 con 0x560a8bf71dc0 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f18001b0ae0 con 0x560a8bf714a0 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f18001afd40 con 0x560a8bf7c680 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.425+0000 7f180a27a740 10 client.0 _ll_register_callbacks cb 0x7f17f000c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf81f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf81f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:50700/0 (socket says 172.21.15.16:50700) 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 -- 172.21.15.16:0/1080106262 learned_addr learned my addr 172.21.15.16:0/1080106262 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f1805c3d640 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf84bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180643e640 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf7c680 0x560a8bf82480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 -- 172.21.15.16:0/1080106262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf7c680 msgr2=0x560a8bf82480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf7c680 0x560a8bf82480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 -- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 msgr2=0x560a8bf84bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf84bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 -- 172.21.15.16:0/1080106262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a8bf82b20 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180543c640 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf81f40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f17f00cd550 tx=0x7f17f000fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c202abef72d6ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f17e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f17e97fa640 1 -- 172.21.15.16:0/1080106262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0096da0 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f17e97fa640 1 -- 172.21.15.16:0/1080106262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f17f009b480 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17f8003970 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f17e97fa640 1 -- 172.21.15.16:0/1080106262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17f001f300 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180a27a740 10 client.16263 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.426+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a8bff6aa0 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f17e97fa640 1 -- 172.21.15.16:0/1080106262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17f8003970 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f17e97fa640 1 -- 172.21.15.16:0/1080106262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f17f00d3160 con 0x560a8bf71dc0 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f180a27a740 10 client.16263 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f180a27a740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f180a27a740 20 client.16263 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f180a27a740 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bff7190 0x560a8bff95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f180a27a740 4 client.16263 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a8bf79ae0 con 0x560a8bff7190 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.427+0000 7f1805c3d640 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bff7190 0x560a8bff95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f1805c3d640 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bff7190 0x560a8bff95b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc869bdb80adc23b server_cookie=c6121fc107804c73 in_seq=0 out_seq=0 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f17e97fa640 10 client.16263 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f17e97fa640 1 -- 172.21.15.16:0/1080106262 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560a8bf79ae0 con 0x560a8bff7190 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f17e97fa640 10 client.16263 handle_command_reply: tid=0 2024-09-15T22:42:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 client.16263 shutdown 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 msgr2=0x560a8bf81f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf81f40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f17f00cd550 tx=0x7f17f000fe30 comp rx=0 tx=0).stop 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 shutdown_connections 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560a8bff7190 0x560a8bff95b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560a8bf7c680 0x560a8bf82480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a8bf71dc0 0x560a8bf81f40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f17e97fa640 0 client.16263 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 --2- 172.21.15.16:0/1080106262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560a8bf714a0 0x560a8bf84bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 >> 172.21.15.16:0/1080106262 conn(0x560a8bf6da30 msgr2=0x560a8bf776b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 shutdown_connections 2024-09-15T22:42:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 1 -- 172.21.15.16:0/1080106262 wait complete. 2024-09-15T22:42:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.428+0000 7f180a27a740 20 client.16263 trim_cache size 0 max 16384 2024-09-15T22:42:22.439 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:42:22.439 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:42:22.439 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:42:22.439 INFO:teuthology.orchestra.run.smithi016.stdout: "4441e06e-47f9-4ef2-87a8-fa02deca4852": { 2024-09-15T22:42:22.439 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:42:22.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "4441e06e-47f9-4ef2-87a8-fa02deca4852", 2024-09-15T22:42:22.440 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:42:22.440 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:42:22.440 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:42:22.440 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:42:22.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17ec0a3430 msgr2=0x7f17ec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17ec0a3430 0x7f17ec0a58f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x560a8bf6e890 tx=0x7f17fc00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800102ac0 msgr2=0x7f180019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800102ac0 0x7f180019aa00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c920 tx=0x7f17f000cd20 comp rx=0 tx=0).stop 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 shutdown_connections 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1800102ea0 0x7f180019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1800102ac0 0x7f180019aa00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f17ec0a3430 0x7f17ec0a58f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 --2- 172.21.15.16:0/264359111 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1800100c10 0x7f180019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 >> 172.21.15.16:0/264359111 conn(0x7f1800077210 msgr2=0x7f18000fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 shutdown_connections 2024-09-15T22:42:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:22.429+0000 7f1807ea1640 1 -- 172.21.15.16:0/264359111 wait complete. 2024-09-15T22:42:23.084 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:22 smithi016 ceph-mon[31492]: pgmap v121: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:42:23.113 INFO:tasks.cephfs.filesystem:scrub status for tag:4441e06e-47f9-4ef2-87a8-fa02deca4852 - {'path': '/', 'tag': '4441e06e-47f9-4ef2-87a8-fa02deca4852', 'options': 'recursive,force'} 2024-09-15T22:42:23.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:22 smithi059 ceph-mon[34872]: pgmap v121: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:42:23.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:22 smithi130 ceph-mon[35742]: pgmap v121: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:42:24.180 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:23 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:42:24.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:23 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:42:24.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:23 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:42:25.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:24 smithi059 ceph-mon[34872]: pgmap v122: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:25.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:24 smithi130 ceph-mon[35742]: pgmap v122: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:25.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:24 smithi016 ceph-mon[31492]: pgmap v122: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:26.094 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:25 smithi059 ceph-mon[34872]: pgmap v123: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:26.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:25 smithi130 ceph-mon[35742]: pgmap v123: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:26.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:25 smithi016 ceph-mon[31492]: pgmap v123: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:28.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:28 smithi059 ceph-mon[34872]: pgmap v124: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:42:28.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:28 smithi130 ceph-mon[35742]: pgmap v124: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:42:28.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:28 smithi016 ceph-mon[31492]: pgmap v124: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:42:30.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:30 smithi059 ceph-mon[34872]: pgmap v125: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:30.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:30 smithi130 ceph-mon[35742]: pgmap v125: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:30.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:30 smithi016 ceph-mon[31492]: pgmap v125: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:42:32.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:32 smithi059 ceph-mon[34872]: pgmap v126: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:42:32.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:32 smithi130 ceph-mon[35742]: pgmap v126: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:42:32.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:32 smithi016 ceph-mon[31492]: pgmap v126: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:42:34.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:34 smithi059 ceph-mon[34872]: pgmap v127: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:34.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:34 smithi130 ceph-mon[35742]: pgmap v127: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:34.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:34 smithi016 ceph-mon[31492]: pgmap v127: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:36.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:36 smithi059 ceph-mon[34872]: pgmap v128: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:36.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:36 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:36.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:36 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:36.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:36 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:36.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:36 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:42:36.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:36 smithi130 ceph-mon[35742]: pgmap v128: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:36.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:36 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:36 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:36 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:36.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:36 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:42:36.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:36 smithi016 ceph-mon[31492]: pgmap v128: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:36.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:36 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:36.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:36 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:36.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:36 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:36.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:36 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:42:38.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:38 smithi059 ceph-mon[34872]: pgmap v129: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:38.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:38 smithi130 ceph-mon[35742]: pgmap v129: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:38.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:38 smithi016 ceph-mon[31492]: pgmap v129: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:40.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:40 smithi059 ceph-mon[34872]: pgmap v130: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:40.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:40 smithi130 ceph-mon[35742]: pgmap v130: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:40.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:40 smithi016 ceph-mon[31492]: pgmap v130: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:42.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:42 smithi059 ceph-mon[34872]: pgmap v131: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:42.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:42 smithi130 ceph-mon[35742]: pgmap v131: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:42.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:42 smithi016 ceph-mon[31492]: pgmap v131: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:44 smithi059 ceph-mon[34872]: pgmap v132: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:44 smithi130 ceph-mon[35742]: pgmap v132: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:44.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:44 smithi016 ceph-mon[31492]: pgmap v132: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:46.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:46 smithi059 ceph-mon[34872]: pgmap v133: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:46.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:46 smithi130 ceph-mon[35742]: pgmap v133: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:46.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:46 smithi016 ceph-mon[31492]: pgmap v133: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:48.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:48 smithi059 ceph-mon[34872]: pgmap v134: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:48.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:48 smithi130 ceph-mon[35742]: pgmap v134: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:48.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:48 smithi016 ceph-mon[31492]: pgmap v134: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:50.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:50 smithi059 ceph-mon[34872]: pgmap v135: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:50.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:50 smithi130 ceph-mon[35742]: pgmap v135: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:50.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:50 smithi016 ceph-mon[31492]: pgmap v135: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:52.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:52 smithi059 ceph-mon[34872]: pgmap v136: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:52.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:52 smithi130 ceph-mon[35742]: pgmap v136: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:52.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:52 smithi016 ceph-mon[31492]: pgmap v136: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:53.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:42:53.391 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.814+0000 7f6f9c9da640 1 -- 172.21.15.16:0/1478788443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 msgr2=0x7f6f98114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.814+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/1478788443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f98114570 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009900 tx=0x7f6f8c02a480 comp rx=0 tx=0).stop 2024-09-15T22:42:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.814+0000 7f6f9c9da640 1 -- 172.21.15.16:0/1478788443 shutdown_connections 2024-09-15T22:42:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.814+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/1478788443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f98114ab0 0x7f6f98116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.814+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/1478788443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f98114570 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.814+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/1478788443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f981091f0 0x7f6f981095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.815+0000 7f6f9c9da640 1 -- 172.21.15.16:0/1478788443 >> 172.21.15.16:0/1478788443 conn(0x7f6f9806e140 msgr2=0x7f6f9806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.815+0000 7f6f9c9da640 1 -- 172.21.15.16:0/1478788443 shutdown_connections 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.815+0000 7f6f9c9da640 1 -- 172.21.15.16:0/1478788443 wait complete. 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.815+0000 7f6f9c9da640 1 Processor -- start 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.815+0000 7f6f9c9da640 1 -- start start 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f9c9da640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f981091f0 0x7f6f9810a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f9c9da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f9810a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f9c9da640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f98114ab0 0x7f6f9810ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f9c9da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9811a7b0 con 0x7f6f98109b10 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f9c9da640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9811a630 con 0x7f6f981091f0 2024-09-15T22:42:53.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f9c9da640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9811a930 con 0x7f6f98114ab0 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f9810a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f9810a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2380351068 learned_addr learned my addr 172.21.15.16:0/2380351068 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f98114ab0 0x7f6f9810ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f9659c640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f981091f0 0x7f6f9810a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2380351068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f98114ab0 msgr2=0x7f6f9810ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f98114ab0 0x7f6f9810ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2380351068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f981091f0 msgr2=0x7f6f9810a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f981091f0 0x7f6f9810a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2380351068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f981af1e0 con 0x7f6f98109b10 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f9810a910 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006010 tx=0x7f6f8c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.816+0000 7f6f877fe640 1 -- 172.21.15.16:0/2380351068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c03f030 con 0x7f6f98109b10 2024-09-15T22:42:53.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.817+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f981af470 con 0x7f6f98109b10 2024-09-15T22:42:53.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.817+0000 7f6f877fe640 1 -- 172.21.15.16:0/2380351068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c003ce0 con 0x7f6f98109b10 2024-09-15T22:42:53.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.817+0000 7f6f877fe640 1 -- 172.21.15.16:0/2380351068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c035070 con 0x7f6f98109b10 2024-09-15T22:42:53.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.817+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f981af9a0 con 0x7f6f98109b10 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.819+0000 7f6f877fe640 1 -- 172.21.15.16:0/2380351068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6f8c042030 con 0x7f6f98109b10 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.819+0000 7f6f9edb3740 1 Processor -- start 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.819+0000 7f6f9edb3740 1 -- start start 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x558927833880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927840a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5589277992e0 con 0x5589278334a0 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558927740ce0 con 0x558927833dc0 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5589277952a0 con 0x55892783e680 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f96d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x558927833880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f96d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x558927833880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51416/0 (socket says 172.21.15.16:51416) 2024-09-15T22:42:53.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f96d9d640 1 -- 172.21.15.16:0/297587565 learned_addr learned my addr 172.21.15.16:0/297587565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f877fe640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6f800a33b0 0x7f6f800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927840a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f9659c640 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f877fe640 1 -- 172.21.15.16:0/2380351068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f6f8c0f0530 con 0x7f6f98109b10 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f96d9d640 1 -- 172.21.15.16:0/297587565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 msgr2=0x558927840a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927840a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f96d9d640 1 -- 172.21.15.16:0/297587565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 msgr2=0x55892783e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.820+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.821+0000 7f6f96d9d640 1 -- 172.21.15.16:0/297587565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558927725440 con 0x5589278334a0 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.821+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6f800a33b0 0x7f6f800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.821+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x558927833880 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f90011470 tx=0x7f6f90011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cab9a22397aa4f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.821+0000 7f6f84ff9640 1 -- 172.21.15.16:0/297587565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90017070 con 0x5589278334a0 2024-09-15T22:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.821+0000 7f6f84ff9640 1 -- 172.21.15.16:0/297587565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9000fc20 con 0x5589278334a0 2024-09-15T22:42:53.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.821+0000 7f6f84ff9640 1 -- 172.21.15.16:0/297587565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9001ca90 con 0x5589278334a0 2024-09-15T22:42:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 -- 172.21.15.16:0/297587565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 msgr2=0x558927833880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x558927833880 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f90011470 tx=0x7f6f90011930 comp rx=0 tx=0).stop 2024-09-15T22:42:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6f800a33b0 0x7f6f800a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f90023000 tx=0x7f6f90024000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 -- 172.21.15.16:0/297587565 shutdown_connections 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927840a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/297587565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x558927833880 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 -- 172.21.15.16:0/297587565 >> 172.21.15.16:0/297587565 conn(0x55892782fa30 msgr2=0x55892782fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 -- 172.21.15.16:0/297587565 shutdown_connections 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 -- 172.21.15.16:0/297587565 wait complete. 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.822+0000 7f6f9edb3740 1 -- start start 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 1 Processor -- start 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x55892783df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x55892783b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558927841250 con 0x5589278334a0 2024-09-15T22:42:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558927754050 con 0x558927833dc0 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5589278413d0 con 0x55892783e680 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9edb3740 10 client.0 _ll_register_callbacks cb 0x558927794160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x55892783df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x55892783df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51428/0 (socket says 172.21.15.16:51428) 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 -- 172.21.15.16:0/761467398 learned_addr learned my addr 172.21.15.16:0/761467398 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x55892783b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 -- 172.21.15.16:0/761467398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 msgr2=0x55892783b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x55892783b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 -- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 msgr2=0x55892783b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f9659c640 1 -- 172.21.15.16:0/761467398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55892750a3c0 con 0x5589278334a0 2024-09-15T22:42:53.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x55892783b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.823+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9659c640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x55892783df70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800bad0 tx=0x7f6f8800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aebfe636b65c6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f7dffb640 1 -- 172.21.15.16:0/761467398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88003e00 con 0x5589278334a0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f7dffb640 1 -- 172.21.15.16:0/761467398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f6f88005020 con 0x5589278334a0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9000fc20 con 0x5589278334a0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f7dffb640 1 -- 172.21.15.16:0/761467398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f88005220 con 0x5589278334a0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9edb3740 10 client.16275 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55892784cc20 con 0x5589278334a0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f7dffb640 1 -- 172.21.15.16:0/761467398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f9000fc20 con 0x5589278334a0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f7dffb640 1 -- 172.21.15.16:0/761467398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6f8801b630 con 0x5589278334a0 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9edb3740 10 client.16275 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:53.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9edb3740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9edb3740 20 client.16275 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.824+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55892784e0e0 0x558927850530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.825+0000 7f6f9edb3740 4 client.16275 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.825+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589277a6850 con 0x55892784e0e0 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.825+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55892784e0e0 0x558927850530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.825+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55892784e0e0 0x558927850530 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa0c75e187ef334d server_cookie=9ee9c80810f29a82 in_seq=0 out_seq=0 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.825+0000 7f6f7dffb640 10 client.16275 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f7dffb640 1 -- 172.21.15.16:0/761467398 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589277a6850 con 0x55892784e0e0 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f7dffb640 10 client.16275 handle_command_reply: tid=0 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 client.16275 shutdown 2024-09-15T22:42:53.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 msgr2=0x55892783df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x55892783df70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800bad0 tx=0x7f6f8800bf90 comp rx=0 tx=0).stop 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 shutdown_connections 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55892784e0e0 0x558927850530 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x55892783b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892783b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/761467398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5589278334a0 0x55892783df70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 >> 172.21.15.16:0/761467398 conn(0x55892782fa30 msgr2=0x55892782eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f7dffb640 0 client.16275 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 shutdown_connections 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 1 -- 172.21.15.16:0/761467398 wait complete. 2024-09-15T22:42:53.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.826+0000 7f6f9edb3740 20 client.16275 trim_cache size 0 max 16384 2024-09-15T22:42:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 Processor -- start 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 -- start start 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892784a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783e680 0x5589278367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783a310 0x558927855e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981bc2d0 con 0x55892783e680 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b9ba0 con 0x558927833dc0 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b79f0 con 0x55892783a310 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783e680 0x5589278367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783e680 0x5589278367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51444/0 (socket says 172.21.15.16:51444) 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f95d9b640 1 -- 172.21.15.16:0/60836257 learned_addr learned my addr 172.21.15.16:0/60836257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f9659c640 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892784a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.844+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783a310 0x558927855e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f95d9b640 1 -- 172.21.15.16:0/60836257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783a310 msgr2=0x558927855e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783a310 0x558927855e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f95d9b640 1 -- 172.21.15.16:0/60836257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 msgr2=0x55892784a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892784a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f95d9b640 1 -- 172.21.15.16:0/60836257 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589278332d0 con 0x55892783e680 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783a310 0x558927855e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783e680 0x5589278367f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0b8a80 tx=0x7f6f8c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8be5f4d51ceff98 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f7ffff640 1 -- 172.21.15.16:0/60836257 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c049040 con 0x55892783e680 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f7ffff640 1 -- 172.21.15.16:0/60836257 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c0bc620 con 0x55892783e680 2024-09-15T22:42:53.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.845+0000 7f6f7ffff640 1 -- 172.21.15.16:0/60836257 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0b8c90 con 0x55892783e680 2024-09-15T22:42:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 -- 172.21.15.16:0/60836257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783e680 msgr2=0x5589278367f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783e680 0x5589278367f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0b8a80 tx=0x7f6f8c02a480 comp rx=0 tx=0).stop 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 -- 172.21.15.16:0/60836257 shutdown_connections 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783a310 0x558927855e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783e680 0x5589278367f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/60836257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x55892784a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 -- 172.21.15.16:0/60836257 >> 172.21.15.16:0/60836257 conn(0x55892782fa30 msgr2=0x5589278360e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 -- 172.21.15.16:0/60836257 shutdown_connections 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 -- 172.21.15.16:0/60836257 wait complete. 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.846+0000 7f6f9edb3740 1 -- start start 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 1 Processor -- start 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x558927846da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783a310 0x5589278420d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927842610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b8060 con 0x55892783a310 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b94d0 con 0x558927833dc0 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6f981b8730 con 0x55892783e680 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9edb3740 10 client.0 _ll_register_callbacks cb 0x7f6f88004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783a310 0x5589278420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783a310 0x5589278420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51450/0 (socket says 172.21.15.16:51450) 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2241155833 learned_addr learned my addr 172.21.15.16:0/2241155833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f9659c640 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x558927846da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f96d9d640 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927842610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2241155833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 msgr2=0x558927842610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927842610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 msgr2=0x558927846da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x558927846da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.847+0000 7f6f95d9b640 1 -- 172.21.15.16:0/2241155833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558927842e10 con 0x55892783a310 2024-09-15T22:42:53.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f95d9b640 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783a310 0x5589278420d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0bc5b0 tx=0x7f6f8c031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04268310e8686dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f7dffb640 1 -- 172.21.15.16:0/2241155833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c0d9070 con 0x55892783a310 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f7dffb640 1 -- 172.21.15.16:0/2241155833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f6f8c0b8c90 con 0x55892783a310 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f8c0bc620 con 0x55892783a310 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f7dffb640 1 -- 172.21.15.16:0/2241155833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6f8c049040 con 0x55892783a310 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f9edb3740 10 client.16281 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5589278b8e10 con 0x55892783a310 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f7dffb640 1 -- 172.21.15.16:0/2241155833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f8c0c51e0 con 0x55892783a310 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.848+0000 7f6f7dffb640 1 -- 172.21.15.16:0/2241155833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6f8c03bbb0 con 0x55892783a310 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9edb3740 10 client.16281 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9edb3740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9edb3740 20 client.16281 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5589278b9550 0x5589278bb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:53.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9edb3740 4 client.16281 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6f8c0bc620 con 0x5589278b9550 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9659c640 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5589278b9550 0x5589278bb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f9659c640 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5589278b9550 0x5589278bb970 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc9d10e7652428b server_cookie=bf3a0103f54c287f in_seq=0 out_seq=0 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.849+0000 7f6f7dffb640 10 client.16281 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f7dffb640 1 -- 172.21.15.16:0/2241155833 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f6f8c0bc620 con 0x5589278b9550 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f7dffb640 10 client.16281 handle_command_reply: tid=0 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 client.16281 shutdown 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783a310 msgr2=0x5589278420d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783a310 0x5589278420d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c0bc5b0 tx=0x7f6f8c031a60 comp rx=0 tx=0).stop 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 shutdown_connections 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5589278b9550 0x5589278bb970 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55892783e680 0x558927842610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55892783a310 0x5589278420d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 --2- 172.21.15.16:0/2241155833 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558927833dc0 0x558927846da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f7dffb640 0 client.16281 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:42:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 >> 172.21.15.16:0/2241155833 conn(0x55892782fa30 msgr2=0x55892782eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 shutdown_connections 2024-09-15T22:42:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 1 -- 172.21.15.16:0/2241155833 wait complete. 2024-09-15T22:42:53.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.850+0000 7f6f9edb3740 20 client.16281 trim_cache size 0 max 16384 2024-09-15T22:42:53.857 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:42:53.858 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:42:53.858 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:42:53.858 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:42:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6f800a33b0 msgr2=0x7f6f800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6f800a33b0 0x7f6f800a5870 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f90023000 tx=0x7f6f90024000 comp rx=0 tx=0).stop 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 msgr2=0x7f6f9810a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f9810a910 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006010 tx=0x7f6f8c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 shutdown_connections 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6f800a33b0 0x7f6f800a5870 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6f98114ab0 0x7f6f9810ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f98109b10 0x7f6f9810a910 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 --2- 172.21.15.16:0/2380351068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f981091f0 0x7f6f9810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 >> 172.21.15.16:0/2380351068 conn(0x7f6f9806e140 msgr2=0x7f6f98070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 shutdown_connections 2024-09-15T22:42:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:53.851+0000 7f6f9c9da640 1 -- 172.21.15.16:0/2380351068 wait complete. 2024-09-15T22:42:54.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:54 smithi059 ceph-mon[34872]: pgmap v137: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:54.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:54 smithi130 ceph-mon[35742]: pgmap v137: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:54.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:54 smithi016 ceph-mon[31492]: pgmap v137: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:54.697 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:42:54.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:42:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 -- 172.21.15.16:0/3394065350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34102790 msgr2=0x7f1c341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/3394065350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34102790 0x7f1c341067e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009900 tx=0x7f1c2802a410 comp rx=0 tx=0).stop 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 -- 172.21.15.16:0/3394065350 shutdown_connections 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/3394065350 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c34106f10 0x7f1c3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/3394065350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34102790 0x7f1c341067e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/3394065350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c34101de0 0x7f1c341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 -- 172.21.15.16:0/3394065350 >> 172.21.15.16:0/3394065350 conn(0x7f1c340fd8c0 msgr2=0x7f1c340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 -- 172.21.15.16:0/3394065350 shutdown_connections 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.422+0000 7f1c3bec8640 1 -- 172.21.15.16:0/3394065350 wait complete. 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 Processor -- start 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 -- start start 2024-09-15T22:42:55.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34101de0 0x7f1c34198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c34102790 0x7f1c34198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c34106f10 0x7f1c3419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34111f30 con 0x7f1c34101de0 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c34111db0 con 0x7f1c34106f10 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3bec8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c341120b0 con 0x7f1c34102790 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34101de0 0x7f1c34198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34101de0 0x7f1c34198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51464/0 (socket says 172.21.15.16:51464) 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 -- 172.21.15.16:0/2824643080 learned_addr learned my addr 172.21.15.16:0/2824643080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39463640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c34102790 0x7f1c34198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3a465640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c34106f10 0x7f1c3419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 -- 172.21.15.16:0/2824643080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c34102790 msgr2=0x7f1c34198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c34102790 0x7f1c34198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 -- 172.21.15.16:0/2824643080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c34106f10 msgr2=0x7f1c3419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:55.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c34106f10 0x7f1c3419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 -- 172.21.15.16:0/2824643080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3419d3e0 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c3a465640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c34106f10 0x7f1c3419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39463640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c34102790 0x7f1c34198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.423+0000 7f1c39c64640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34101de0 0x7f1c34198430 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00d950 tx=0x7f1c2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.424+0000 7f1c32ffd640 1 -- 172.21.15.16:0/2824643080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c017030 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.424+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3419d6d0 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.424+0000 7f1c32ffd640 1 -- 172.21.15.16:0/2824643080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c2c004020 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.424+0000 7f1c32ffd640 1 -- 172.21.15.16:0/2824643080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0047a0 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.424+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341a4fb0 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.425+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c341021c0 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.426+0000 7f1c32ffd640 1 -- 172.21.15.16:0/2824643080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1c2c0041d0 con 0x7f1c34101de0 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.427+0000 7f1c32ffd640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c1c0a33b0 0x7f1c1c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:55.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.427+0000 7f1c39463640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c1c0a33b0 0x7f1c1c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:55.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.427+0000 7f1c32ffd640 1 -- 172.21.15.16:0/2824643080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1c2c0cfb70 con 0x7f1c34101de0 2024-09-15T22:42:55.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.427+0000 7f1c39463640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c1c0a33b0 0x7f1c1c0a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c340fdce0 tx=0x7f1c28005a20 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:55.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.431+0000 7f1c32ffd640 1 -- 172.21.15.16:0/2824643080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c2c09c190 con 0x7f1c34101de0 2024-09-15T22:42:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.641+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c3410a120 con 0x7f1c34101de0 2024-09-15T22:42:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.642+0000 7f1c32ffd640 1 -- 172.21.15.16:0/2824643080 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f1c2c09ba90 con 0x7f1c34101de0 2024-09-15T22:42:55.644 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:42:55.644 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:42:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:42:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c1c0a33b0 msgr2=0x7f1c1c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c1c0a33b0 0x7f1c1c0a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c340fdce0 tx=0x7f1c28005a20 comp rx=0 tx=0).stop 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34101de0 msgr2=0x7f1c34198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34101de0 0x7f1c34198430 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00d950 tx=0x7f1c2c00de10 comp rx=0 tx=0).stop 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 shutdown_connections 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c34106f10 0x7f1c3419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1c1c0a33b0 0x7f1c1c0a5870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c34102790 0x7f1c34198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 --2- 172.21.15.16:0/2824643080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c34101de0 0x7f1c34198430 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 >> 172.21.15.16:0/2824643080 conn(0x7f1c340fd8c0 msgr2=0x7f1c340fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 shutdown_connections 2024-09-15T22:42:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:55.645+0000 7f1c3bec8640 1 -- 172.21.15.16:0/2824643080 wait complete. 2024-09-15T22:42:56.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:56 smithi016 ceph-mon[31492]: pgmap v138: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:56.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:56 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2824643080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:42:56.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:42:56.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:56 smithi059 ceph-mon[34872]: pgmap v138: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:56.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:56 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2824643080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:42:56.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:56 smithi130 ceph-mon[35742]: pgmap v138: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:42:56.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:56 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2824643080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:42:56.692 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 -- 172.21.15.16:0/2395026702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4102d40 msgr2=0x7f55c4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 --2- 172.21.15.16:0/2395026702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4102d40 0x7f55c4103120 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b402a480 comp rx=0 tx=0).stop 2024-09-15T22:42:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 -- 172.21.15.16:0/2395026702 shutdown_connections 2024-09-15T22:42:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 --2- 172.21.15.16:0/2395026702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55c410c170 0x7f55c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 --2- 172.21.15.16:0/2395026702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55c4103660 0x7f55c410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 --2- 172.21.15.16:0/2395026702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4102d40 0x7f55c4103120 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 -- 172.21.15.16:0/2395026702 >> 172.21.15.16:0/2395026702 conn(0x7f55c40fc7a0 msgr2=0x7f55c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 -- 172.21.15.16:0/2395026702 shutdown_connections 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.164+0000 7f55c898a640 1 -- 172.21.15.16:0/2395026702 wait complete. 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 Processor -- start 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 -- start start 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55c4102d40 0x7f55c41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4103660 0x7f55c4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55c410c170 0x7f55c4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4111f70 con 0x7f55c4103660 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4111df0 con 0x7f55c410c170 2024-09-15T22:42:57.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c898a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41120f0 con 0x7f55c4102d40 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4103660 0x7f55c4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55c4102d40 0x7f55c41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4103660 0x7f55c4100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51496/0 (socket says 172.21.15.16:51496) 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c1d9b640 1 -- 172.21.15.16:0/1487087445 learned_addr learned my addr 172.21.15.16:0/1487087445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.165+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55c410c170 0x7f55c4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c1d9b640 1 -- 172.21.15.16:0/1487087445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55c4102d40 msgr2=0x7f55c41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55c4102d40 0x7f55c41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c1d9b640 1 -- 172.21.15.16:0/1487087445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55c410c170 msgr2=0x7f55c4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55c410c170 0x7f55c4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c1d9b640 1 -- 172.21.15.16:0/1487087445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c41a68f0 con 0x7f55c4103660 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4103660 0x7f55c4100ee0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f55b800c940 tx=0x7f55b800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:57.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55b37fe640 1 -- 172.21.15.16:0/1487087445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8004220 con 0x7f55c4103660 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41a6be0 con 0x7f55c4103660 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55b37fe640 1 -- 172.21.15.16:0/1487087445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b80043c0 con 0x7f55c4103660 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55b37fe640 1 -- 172.21.15.16:0/1487087445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8004a00 con 0x7f55c4103660 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.166+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41a7120 con 0x7f55c4103660 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 Processor -- start 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 -- start start 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c23ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c23ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c240c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f4c23652f0 con 0x55f4c240a370 2024-09-15T22:42:57.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f4c230ccf0 con 0x55f4c2400b40 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55cad63740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f4c23612b0 con 0x55f4c23ff5d0 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55b37fe640 1 -- 172.21.15.16:0/1487087445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f55b80079b0 con 0x7f55c4103660 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55c2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c23ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55c259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c23ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55c259c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c23ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:34438/0 (socket says 172.21.15.16:34438) 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.169+0000 7f55c259c640 1 -- 172.21.15.16:0/1955659999 learned_addr learned my addr 172.21.15.16:0/1955659999 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55c259c640 1 -- 172.21.15.16:0/1955659999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 msgr2=0x55f4c23ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55c259c640 1 --2- 172.21.15.16:0/1955659999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c23ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55c259c640 1 -- 172.21.15.16:0/1955659999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 msgr2=0x55f4c240c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55c259c640 1 --2- 172.21.15.16:0/1955659999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c240c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55c259c640 1 -- 172.21.15.16:0/1955659999 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c22f13a0 con 0x55f4c23ff5d0 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55c259c640 1 --2- 172.21.15.16:0/1955659999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c23ffa10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b4031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cfa444df14c07ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55b0ff9640 1 -- 172.21.15.16:0/1955659999 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4040070 con 0x55f4c23ff5d0 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55b37fe640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f55ac0a33b0 0x7f55ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55b0ff9640 1 -- 172.21.15.16:0/1955659999 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b4005020 con 0x55f4c23ff5d0 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.170+0000 7f55b0ff9640 1 -- 172.21.15.16:0/1955659999 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4035040 con 0x55f4c23ff5d0 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f55ac0a33b0 0x7f55ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55b37fe640 1 -- 172.21.15.16:0/1487087445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f55b80cffc0 con 0x7f55c4103660 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 -- 172.21.15.16:0/1955659999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 msgr2=0x55f4c23ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f55ac0a33b0 0x7f55ac0a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc009600 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 --2- 172.21.15.16:0/1955659999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c23ffa10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009900 tx=0x7f55b4031630 comp rx=0 tx=0).stop 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 -- 172.21.15.16:0/1955659999 shutdown_connections 2024-09-15T22:42:57.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 --2- 172.21.15.16:0/1955659999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c240c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 --2- 172.21.15.16:0/1955659999 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c23ffa10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 --2- 172.21.15.16:0/1955659999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c23ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 -- 172.21.15.16:0/1955659999 >> 172.21.15.16:0/1955659999 conn(0x55f4c23c8310 msgr2=0x55f4c23fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 -- 172.21.15.16:0/1955659999 shutdown_connections 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.171+0000 7f55cad63740 1 -- 172.21.15.16:0/1955659999 wait complete. 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 -- start start 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 Processor -- start 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c2409d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c2400b40 0x55f4c2406ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c240a370 0x55f4c24075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f4c240d010 con 0x55f4c2400b40 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f4c2320060 con 0x55f4c240a370 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f4c240d190 con 0x55f4c23ff5d0 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55cad63740 10 client.0 _ll_register_callbacks cb 0x55f4c20738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c2400b40 0x55f4c2406ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55c1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c2400b40 0x55f4c2406ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51514/0 (socket says 172.21.15.16:51514) 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55c1d9b640 1 -- 172.21.15.16:0/2441952444 learned_addr learned my addr 172.21.15.16:0/2441952444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55c259c640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c2409d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.172+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c240a370 0x55f4c24075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55c1d9b640 1 -- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 msgr2=0x55f4c2409d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c2409d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55c1d9b640 1 -- 172.21.15.16:0/2441952444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c240a370 msgr2=0x55f4c24075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c240a370 0x55f4c24075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55c1d9b640 1 -- 172.21.15.16:0/2441952444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c20d63c0 con 0x55f4c2400b40 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c240a370 0x55f4c24075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c2400b40 0x55f4c2406ff0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f55b800a580 tx=0x7f55b809bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3221c717ca4d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2441952444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b801d070 con 0x55f4c2400b40 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2441952444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f55b80d20b0 con 0x55f4c2400b40 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b4005020 con 0x55f4c2400b40 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2441952444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f55b80a02f0 con 0x55f4c2400b40 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55cad63740 10 client.16296 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4c2418a80 con 0x55f4c2400b40 2024-09-15T22:42:57.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2441952444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b4005020 con 0x55f4c2400b40 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2441952444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f55b8086050 con 0x55f4c2400b40 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55cad63740 10 client.16296 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55cad63740 10 client.16296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55cad63740 20 client.16296 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.173+0000 7f55cad63740 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c2419f40 0x55f4c241c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.174+0000 7f55cad63740 4 client.16296 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.174+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4c2372860 con 0x55f4c2419f40 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.174+0000 7f55c259c640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c2419f40 0x55f4c241c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.174+0000 7f55c259c640 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c2419f40 0x55f4c241c330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7df828286674b05 server_cookie=a2fb03768f57581f in_seq=0 out_seq=0 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.174+0000 7f55a9ffb640 10 client.16296 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2441952444 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4c2372860 con 0x55f4c2419f40 2024-09-15T22:42:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55a9ffb640 10 client.16296 handle_command_reply: tid=0 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 client.16296 shutdown 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c2400b40 msgr2=0x55f4c2406ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c2400b40 0x55f4c2406ff0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f55b800a580 tx=0x7f55b809bb30 comp rx=0 tx=0).stop 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 shutdown_connections 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c2419f40 0x55f4c241c330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c240a370 0x55f4c24075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c2400b40 0x55f4c2406ff0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 --2- 172.21.15.16:0/2441952444 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c23ff5d0 0x55f4c2409d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 >> 172.21.15.16:0/2441952444 conn(0x55f4c23c8310 msgr2=0x55f4c2401960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55a9ffb640 0 client.16296 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 shutdown_connections 2024-09-15T22:42:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 1 -- 172.21.15.16:0/2441952444 wait complete. 2024-09-15T22:42:57.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.175+0000 7f55cad63740 20 client.16296 trim_cache size 0 max 16384 2024-09-15T22:42:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.190+0000 7f55cad63740 1 Processor -- start 2024-09-15T22:42:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.190+0000 7f55cad63740 1 -- start start 2024-09-15T22:42:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55cad63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c23ff5d0 0x55f4c24162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55cad63740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c2400b40 0x55f4c2405fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55cad63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c2421b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55cad63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41b3960 con 0x55f4c240a370 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55cad63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41b1230 con 0x55f4c23ff5d0 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55cad63740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41af080 con 0x55f4c2400b40 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c2421b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c2421b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51516/0 (socket says 172.21.15.16:51516) 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 -- 172.21.15.16:0/1454953700 learned_addr learned my addr 172.21.15.16:0/1454953700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c2400b40 0x55f4c2405fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c259c640 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c23ff5d0 0x55f4c24162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 -- 172.21.15.16:0/1454953700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c2400b40 msgr2=0x55f4c2405fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c2400b40 0x55f4c2405fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 -- 172.21.15.16:0/1454953700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c23ff5d0 msgr2=0x55f4c24162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c23ff5d0 0x55f4c24162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 -- 172.21.15.16:0/1454953700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c23fcdd0 con 0x55f4c240a370 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c2421b90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0092f0 tx=0x7f55bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72f1f137088aed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.191+0000 7f55abfff640 1 -- 172.21.15.16:0/1454953700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0217f0 con 0x55f4c240a370 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55abfff640 1 -- 172.21.15.16:0/1454953700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc01d420 con 0x55f4c240a370 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55abfff640 1 -- 172.21.15.16:0/1454953700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc029640 con 0x55f4c240a370 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 -- 172.21.15.16:0/1454953700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 msgr2=0x55f4c2421b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c2421b90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0092f0 tx=0x7f55bc016000 comp rx=0 tx=0).stop 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 -- 172.21.15.16:0/1454953700 shutdown_connections 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c240a370 0x55f4c2421b90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c2400b40 0x55f4c2405fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 --2- 172.21.15.16:0/1454953700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c23ff5d0 0x55f4c24162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 -- 172.21.15.16:0/1454953700 >> 172.21.15.16:0/1454953700 conn(0x55f4c23c8310 msgr2=0x55f4c24058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 -- 172.21.15.16:0/1454953700 shutdown_connections 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.192+0000 7f55cad63740 1 -- 172.21.15.16:0/1454953700 wait complete. 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 -- start start 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 Processor -- start 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c23ff5d0 0x55f4c2412420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c2412960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c240a370 0x55f4c240f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41af6f0 con 0x55f4c23ff5d0 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41b0b60 con 0x55f4c2400b40 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f55c41afdc0 con 0x55f4c240a370 2024-09-15T22:42:57.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55cad63740 10 client.0 _ll_register_callbacks cb 0x7f55b80058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c23ff5d0 0x55f4c2412420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c23ff5d0 0x55f4c2412420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51532/0 (socket says 172.21.15.16:51532) 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 -- 172.21.15.16:0/2148642751 learned_addr learned my addr 172.21.15.16:0/2148642751 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c2412960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c240a370 0x55f4c240f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 -- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c240a370 msgr2=0x55f4c240f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c240a370 0x55f4c240f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 -- 172.21.15.16:0/2148642751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 msgr2=0x55f4c2412960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c2412960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.193+0000 7f55c259c640 1 -- 172.21.15.16:0/2148642751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c2402120 con 0x55f4c23ff5d0 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55c2d9d640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c240a370 0x55f4c240f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:57.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c2412960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55c259c640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c23ff5d0 0x55f4c2412420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f55b4031550 tx=0x7f55b4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97debdb0e1a09c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2148642751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4054060 con 0x55f4c23ff5d0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2148642751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f55b4004240 con 0x55f4c23ff5d0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55bc01d420 con 0x55f4c23ff5d0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2148642751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f55b405b050 con 0x55f4c23ff5d0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55cad63740 10 client.16302 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4c2485030 con 0x55f4c23ff5d0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2148642751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc01d420 con 0x55f4c23ff5d0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2148642751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f55b403c5e0 con 0x55f4c23ff5d0 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55cad63740 10 client.16302 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:57.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55cad63740 10 client.16302 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.194+0000 7f55cad63740 20 client.16302 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55cad63740 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c24853c0 0x55f4c24877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55cad63740 4 client.16302 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4c236cd70 con 0x55f4c24853c0 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c24853c0 0x55f4c24877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55c1d9b640 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c24853c0 0x55f4c24877b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c005b50c6d4eceec server_cookie=f4d3e260321a9eaa in_seq=0 out_seq=0 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55a9ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55a9ffb640 1 -- 172.21.15.16:0/2148642751 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4c236cd70 con 0x55f4c24853c0 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55a9ffb640 10 client.16302 handle_command_reply: tid=0 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55cad63740 1 client.16302 shutdown 2024-09-15T22:42:57.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c23ff5d0 msgr2=0x55f4c2412420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.195+0000 7f55cad63740 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c23ff5d0 0x55f4c2412420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f55b4031550 tx=0x7f55b4031400 comp rx=0 tx=0).stop 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 shutdown_connections 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55f4c24853c0 0x55f4c24877b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55a9ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f4c240a370 0x55f4c240f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4c2400b40 0x55f4c2412960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 --2- 172.21.15.16:0/2148642751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f4c23ff5d0 0x55f4c2412420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 >> 172.21.15.16:0/2148642751 conn(0x55f4c23c8310 msgr2=0x55f4c2408850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 shutdown_connections 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 1 -- 172.21.15.16:0/2148642751 wait complete. 2024-09-15T22:42:57.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55cad63740 20 client.16302 trim_cache size 0 max 16384 2024-09-15T22:42:57.209 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f55ac0a33b0 msgr2=0x7f55ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f55ac0a33b0 0x7f55ac0a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55bc005fd0 tx=0x7f55bc009600 comp rx=0 tx=0).stop 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4103660 msgr2=0x7f55c4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4103660 0x7f55c4100ee0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f55b800c940 tx=0x7f55b800ce00 comp rx=0 tx=0).stop 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 shutdown_connections 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f55c410c170 0x7f55c4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55c4103660 0x7f55c4100ee0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f55ac0a33b0 0x7f55ac0a5870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 --2- 172.21.15.16:0/1487087445 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f55c4102d40 0x7f55c41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:57.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.196+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 >> 172.21.15.16:0/1487087445 conn(0x7f55c40fc7a0 msgr2=0x7f55c40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:57.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.197+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 shutdown_connections 2024-09-15T22:42:57.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:57.197+0000 7f55c898a640 1 -- 172.21.15.16:0/1487087445 wait complete. 2024-09-15T22:42:58.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:42:58.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:42:58 smithi016 ceph-mon[31492]: pgmap v139: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:58.335 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:42:58.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:42:58 smithi059 ceph-mon[34872]: pgmap v139: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:58.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:42:58 smithi130 ceph-mon[35742]: pgmap v139: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:42:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 -- 172.21.15.16:0/1069960951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c101420 msgr2=0x7fbe8c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 --2- 172.21.15.16:0/1069960951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c101420 0x7fbe8c10d950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009cd0 tx=0x7fbe8402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 -- 172.21.15.16:0/1069960951 shutdown_connections 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 --2- 172.21.15.16:0/1069960951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c101420 0x7fbe8c10d950 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 --2- 172.21.15.16:0/1069960951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe8c100a80 0x7fbe8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 --2- 172.21.15.16:0/1069960951 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe8c106a80 0x7fbe8c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 -- 172.21.15.16:0/1069960951 >> 172.21.15.16:0/1069960951 conn(0x7fbe8c0fc7a0 msgr2=0x7fbe8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 -- 172.21.15.16:0/1069960951 shutdown_connections 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.826+0000 7fbe915f3640 1 -- 172.21.15.16:0/1069960951 wait complete. 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 Processor -- start 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 -- start start 2024-09-15T22:42:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c100a80 0x7fbe8c102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe8c101420 0x7fbe8c102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe8c106a80 0x7fbe8c1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c110e80 con 0x7fbe8c100a80 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c110d00 con 0x7fbe8c101420 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe915f3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c111000 con 0x7fbe8c106a80 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c100a80 0x7fbe8c102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c100a80 0x7fbe8c102a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51536/0 (socket says 172.21.15.16:51536) 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 -- 172.21.15.16:0/2598868313 learned_addr learned my addr 172.21.15.16:0/2598868313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe8c106a80 0x7fbe8c1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 -- 172.21.15.16:0/2598868313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe8c106a80 msgr2=0x7fbe8c1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe83fff640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe8c101420 0x7fbe8c102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe8c106a80 0x7fbe8c1034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 -- 172.21.15.16:0/2598868313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe8c101420 msgr2=0x7fbe8c102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe8c101420 0x7fbe8c102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe8affd640 1 -- 172.21.15.16:0/2598868313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c1a68e0 con 0x7fbe8c100a80 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.827+0000 7fbe83fff640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe8c101420 0x7fbe8c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.828+0000 7fbe8affd640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c100a80 0x7fbe8c102a30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d950 tx=0x7fbe7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.828+0000 7fbe88ff9640 1 -- 172.21.15.16:0/2598868313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c017030 con 0x7fbe8c100a80 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.828+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c1a6bd0 con 0x7fbe8c100a80 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.828+0000 7fbe88ff9640 1 -- 172.21.15.16:0/2598868313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe7c004020 con 0x7fbe8c100a80 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.828+0000 7fbe88ff9640 1 -- 172.21.15.16:0/2598868313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0047a0 con 0x7fbe8c100a80 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.828+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c1a7160 con 0x7fbe8c100a80 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 Processor -- start 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 -- start start 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe88ff9640 1 -- 172.21.15.16:0/2598868313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbe7c0041d0 con 0x7fbe8c100a80 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b4ab6562f0 con 0x55b4ab6fb370 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b4ab5fdcf0 con 0x55b4ab6f1b40 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe939cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b4ab6522b0 con 0x55b4ab6f05d0 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.831+0000 7fbe83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51550/0 (socket says 172.21.15.16:51550) 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe8affd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 -- 172.21.15.16:0/3956254723 learned_addr learned my addr 172.21.15.16:0/3956254723 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 -- 172.21.15.16:0/3956254723 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 msgr2=0x55b4ab6f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 -- 172.21.15.16:0/3956254723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f1b40 msgr2=0x55b4ab6f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 -- 172.21.15.16:0/3956254723 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ab5e23a0 con 0x55b4ab6fb370 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe88ff9640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbe780a33b0 0x7fbe780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe83fff640 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fd760 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009c00 tx=0x7fbe74010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61eb81f9d1cc3ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbe780a33b0 0x7fbe780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe88ff9640 1 -- 172.21.15.16:0/2598868313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fbe7c0cfac0 con 0x7fbe8c100a80 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe817fa640 1 -- 172.21.15.16:0/3956254723 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74004030 con 0x55b4ab6fb370 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe817fa640 1 -- 172.21.15.16:0/3956254723 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe740041d0 con 0x55b4ab6fb370 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.832+0000 7fbe817fa640 1 -- 172.21.15.16:0/3956254723 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe740126b0 con 0x55b4ab6fb370 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 -- 172.21.15.16:0/3956254723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 msgr2=0x55b4ab6fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbe780a33b0 0x7fbe780a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005cc0 tx=0x7fbe84005c30 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fd760 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009c00 tx=0x7fbe74010740 comp rx=0 tx=0).stop 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 -- 172.21.15.16:0/3956254723 shutdown_connections 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fd760 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 --2- 172.21.15.16:0/3956254723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 -- 172.21.15.16:0/3956254723 >> 172.21.15.16:0/3956254723 conn(0x55b4ab6b9310 msgr2=0x55b4ab6ecc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 -- 172.21.15.16:0/3956254723 shutdown_connections 2024-09-15T22:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.833+0000 7fbe939cc740 1 -- 172.21.15.16:0/3956254723 wait complete. 2024-09-15T22:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 -- start start 2024-09-15T22:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 Processor -- start 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6fad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 0x55b4ab6f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b4ab6fe070 con 0x55b4ab6f05d0 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b4ab611060 con 0x55b4ab6fb370 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b4ab6fe1f0 con 0x55b4ab6f1b40 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe939cc740 10 client.0 _ll_register_callbacks cb 0x55b4ab3648c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe8affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6fad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51562/0 (socket says 172.21.15.16:51562) 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.834+0000 7fbe8affd640 1 -- 172.21.15.16:0/2891325573 learned_addr learned my addr 172.21.15.16:0/2891325573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 0x55b4ab6f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe83fff640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe8affd640 1 -- 172.21.15.16:0/2891325573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 msgr2=0x55b4ab6f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe8affd640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe8affd640 1 -- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 msgr2=0x55b4ab6f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe8affd640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 0x55b4ab6f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe8affd640 1 -- 172.21.15.16:0/2891325573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ab3c73c0 con 0x55b4ab6f05d0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe83fff640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe8affd640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6fad30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c000c00 tx=0x7fbe7c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca199d9b871c882 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe727fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe727fc640 1 -- 172.21.15.16:0/2891325573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c09b3a0 con 0x55b4ab6f05d0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe727fc640 1 -- 172.21.15.16:0/2891325573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fbe7c0980b0 con 0x55b4ab6f05d0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe740041d0 con 0x55b4ab6f05d0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe727fc640 1 -- 172.21.15.16:0/2891325573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe7c0b4050 con 0x55b4ab6f05d0 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe939cc740 10 client.16314 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b4ab709b30 con 0x55b4ab6f05d0 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.835+0000 7fbe727fc640 1 -- 172.21.15.16:0/2891325573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe740041d0 con 0x55b4ab6f05d0 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe727fc640 1 -- 172.21.15.16:0/2891325573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbe7c0a4220 con 0x55b4ab6f05d0 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe939cc740 10 client.16314 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe939cc740 10 client.16314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe939cc740 20 client.16314 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe939cc740 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab70d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe939cc740 4 client.16314 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4ab663860 con 0x55b4ab70af90 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe83fff640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab70d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe83fff640 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab70d400 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97a46817f6ccc607 server_cookie=59d2c81dd3569568 in_seq=0 out_seq=0 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.836+0000 7fbe727fc640 10 client.16314 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe727fc640 1 -- 172.21.15.16:0/2891325573 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4ab663860 con 0x55b4ab70af90 2024-09-15T22:42:58.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe727fc640 10 client.16314 handle_command_reply: tid=0 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 client.16314 shutdown 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f05d0 msgr2=0x55b4ab6fad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6fad30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c000c00 tx=0x7fbe7c005da0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 shutdown_connections 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab70d400 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe727fc640 0 client.16314 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 0x55b4ab6f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 --2- 172.21.15.16:0/2891325573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6fad30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 >> 172.21.15.16:0/2891325573 conn(0x55b4ab6b9310 msgr2=0x55b4ab6f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 shutdown_connections 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.837+0000 7fbe939cc740 1 -- 172.21.15.16:0/2891325573 wait complete. 2024-09-15T22:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.838+0000 7fbe939cc740 20 client.16314 trim_cache size 0 max 16384 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.850+0000 7fbe939cc740 1 Processor -- start 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe939cc740 1 -- start start 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe939cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1b3a50 con 0x55b4ab6f1b40 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe939cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1b1320 con 0x55b4ab6fb370 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe939cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1af170 con 0x55b4ab6f05d0 2024-09-15T22:42:58.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6fc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51576/0 (socket says 172.21.15.16:51576) 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 -- 172.21.15.16:0/4124956892 learned_addr learned my addr 172.21.15.16:0/4124956892 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe8affd640 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 -- 172.21.15.16:0/4124956892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 msgr2=0x55b4ab6f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 -- 172.21.15.16:0/4124956892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 msgr2=0x55b4ab6fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 -- 172.21.15.16:0/4124956892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ab6f1400 con 0x55b4ab6f1b40 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.851+0000 7fbe83fff640 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6fc760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x55b4ab6f1070 tx=0x7fbe74000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf1d6257e7e6cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe80ff9640 1 -- 172.21.15.16:0/4124956892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74005190 con 0x55b4ab6f1b40 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe80ff9640 1 -- 172.21.15.16:0/4124956892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe74003ad0 con 0x55b4ab6f1b40 2024-09-15T22:42:58.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe80ff9640 1 -- 172.21.15.16:0/4124956892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe740127b0 con 0x55b4ab6f1b40 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe939cc740 1 -- 172.21.15.16:0/4124956892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f1b40 msgr2=0x55b4ab6fc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe939cc740 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6fc760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x55b4ab6f1070 tx=0x7fbe74000bc0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe939cc740 1 -- 172.21.15.16:0/4124956892 shutdown_connections 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe939cc740 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6fb370 0x55b4ab6fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe939cc740 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6f1b40 0x55b4ab6fc760 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe939cc740 1 --2- 172.21.15.16:0/4124956892 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f05d0 0x55b4ab6f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.852+0000 7fbe939cc740 1 -- 172.21.15.16:0/4124956892 >> 172.21.15.16:0/4124956892 conn(0x55b4ab6b9310 msgr2=0x55b4ab6f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:58.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 -- 172.21.15.16:0/4124956892 shutdown_connections 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 -- 172.21.15.16:0/4124956892 wait complete. 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 -- start start 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 Processor -- start 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f05d0 0x55b4ab703910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab712350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab712890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1af7e0 con 0x55b4ab6fb370 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1b0c50 con 0x55b4ab6f05d0 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1afeb0 con 0x55b4ab6f1b40 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe939cc740 10 client.0 _ll_register_callbacks cb 0x7fbe7c005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab712890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab712890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51580/0 (socket says 172.21.15.16:51580) 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.853+0000 7fbe8b7fe640 1 -- 172.21.15.16:0/281418346 learned_addr learned my addr 172.21.15.16:0/281418346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe8affd640 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f05d0 0x55b4ab703910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe83fff640 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab712350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe8b7fe640 1 -- 172.21.15.16:0/281418346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 msgr2=0x55b4ab712350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab712350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe8b7fe640 1 -- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f05d0 msgr2=0x55b4ab703910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f05d0 0x55b4ab703910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe8b7fe640 1 -- 172.21.15.16:0/281418346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ab712e50 con 0x55b4ab6fb370 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe8b7fe640 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab712890 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbe84035000 tx=0x7fbe8402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a7cdba00cb0100 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe727fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe727fc640 1 -- 172.21.15.16:0/281418346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84054060 con 0x55b4ab6fb370 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe727fc640 1 -- 172.21.15.16:0/281418346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fbe84003e00 con 0x55b4ab6fb370 2024-09-15T22:42:58.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe74003ad0 con 0x55b4ab6fb370 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe727fc640 1 -- 172.21.15.16:0/281418346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbe84004750 con 0x55b4ab6fb370 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe939cc740 10 client.16320 fetch_fsmap learned FSMap version 26 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.854+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b4ab702290 con 0x55b4ab6fb370 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe727fc640 1 -- 172.21.15.16:0/281418346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe74003ad0 con 0x55b4ab6fb370 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe727fc640 1 -- 172.21.15.16:0/281418346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbe84030070 con 0x55b4ab6fb370 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe939cc740 10 client.16320 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe939cc740 10 client.16320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe939cc740 20 client.16320 populate_metadata read hostname 'smithi016' 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe939cc740 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab778220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe939cc740 4 client.16320 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4ab65dd70 con 0x55b4ab70af90 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.855+0000 7fbe8affd640 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab778220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:58.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe8affd640 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab778220 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57ad49f284efd821 server_cookie=d4e01dd7381151a4 in_seq=0 out_seq=0 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe727fc640 10 client.16320 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe727fc640 1 -- 172.21.15.16:0/281418346 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4ab65dd70 con 0x55b4ab70af90 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe727fc640 10 client.16320 handle_command_reply: tid=0 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 client.16320 shutdown 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 msgr2=0x55b4ab712890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab712890 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbe84035000 tx=0x7fbe8402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 shutdown_connections 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55b4ab70af90 0x55b4ab778220 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b4ab6fb370 0x55b4ab712890 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b4ab6f1b40 0x55b4ab712350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 --2- 172.21.15.16:0/281418346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b4ab6f05d0 0x55b4ab703910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 >> 172.21.15.16:0/281418346 conn(0x55b4ab6b9310 msgr2=0x55b4ab6fa060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe727fc640 0 client.16320 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:42:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 shutdown_connections 2024-09-15T22:42:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 1 -- 172.21.15.16:0/281418346 wait complete. 2024-09-15T22:42:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.856+0000 7fbe939cc740 20 client.16320 trim_cache size 0 max 16384 2024-09-15T22:42:58.863 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbe780a33b0 msgr2=0x7fbe780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbe780a33b0 0x7fbe780a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005cc0 tx=0x7fbe84005c30 comp rx=0 tx=0).stop 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c100a80 msgr2=0x7fbe8c102a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c100a80 0x7fbe8c102a30 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d950 tx=0x7fbe7c00de10 comp rx=0 tx=0).stop 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 shutdown_connections 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbe780a33b0 0x7fbe780a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbe8c106a80 0x7fbe8c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe8c101420 0x7fbe8c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 --2- 172.21.15.16:0/2598868313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c100a80 0x7fbe8c102a30 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 >> 172.21.15.16:0/2598868313 conn(0x7fbe8c0fc7a0 msgr2=0x7fbe8c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 shutdown_connections 2024-09-15T22:42:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:42:58.857+0000 7fbe915f3640 1 -- 172.21.15.16:0/2598868313 wait complete. 2024-09-15T22:42:59.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:43:00.134 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:00.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:00 smithi059 ceph-mon[34872]: pgmap v140: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:00.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:00 smithi130 ceph-mon[35742]: pgmap v140: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:00.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:00 smithi016 ceph-mon[31492]: pgmap v140: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2317593483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 msgr2=0x7f1a2410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2317593483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2410da40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009900 tx=0x7f1a1c02a480 comp rx=0 tx=0).stop 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2317593483 shutdown_connections 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2317593483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2410da40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2317593483 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a24102570 0x7f1a241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2317593483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a24100e20 0x7f1a24101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2317593483 >> 172.21.15.16:0/2317593483 conn(0x7f1a240fc7d0 msgr2=0x7f1a240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2317593483 shutdown_connections 2024-09-15T22:43:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2317593483 wait complete. 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.614+0000 7f1a2bc0f640 1 Processor -- start 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2bc0f640 1 -- start start 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2bc0f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a24100e20 0x7f1a24197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2bc0f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a24102570 0x7f1a24197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2bc0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2bc0f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24110f80 con 0x7f1a24102ef0 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2bc0f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24110e00 con 0x7f1a24100e20 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2bc0f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1a24111100 con 0x7f1a24102570 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51590/0 (socket says 172.21.15.16:51590) 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 -- 172.21.15.16:0/2421643692 learned_addr learned my addr 172.21.15.16:0/2421643692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a291aa640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a24102570 0x7f1a24197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a299ab640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a24100e20 0x7f1a24197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 -- 172.21.15.16:0/2421643692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a24102570 msgr2=0x7f1a24197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a24102570 0x7f1a24197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 -- 172.21.15.16:0/2421643692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a24100e20 msgr2=0x7f1a24197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a24100e20 0x7f1a24197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a2a1ac640 1 -- 172.21.15.16:0/2421643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2419c2b0 con 0x7f1a24102ef0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a291aa640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a24102570 0x7f1a24197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.615+0000 7f1a299ab640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a24100e20 0x7f1a24197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.616+0000 7f1a2a1ac640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2419bbd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c004770 tx=0x7f1a1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.616+0000 7f1a22ffd640 1 -- 172.21.15.16:0/2421643692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c03f070 con 0x7f1a24102ef0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.616+0000 7f1a22ffd640 1 -- 172.21.15.16:0/2421643692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1c031c30 con 0x7f1a24102ef0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.616+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2419c540 con 0x7f1a24102ef0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.616+0000 7f1a22ffd640 1 -- 172.21.15.16:0/2421643692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c035030 con 0x7f1a24102ef0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.616+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a241a3e80 con 0x7f1a24102ef0 2024-09-15T22:43:00.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.618+0000 7f1a22ffd640 1 -- 172.21.15.16:0/2421643692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1a1c042070 con 0x7f1a24102ef0 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.618+0000 7f1a2dfe8740 1 Processor -- start 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2dfe8740 1 -- start start 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41758b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c41758b5d0 0x55c41758ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c417596370 0x55c417598760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c4174f12f0 con 0x55c417596370 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c417498cf0 con 0x55c41758cb40 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c4174ed2b0 con 0x55c41758b5d0 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c417596370 0x55c417598760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a22ffd640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1a0c0a3430 0x7f1a0c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c417596370 0x55c417598760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51600/0 (socket says 172.21.15.16:51600) 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 -- 172.21.15.16:0/3315379895 learned_addr learned my addr 172.21.15.16:0/3315379895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a299ab640 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c41758b5d0 0x55c41758ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a2a1ac640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1a0c0a3430 0x7f1a0c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41758b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 -- 172.21.15.16:0/3315379895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c41758b5d0 msgr2=0x55c41758ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c41758b5d0 0x55c41758ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 -- 172.21.15.16:0/3315379895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 msgr2=0x55c41758b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41758b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.619+0000 7f1a291aa640 1 -- 172.21.15.16:0/3315379895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41747d3a0 con 0x55c417596370 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.620+0000 7f1a22ffd640 1 -- 172.21.15.16:0/2421643692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1a1c0f0900 con 0x7f1a24102ef0 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.620+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c417596370 0x55c417598760 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1a18013690 tx=0x7f1a1800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90182378812349ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.620+0000 7f1a2a1ac640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1a0c0a3430 0x7f1a0c0a58f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0bc7b0 tx=0x7f1a1c08e5a0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.620+0000 7f1a0bfff640 1 -- 172.21.15.16:0/3315379895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18019030 con 0x55c417596370 2024-09-15T22:43:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.620+0000 7f1a0bfff640 1 -- 172.21.15.16:0/3315379895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a18011ab0 con 0x55c417596370 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.620+0000 7f1a0bfff640 1 -- 172.21.15.16:0/3315379895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800fc30 con 0x55c417596370 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3315379895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c417596370 msgr2=0x55c417598760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c417596370 0x55c417598760 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1a18013690 tx=0x7f1a1800c750 comp rx=0 tx=0).stop 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3315379895 shutdown_connections 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c417596370 0x55c417598760 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c41758b5d0 0x55c41758ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3315379895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41758b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3315379895 >> 172.21.15.16:0/3315379895 conn(0x55c417554310 msgr2=0x55c417587c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3315379895 shutdown_connections 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3315379895 wait complete. 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 -- start start 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.621+0000 7f1a2dfe8740 1 Processor -- start 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758b5d0 0x55c417595d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758cb40 0x55c417593020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c417593610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c417599070 con 0x55c41758cb40 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c4174ac060 con 0x55c41758b5d0 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c4175991f0 con 0x55c417596370 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a2dfe8740 10 client.0 _ll_register_callbacks cb 0x55c4171ff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758cb40 0x55c417593020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758cb40 0x55c417593020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51606/0 (socket says 172.21.15.16:51606) 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 -- 172.21.15.16:0/3111170998 learned_addr learned my addr 172.21.15.16:0/3111170998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c417593610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758b5d0 0x55c417595d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 -- 172.21.15.16:0/3111170998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 msgr2=0x55c417593610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c417593610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 -- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758b5d0 msgr2=0x55c417595d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758b5d0 0x55c417595d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 -- 172.21.15.16:0/3111170998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4172623c0 con 0x55c41758cb40 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c417593610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a299ab640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758cb40 0x55c417593020 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1a14011d50 tx=0x7f1a1400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9898d450d0d53a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.622+0000 7f1a097fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a097fa640 1 -- 172.21.15.16:0/3111170998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400ee10 con 0x55c41758cb40 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a097fa640 1 -- 172.21.15.16:0/3111170998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1a14004b20 con 0x55c41758cb40 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a18011ab0 con 0x55c41758cb40 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a097fa640 1 -- 172.21.15.16:0/3111170998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a14024050 con 0x55c41758cb40 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 10 client.16332 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c4175a4b70 con 0x55c41758cb40 2024-09-15T22:43:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a097fa640 1 -- 172.21.15.16:0/3111170998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a18011ab0 con 0x55c41758cb40 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a097fa640 1 -- 172.21.15.16:0/3111170998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1a1401b810 con 0x55c41758cb40 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 10 client.16332 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 20 client.16332 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c4175a6080 0x55c4175a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 4 client.16332 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.623+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4174fe860 con 0x55c4175a6080 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.624+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c4175a6080 0x55c4175a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.624+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c4175a6080 0x55c4175a84a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d243cc517bb546 server_cookie=9e1227b1fb91fd23 in_seq=0 out_seq=0 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.624+0000 7f1a097fa640 10 client.16332 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a097fa640 1 -- 172.21.15.16:0/3111170998 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4174fe860 con 0x55c4175a6080 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a097fa640 10 client.16332 handle_command_reply: tid=0 2024-09-15T22:43:00.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 client.16332 shutdown 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758cb40 msgr2=0x55c417593020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758cb40 0x55c417593020 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1a14011d50 tx=0x7f1a1400e780 comp rx=0 tx=0).stop 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 shutdown_connections 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c4175a6080 0x55c4175a84a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c417593610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758cb40 0x55c417593020 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3111170998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758b5d0 0x55c417595d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 >> 172.21.15.16:0/3111170998 conn(0x55c417554310 msgr2=0x55c41758d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a097fa640 0 client.16332 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 shutdown_connections 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3111170998 wait complete. 2024-09-15T22:43:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.625+0000 7f1a2dfe8740 20 client.16332 trim_cache size 0 max 16384 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.638+0000 7f1a2dfe8740 1 Processor -- start 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a2dfe8740 1 -- start start 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417592ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c417591f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c4175a2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241af4a0 con 0x55c41758b5d0 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241acd70 con 0x55c41758cb40 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241aabc0 con 0x55c417596370 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417592ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417592ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51612/0 (socket says 172.21.15.16:51612) 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 -- 172.21.15.16:0/3799239103 learned_addr learned my addr 172.21.15.16:0/3799239103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c4175a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a299ab640 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c417591f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 -- 172.21.15.16:0/3799239103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 msgr2=0x55c4175a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c4175a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 -- 172.21.15.16:0/3799239103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 msgr2=0x55c417591f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c417591f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 -- 172.21.15.16:0/3799239103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c417588960 con 0x55c41758b5d0 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.639+0000 7f1a291aa640 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417592ff0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800f5c0 tx=0x7f1a1800f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3515f1ed934cf07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a0b7fe640 1 -- 172.21.15.16:0/3799239103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18019050 con 0x55c41758b5d0 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a0b7fe640 1 -- 172.21.15.16:0/3799239103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1801a040 con 0x55c41758b5d0 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a0b7fe640 1 -- 172.21.15.16:0/3799239103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18027040 con 0x55c41758b5d0 2024-09-15T22:43:00.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3799239103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 msgr2=0x55c417592ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417592ff0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800f5c0 tx=0x7f1a1800f440 comp rx=0 tx=0).stop 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3799239103 shutdown_connections 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c4175a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c417591f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/3799239103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417592ff0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3799239103 >> 172.21.15.16:0/3799239103 conn(0x55c417554310 msgr2=0x55c4175928e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3799239103 shutdown_connections 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.640+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/3799239103 wait complete. 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 -- start start 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 Processor -- start 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417590d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41759e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c41759ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241ab230 con 0x55c41758b5d0 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241ac6a0 con 0x55c41758cb40 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1a241ab900 con 0x55c417596370 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a2dfe8740 10 client.0 _ll_register_callbacks cb 0x7f1a14007ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.641+0000 7f1a291aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417590d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417590d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51618/0 (socket says 172.21.15.16:51618) 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 -- 172.21.15.16:0/2543059161 learned_addr learned my addr 172.21.15.16:0/2543059161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a299ab640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41759e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c41759ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 -- 172.21.15.16:0/2543059161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 msgr2=0x55c41759ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c41759ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 -- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 msgr2=0x55c41759e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41759e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 -- 172.21.15.16:0/2543059161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41758f2e0 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a299ab640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41759e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a291aa640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417590d10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c790 tx=0x7f1a18003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39371390b8ef98e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a097fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a097fa640 1 -- 172.21.15.16:0/2543059161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18019040 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a097fa640 1 -- 172.21.15.16:0/2543059161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1a180041c0 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1801a040 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a097fa640 1 -- 172.21.15.16:0/2543059161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a180043e0 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a2dfe8740 10 client.16338 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.642+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c417610030 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a097fa640 1 -- 172.21.15.16:0/2543059161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a1801a040 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a097fa640 1 -- 172.21.15.16:0/2543059161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1a180115d0 con 0x55c41758b5d0 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a2dfe8740 10 client.16338 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a2dfe8740 10 client.16338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:00.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a2dfe8740 20 client.16338 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c417610460 0x55c417612850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a2dfe8740 4 client.16338 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4174f8d70 con 0x55c417610460 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a299ab640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c417610460 0x55c417612850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a097fa640 10 client.16338 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.643+0000 7f1a299ab640 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c417610460 0x55c417612850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e9d96b2bdd5faa server_cookie=1a408c0a9e5278c3 in_seq=0 out_seq=0 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a097fa640 1 -- 172.21.15.16:0/2543059161 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4174f8d70 con 0x55c417610460 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a097fa640 10 client.16338 handle_command_reply: tid=0 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 client.16338 shutdown 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 msgr2=0x55c417590d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417590d10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c790 tx=0x7f1a18003850 comp rx=0 tx=0).stop 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 shutdown_connections 2024-09-15T22:43:00.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c417610460 0x55c417612850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c417596370 0x55c41759ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c41758cb40 0x55c41759e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a097fa640 0 client.16338 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 --2- 172.21.15.16:0/2543059161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c41758b5d0 0x55c417590d10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 >> 172.21.15.16:0/2543059161 conn(0x55c417554310 msgr2=0x55c417594880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 shutdown_connections 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 1 -- 172.21.15.16:0/2543059161 wait complete. 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.644+0000 7f1a2dfe8740 20 client.16338 trim_cache size 0 max 16384 2024-09-15T22:43:00.657 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1a0c0a3430 msgr2=0x7f1a0c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1a0c0a3430 0x7f1a0c0a58f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0bc7b0 tx=0x7f1a1c08e5a0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 msgr2=0x7f1a2419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2419bbd0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c004770 tx=0x7f1a1c003c10 comp rx=0 tx=0).stop 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 shutdown_connections 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a24102ef0 0x7f1a2419bbd0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1a0c0a3430 0x7f1a0c0a58f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1a24102570 0x7f1a24197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 --2- 172.21.15.16:0/2421643692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a24100e20 0x7f1a24197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 >> 172.21.15.16:0/2421643692 conn(0x7f1a240fc7d0 msgr2=0x7f1a241067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 shutdown_connections 2024-09-15T22:43:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:00.645+0000 7f1a2bc0f640 1 -- 172.21.15.16:0/2421643692 wait complete. 2024-09-15T22:43:02.415 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:43:02.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:43:02.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:02 smithi059 ceph-mon[34872]: pgmap v141: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:02.444 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:02 smithi016 ceph-mon[31492]: pgmap v141: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:02.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:02 smithi130 ceph-mon[35742]: pgmap v141: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:02.687 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 -- 172.21.15.16:0/1274083741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c101280 msgr2=0x7f878c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 --2- 172.21.15.16:0/1274083741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c101280 0x7f878c10df40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f877c009900 tx=0x7f877c02a410 comp rx=0 tx=0).stop 2024-09-15T22:43:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 -- 172.21.15.16:0/1274083741 shutdown_connections 2024-09-15T22:43:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 --2- 172.21.15.16:0/1274083741 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f878c10e480 0x7f878c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 --2- 172.21.15.16:0/1274083741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c101280 0x7f878c10df40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 --2- 172.21.15.16:0/1274083741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f878c1008d0 0x7f878c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 -- 172.21.15.16:0/1274083741 >> 172.21.15.16:0/1274083741 conn(0x7f878c0fc7a0 msgr2=0x7f878c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 -- 172.21.15.16:0/1274083741 shutdown_connections 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 -- 172.21.15.16:0/1274083741 wait complete. 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 Processor -- start 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.181+0000 7f8793a87640 1 -- start start 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8793a87640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f878c1008d0 0x7f878c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8793a87640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f878c101280 0x7f878c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8793a87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c10e480 0x7f878c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8793a87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1141a0 con 0x7f878c10e480 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8793a87640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f878c114020 con 0x7f878c101280 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8793a87640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f878c114320 con 0x7f878c1008d0 2024-09-15T22:43:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c10e480 0x7f878c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c10e480 0x7f878c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54046/0 (socket says 172.21.15.16:54046) 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 -- 172.21.15.16:0/3147331414 learned_addr learned my addr 172.21.15.16:0/3147331414 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8791823640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f878c1008d0 0x7f878c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8791022640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f878c101280 0x7f878c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 -- 172.21.15.16:0/3147331414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f878c1008d0 msgr2=0x7f878c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f878c1008d0 0x7f878c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 -- 172.21.15.16:0/3147331414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f878c101280 msgr2=0x7f878c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f878c101280 0x7f878c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 -- 172.21.15.16:0/3147331414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c19f6b0 con 0x7f878c10e480 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8791823640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f878c1008d0 0x7f878c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.182+0000 7f8792024640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c10e480 0x7f878c19efd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f878400cab0 tx=0x7f878400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.183+0000 7f8782ffd640 1 -- 172.21.15.16:0/3147331414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784007e10 con 0x7f878c10e480 2024-09-15T22:43:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.183+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c19f9a0 con 0x7f878c10e480 2024-09-15T22:43:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.183+0000 7f8782ffd640 1 -- 172.21.15.16:0/3147331414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878400f930 con 0x7f878c10e480 2024-09-15T22:43:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.183+0000 7f8782ffd640 1 -- 172.21.15.16:0/3147331414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784018b80 con 0x7f878c10e480 2024-09-15T22:43:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.183+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1a7280 con 0x7f878c10e480 2024-09-15T22:43:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.185+0000 7f8782ffd640 1 -- 172.21.15.16:0/3147331414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f878400fae0 con 0x7f878c10e480 2024-09-15T22:43:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.185+0000 7f8795e60740 1 Processor -- start 2024-09-15T22:43:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.185+0000 7f8795e60740 1 -- start start 2024-09-15T22:43:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8795e60740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5db60 0x558ca7f5df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8795e60740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5e480 0x558ca7f68800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8795e60740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f6b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8795e60740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ca7ec3280 con 0x558ca7f68d40 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8795e60740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ca7e6ab10 con 0x558ca7f5e480 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8795e60740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558ca7ebf240 con 0x558ca7f5db60 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f6b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f6b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54060/0 (socket says 172.21.15.16:54060) 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 -- 172.21.15.16:0/2304646806 learned_addr learned my addr 172.21.15.16:0/2304646806 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791823640 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5e480 0x558ca7f68800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5db60 0x558ca7f5df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 -- 172.21.15.16:0/2304646806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5db60 msgr2=0x558ca7f5df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5db60 0x558ca7f5df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 -- 172.21.15.16:0/2304646806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5e480 msgr2=0x558ca7f68800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5e480 0x558ca7f68800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8782ffd640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f87780a33b0 0x7f87780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791022640 1 -- 172.21.15.16:0/2304646806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca7e771f0 con 0x558ca7f68d40 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.186+0000 7f8791823640 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5e480 0x558ca7f68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.187+0000 7f8792024640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f87780a33b0 0x7f87780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.187+0000 7f8791022640 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f6b130 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f877c031550 tx=0x7f877c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c16a6a7b5d69037 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.187+0000 7f8782ffd640 1 -- 172.21.15.16:0/3147331414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f87840d0470 con 0x7f878c10e480 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.187+0000 7f8777fff640 1 -- 172.21.15.16:0/2304646806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c035040 con 0x558ca7f68d40 2024-09-15T22:43:03.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.187+0000 7f8777fff640 1 -- 172.21.15.16:0/2304646806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c0319e0 con 0x558ca7f68d40 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.187+0000 7f8777fff640 1 -- 172.21.15.16:0/2304646806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c004a00 con 0x558ca7f68d40 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 -- 172.21.15.16:0/2304646806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 msgr2=0x558ca7f6b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8792024640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f87780a33b0 0x7f87780a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8784098070 tx=0x7f8784086c80 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f6b130 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f877c031550 tx=0x7f877c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 -- 172.21.15.16:0/2304646806 shutdown_connections 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f6b130 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5e480 0x558ca7f68800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 --2- 172.21.15.16:0/2304646806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5db60 0x558ca7f5df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 -- 172.21.15.16:0/2304646806 >> 172.21.15.16:0/2304646806 conn(0x558ca7f5a0f0 msgr2=0x558ca7f5a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 -- 172.21.15.16:0/2304646806 shutdown_connections 2024-09-15T22:43:03.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 -- 172.21.15.16:0/2304646806 wait complete. 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 -- start start 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.188+0000 7f8795e60740 1 Processor -- start 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8795e60740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f65c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8795e60740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8795e60740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f666e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8795e60740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558ca7f6b910 con 0x558ca7f5db60 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8795e60740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558ca7e7df10 con 0x558ca7f68d40 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8795e60740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x558ca7f6ba90 con 0x558ca7f5e480 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8795e60740 10 client.0 _ll_register_callbacks cb 0x558ca7e46a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:03.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f65c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f65c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54076/0 (socket says 172.21.15.16:54076) 2024-09-15T22:43:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 -- 172.21.15.16:0/2274838885 learned_addr learned my addr 172.21.15.16:0/2274838885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791823640 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f666e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 -- 172.21.15.16:0/2274838885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 msgr2=0x558ca7f661a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f661a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 -- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 msgr2=0x558ca7f666e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f666e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 -- 172.21.15.16:0/2274838885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca7c346a0 con 0x558ca7f5db60 2024-09-15T22:43:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f8791022640 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f65c60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f877c004650 tx=0x7f877c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b56a5859dd169d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.189+0000 7f87757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:03.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f87757fa640 1 -- 172.21.15.16:0/2274838885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c035070 con 0x558ca7f5db60 2024-09-15T22:43:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f87757fa640 1 -- 172.21.15.16:0/2274838885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f877c004cf0 con 0x558ca7f5db60 2024-09-15T22:43:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c0319e0 con 0x558ca7f5db60 2024-09-15T22:43:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f87757fa640 1 -- 172.21.15.16:0/2274838885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f877c059050 con 0x558ca7f5db60 2024-09-15T22:43:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 10 client.16350 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ca7f77550 con 0x558ca7f5db60 2024-09-15T22:43:03.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f87757fa640 1 -- 172.21.15.16:0/2274838885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c0319e0 con 0x558ca7f5db60 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f87757fa640 1 -- 172.21.15.16:0/2274838885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f877c03a040 con 0x558ca7f5db60 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 10 client.16350 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 20 client.16350 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7f788a0 0x558ca7f7ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 4 client.16350 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.190+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ca7ed07f0 con 0x558ca7f788a0 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.191+0000 7f8791823640 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7f788a0 0x558ca7f7ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.191+0000 7f87757fa640 10 client.16350 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.191+0000 7f8791823640 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7f788a0 0x558ca7f7ac90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75dc3a87439e02cf server_cookie=9edd047e8532b836 in_seq=0 out_seq=0 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f87757fa640 1 -- 172.21.15.16:0/2274838885 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ca7ed07f0 con 0x558ca7f788a0 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f87757fa640 10 client.16350 handle_command_reply: tid=0 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 client.16350 shutdown 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 msgr2=0x558ca7f65c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f65c60 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f877c004650 tx=0x7f877c004530 comp rx=0 tx=0).stop 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 shutdown_connections 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7f788a0 0x558ca7f7ac90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f666e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f661a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 --2- 172.21.15.16:0/2274838885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f65c60 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 >> 172.21.15.16:0/2274838885 conn(0x558ca7f5a0f0 msgr2=0x558ca7f5fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f87757fa640 0 client.16350 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 shutdown_connections 2024-09-15T22:43:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 1 -- 172.21.15.16:0/2274838885 wait complete. 2024-09-15T22:43:03.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.192+0000 7f8795e60740 20 client.16350 trim_cache size 0 max 16384 2024-09-15T22:43:03.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 Processor -- start 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 -- start start 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5db60 0x558ca7f64ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f63400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b3ae0 con 0x558ca7f68d40 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b13b0 con 0x558ca7f5db60 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8795e60740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1af200 con 0x558ca7f5e480 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f63400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f63400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54080/0 (socket says 172.21.15.16:54080) 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 -- 172.21.15.16:0/281330023 learned_addr learned my addr 172.21.15.16:0/281330023 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:03.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791823640 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5db60 0x558ca7f64ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 -- 172.21.15.16:0/281330023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 msgr2=0x558ca7f62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 -- 172.21.15.16:0/281330023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5db60 msgr2=0x558ca7f64ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5db60 0x558ca7f64ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791022640 1 -- 172.21.15.16:0/281330023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ca7f55830 con 0x558ca7f68d40 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.210+0000 7f8791823640 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8791022640 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f63400 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f877c04b5f0 tx=0x7f877c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30534e5a192d68e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f87777fe640 1 -- 172.21.15.16:0/281330023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c035030 con 0x558ca7f68d40 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f87777fe640 1 -- 172.21.15.16:0/281330023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c050070 con 0x558ca7f68d40 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f87777fe640 1 -- 172.21.15.16:0/281330023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c03d480 con 0x558ca7f68d40 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 -- 172.21.15.16:0/281330023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 msgr2=0x558ca7f63400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f63400 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f877c04b5f0 tx=0x7f877c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 -- 172.21.15.16:0/281330023 shutdown_connections 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f68d40 0x558ca7f63400 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 --2- 172.21.15.16:0/281330023 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f5db60 0x558ca7f64ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 -- 172.21.15.16:0/281330023 >> 172.21.15.16:0/281330023 conn(0x558ca7f5a0f0 msgr2=0x558ca7f67bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.211+0000 7f8795e60740 1 -- 172.21.15.16:0/281330023 shutdown_connections 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 -- 172.21.15.16:0/281330023 wait complete. 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 -- start start 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 Processor -- start 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f7dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f7b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f7b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1af870 con 0x558ca7f5db60 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b0ce0 con 0x558ca7f68d40 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1aff40 con 0x558ca7f5e480 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8795e60740 10 client.0 _ll_register_callbacks cb 0x7f878800c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f7dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8791022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f7dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54088/0 (socket says 172.21.15.16:54088) 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.212+0000 7f8791022640 1 -- 172.21.15.16:0/1823604300 learned_addr learned my addr 172.21.15.16:0/1823604300 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791022640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f7b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791823640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f7b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791022640 1 -- 172.21.15.16:0/1823604300 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 msgr2=0x558ca7f7b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791022640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f7b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791022640 1 -- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 msgr2=0x558ca7f7b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791022640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f7b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791022640 1 -- 172.21.15.16:0/1823604300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ca7f7bdf0 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791823640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8791022640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f7dec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f877c04b770 tx=0x7f877c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868584b8b5851c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f87757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f87757fa640 1 -- 172.21.15.16:0/1823604300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c009780 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f87757fa640 1 -- 172.21.15.16:0/1823604300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f877c004e00 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c050070 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f87757fa640 1 -- 172.21.15.16:0/1823604300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f877c0318b0 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8795e60740 10 client.16356 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.213+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ca7fe30b0 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f87757fa640 1 -- 172.21.15.16:0/1823604300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c050070 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f87757fa640 1 -- 172.21.15.16:0/1823604300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f877c03db20 con 0x558ca7f5db60 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8795e60740 10 client.16356 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8795e60740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8795e60740 20 client.16356 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8795e60740 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7fe3480 0x558ca7fe5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8795e60740 4 client.16356 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ca7ecad00 con 0x558ca7fe3480 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8791823640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7fe3480 0x558ca7fe5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f87757fa640 10 client.16356 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.214+0000 7f8791823640 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7fe3480 0x558ca7fe5870 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e32673d256def8 server_cookie=89b3ba7fc6658025 in_seq=0 out_seq=0 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.215+0000 7f87757fa640 1 -- 172.21.15.16:0/1823604300 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ca7ecad00 con 0x558ca7fe3480 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.215+0000 7f87757fa640 10 client.16356 handle_command_reply: tid=0 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.215+0000 7f8795e60740 1 client.16356 shutdown 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 msgr2=0x558ca7f7dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f7dec0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f877c04b770 tx=0x7f877c004af0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 shutdown_connections 2024-09-15T22:43:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x558ca7fe3480 0x558ca7fe5870 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ca7f68d40 0x558ca7f7b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x558ca7f5e480 0x558ca7f7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 --2- 172.21.15.16:0/1823604300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558ca7f5db60 0x558ca7f7dec0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 >> 172.21.15.16:0/1823604300 conn(0x558ca7f5a0f0 msgr2=0x558ca7f73ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f87757fa640 0 client.16356 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "02214f93-2ff2-44aa-a503-04c6343770c4", 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:43:03.231 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:43:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 shutdown_connections 2024-09-15T22:43:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 1 -- 172.21.15.16:0/1823604300 wait complete. 2024-09-15T22:43:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8795e60740 20 client.16356 trim_cache size 0 max 16384 2024-09-15T22:43:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f87780a33b0 msgr2=0x7f87780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8793a87640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f87780a33b0 0x7f87780a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8784098070 tx=0x7f8784086c80 comp rx=0 tx=0).stop 2024-09-15T22:43:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c10e480 msgr2=0x7f878c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8793a87640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c10e480 0x7f878c19efd0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f878400cab0 tx=0x7f878400cf70 comp rx=0 tx=0).stop 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 shutdown_connections 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8793a87640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f878c10e480 0x7f878c19efd0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.216+0000 7f8793a87640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f878c101280 0x7f878c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.217+0000 7f8793a87640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f87780a33b0 0x7f87780a5870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.217+0000 7f8793a87640 1 --2- 172.21.15.16:0/3147331414 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f878c1008d0 0x7f878c19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.217+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 >> 172.21.15.16:0/3147331414 conn(0x7f878c0fc7a0 msgr2=0x7f878c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.217+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 shutdown_connections 2024-09-15T22:43:03.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:03.217+0000 7f8793a87640 1 -- 172.21.15.16:0/3147331414 wait complete. 2024-09-15T22:43:04.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:43:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:04 smithi059 ceph-mon[34872]: pgmap v142: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:43:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:04 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:43:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:04 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:04 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:43:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:04 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:04.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:04 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:43:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:04 smithi016 ceph-mon[31492]: pgmap v142: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:43:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:04 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:43:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:04 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:04 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:43:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:04 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:04.503 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:04 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:43:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:04 smithi130 ceph-mon[35742]: pgmap v142: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:43:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:04 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:43:04.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:04 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:04 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:43:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:04 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:04.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:04 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 -- 172.21.15.16:0/1012558220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 msgr2=0x7fbb00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 --2- 172.21.15.16:0/1012558220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb00100ee0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009930 tx=0x7fbaf402a420 comp rx=0 tx=0).stop 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 -- 172.21.15.16:0/1012558220 shutdown_connections 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 --2- 172.21.15.16:0/1012558220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb00101420 0x7fbb0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 --2- 172.21.15.16:0/1012558220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb00100ee0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 --2- 172.21.15.16:0/1012558220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb00106a80 0x7fbb00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 -- 172.21.15.16:0/1012558220 >> 172.21.15.16:0/1012558220 conn(0x7fbb000fc7a0 msgr2=0x7fbb000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:04.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 -- 172.21.15.16:0/1012558220 shutdown_connections 2024-09-15T22:43:04.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.939+0000 7fbb06b43640 1 -- 172.21.15.16:0/1012558220 wait complete. 2024-09-15T22:43:04.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 Processor -- start 2024-09-15T22:43:04.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 -- start start 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb001984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb00101420 0x7fbb001989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb00106a80 0x7fbb0019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00110e80 con 0x7fbb00100a80 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00110d00 con 0x7fbb00106a80 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb06b43640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00111000 con 0x7fbb00101420 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb048df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb001984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb048df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb001984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54096/0 (socket says 172.21.15.16:54096) 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.940+0000 7fbb048df640 1 -- 172.21.15.16:0/3190848956 learned_addr learned my addr 172.21.15.16:0/3190848956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbafffff640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb00101420 0x7fbb001989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb00106a80 0x7fbb0019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb048df640 1 -- 172.21.15.16:0/3190848956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb00101420 msgr2=0x7fbb001989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb048df640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb00101420 0x7fbb001989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb048df640 1 -- 172.21.15.16:0/3190848956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb00106a80 msgr2=0x7fbb0019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb048df640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb00106a80 0x7fbb0019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb048df640 1 -- 172.21.15.16:0/3190848956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0019d460 con 0x7fbb00100a80 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb048df640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb001984b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d950 tx=0x7fbaf000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbafdffb640 1 -- 172.21.15.16:0/3190848956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0017070 con 0x7fbb00100a80 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbafdffb640 1 -- 172.21.15.16:0/3190848956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf0009c30 con 0x7fbb00100a80 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0019d750 con 0x7fbb00100a80 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.941+0000 7fbafdffb640 1 -- 172.21.15.16:0/3190848956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000f6b0 con 0x7fbb00100a80 2024-09-15T22:43:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.942+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb001a5030 con 0x7fbb00100a80 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.944+0000 7fbafdffb640 1 -- 172.21.15.16:0/3190848956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbaf00109d0 con 0x7fbb00100a80 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.944+0000 7fbb08f1c740 1 Processor -- start 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.944+0000 7fbb08f1c740 1 -- start start 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbafdffb640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbae80a3430 0x7fbae80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf8314a0 0x563edf831880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf83c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf83c680 0x563edf83ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563edf7972e0 con 0x563edf8314a0 2024-09-15T22:43:04.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563edf73ece0 con 0x563edf83c680 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563edf7932a0 con 0x563edf831dc0 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbae80a3430 0x7fbae80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf8314a0 0x563edf831880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf8314a0 0x563edf831880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54098/0 (socket says 172.21.15.16:54098) 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 -- 172.21.15.16:0/2142071103 learned_addr learned my addr 172.21.15.16:0/2142071103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbafdffb640 1 -- 172.21.15.16:0/3190848956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fbaf00cfef0 con 0x7fbb00100a80 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb050e0640 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf83c680 0x563edf83ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbafffff640 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf83c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 -- 172.21.15.16:0/2142071103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 msgr2=0x563edf83c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf83c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 -- 172.21.15.16:0/2142071103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf83c680 msgr2=0x563edf83ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf83c680 0x563edf83ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 -- 172.21.15.16:0/2142071103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edf723440 con 0x563edf8314a0 2024-09-15T22:43:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbae80a3430 0x7fbae80a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a780 tx=0x7fbaf8013ae0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.945+0000 7fbb048df640 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf8314a0 0x563edf831880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0098320 tx=0x7fbaf001e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c78283c5770423 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbae6ffd640 1 -- 172.21.15.16:0/2142071103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00b51f0 con 0x563edf8314a0 2024-09-15T22:43:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbae6ffd640 1 -- 172.21.15.16:0/2142071103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf0009de0 con 0x563edf8314a0 2024-09-15T22:43:04.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbae6ffd640 1 -- 172.21.15.16:0/2142071103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00b5010 con 0x563edf8314a0 2024-09-15T22:43:04.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 -- 172.21.15.16:0/2142071103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf8314a0 msgr2=0x563edf831880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf8314a0 0x563edf831880 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0098320 tx=0x7fbaf001e820 comp rx=0 tx=0).stop 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 -- 172.21.15.16:0/2142071103 shutdown_connections 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf83c680 0x563edf83ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf83c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/2142071103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf8314a0 0x563edf831880 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 -- 172.21.15.16:0/2142071103 >> 172.21.15.16:0/2142071103 conn(0x563edf82da30 msgr2=0x563edf82de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.946+0000 7fbb08f1c740 1 -- 172.21.15.16:0/2142071103 shutdown_connections 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 -- 172.21.15.16:0/2142071103 wait complete. 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 -- start start 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 Processor -- start 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf8392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563edf83f1f0 con 0x563edf83c680 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563edf752050 con 0x563edf8314a0 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.947+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563edf83f370 con 0x563edf831dc0 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb08f1c740 10 client.0 _ll_register_callbacks cb 0x563edf792160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbafffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf8392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8398e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54106/0 (socket says 172.21.15.16:54106) 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbafffff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf8392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:48114/0 (socket says 172.21.15.16:48114) 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 -- 172.21.15.16:0/3627410760 learned_addr learned my addr 172.21.15.16:0/3627410760 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb048df640 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 -- 172.21.15.16:0/3627410760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 msgr2=0x563edf8392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf8392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 -- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 msgr2=0x563edf83c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 -- 172.21.15.16:0/3627410760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563edf5083c0 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8398e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8011ab0 tx=0x7fbaf8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabba6bde094293e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.948+0000 7fbadffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbadffff640 1 -- 172.21.15.16:0/3627410760 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8003970 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbadffff640 1 -- 172.21.15.16:0/3627410760 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fbaf8003b10 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf0009de0 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbadffff640 1 -- 172.21.15.16:0/3627410760 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbaf8003d10 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 10 client.16368 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563edf84ac60 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbadffff640 1 -- 172.21.15.16:0/3627410760 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf0009de0 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbadffff640 1 -- 172.21.15.16:0/3627410760 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbaf801f650 con 0x563edf83c680 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 10 client.16368 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:04.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 20 client.16368 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf84e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 4 client.16368 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.949+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563edf7a4850 con 0x563edf84c120 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.950+0000 7fbb048df640 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf84e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.950+0000 7fbb048df640 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf84e510 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4d4c25ba51f458 server_cookie=165a81d4c699df27 in_seq=0 out_seq=0 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.950+0000 7fbadffff640 10 client.16368 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbadffff640 1 -- 172.21.15.16:0/3627410760 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563edf7a4850 con 0x563edf84c120 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbadffff640 10 client.16368 handle_command_reply: tid=0 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 client.16368 shutdown 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 msgr2=0x563edf8398e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8398e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8011ab0 tx=0x7fbaf8004ab0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 shutdown_connections 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf84e510 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8398e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf8392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbadffff640 0 client.16368 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3627410760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 >> 172.21.15.16:0/3627410760 conn(0x563edf82da30 msgr2=0x563edf833760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 shutdown_connections 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3627410760 wait complete. 2024-09-15T22:43:04.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.951+0000 7fbb08f1c740 20 client.16368 trim_cache size 0 max 16384 2024-09-15T22:43:04.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.968+0000 7fbb08f1c740 1 Processor -- start 2024-09-15T22:43:04.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.968+0000 7fbb08f1c740 1 -- start start 2024-09-15T22:43:04.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf853d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf854260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001b18c0 con 0x563edf83c680 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001af190 con 0x563edf8314a0 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001acfe0 con 0x563edf831dc0 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf854260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf854260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54116/0 (socket says 172.21.15.16:54116) 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 -- 172.21.15.16:0/3815465081 learned_addr learned my addr 172.21.15.16:0/3815465081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbafffff640 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf853d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb048df640 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 -- 172.21.15.16:0/3815465081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 msgr2=0x563edf853d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf853d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 -- 172.21.15.16:0/3815465081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 msgr2=0x563edf83b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 -- 172.21.15.16:0/3815465081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edf83c3b0 con 0x563edf83c680 2024-09-15T22:43:04.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf854260 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a2c0 tx=0x7fbaf8004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b489aca3a3265823 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:04.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.969+0000 7fbae67fc640 1 -- 172.21.15.16:0/3815465081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8039070 con 0x563edf83c680 2024-09-15T22:43:04.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbae67fc640 1 -- 172.21.15.16:0/3815465081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf8012d60 con 0x563edf83c680 2024-09-15T22:43:04.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbae67fc640 1 -- 172.21.15.16:0/3815465081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8003eb0 con 0x563edf83c680 2024-09-15T22:43:04.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3815465081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 msgr2=0x563edf854260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf854260 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a2c0 tx=0x7fbaf8004a00 comp rx=0 tx=0).stop 2024-09-15T22:43:04.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3815465081 shutdown_connections 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf854260 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf853d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3815465081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf83b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3815465081 >> 172.21.15.16:0/3815465081 conn(0x563edf82da30 msgr2=0x563edf83ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3815465081 shutdown_connections 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3815465081 wait complete. 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.970+0000 7fbb08f1c740 1 -- start start 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 1 Processor -- start 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf842740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf842c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001ad650 con 0x563edf83c680 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001aeac0 con 0x563edf8314a0 2024-09-15T22:43:04.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001add20 con 0x563edf831dc0 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb08f1c740 10 client.0 _ll_register_callbacks cb 0x7fbaf8013040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb050e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb048df640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf842740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb050e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54126/0 (socket says 172.21.15.16:54126) 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbb050e0640 1 -- 172.21.15.16:0/3930930734 learned_addr learned my addr 172.21.15.16:0/3930930734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.971+0000 7fbafffff640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf842c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb050e0640 1 -- 172.21.15.16:0/3930930734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 msgr2=0x563edf842c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf842c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb050e0640 1 -- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 msgr2=0x563edf842740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf842740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb050e0640 1 -- 172.21.15.16:0/3930930734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563edf8b6240 con 0x563edf83c680 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbafffff640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf842c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:04.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb048df640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf842740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb050e0640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8b5d00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a510 tx=0x7fbaf801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a4ea8d53b87e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbadffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbadffff640 1 -- 172.21.15.16:0/3930930734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8042060 con 0x563edf83c680 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbadffff640 1 -- 172.21.15.16:0/3930930734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fbaf8013400 con 0x563edf83c680 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf8012d60 con 0x563edf83c680 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbadffff640 1 -- 172.21.15.16:0/3930930734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbaf8049050 con 0x563edf83c680 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb08f1c740 10 client.16374 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563edf8bb800 con 0x563edf83c680 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.972+0000 7fbadffff640 1 -- 172.21.15.16:0/3930930734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf8012d60 con 0x563edf83c680 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbadffff640 1 -- 172.21.15.16:0/3930930734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbaf803c070 con 0x563edf83c680 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb08f1c740 10 client.16374 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb08f1c740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:04.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb08f1c740 20 client.16374 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf8bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb08f1c740 4 client.16374 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563edf79ed60 con 0x563edf84c120 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb048df640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf8bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbadffff640 10 client.16374 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb048df640 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf8bdea0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6966099bb270ea server_cookie=f275f4e7726f0c42 in_seq=0 out_seq=0 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbadffff640 1 -- 172.21.15.16:0/3930930734 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563edf79ed60 con 0x563edf84c120 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbadffff640 10 client.16374 handle_command_reply: tid=0 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.973+0000 7fbb08f1c740 1 client.16374 shutdown 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 msgr2=0x563edf8b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8b5d00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a510 tx=0x7fbaf801f3f0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 shutdown_connections 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563edf84c120 0x563edf8bdea0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbadffff640 0 client.16374 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563edf83c680 0x563edf8b5d00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563edf831dc0 0x563edf842c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 --2- 172.21.15.16:0/3930930734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edf8314a0 0x563edf842740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 >> 172.21.15.16:0/3930930734 conn(0x563edf82da30 msgr2=0x563edf83b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 shutdown_connections 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 1 -- 172.21.15.16:0/3930930734 wait complete. 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb08f1c740 20 client.16374 trim_cache size 0 max 16384 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:43:04.982 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:43:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbae80a3430 msgr2=0x7fbae80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb06b43640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbae80a3430 0x7fbae80a58f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800a780 tx=0x7fbaf8013ae0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 msgr2=0x7fbb001984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.974+0000 7fbb06b43640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb001984b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d950 tx=0x7fbaf000de10 comp rx=0 tx=0).stop 2024-09-15T22:43:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 shutdown_connections 2024-09-15T22:43:04.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb00106a80 0x7fbb0019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbae80a3430 0x7fbae80a58f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbb00101420 0x7fbb001989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 --2- 172.21.15.16:0/3190848956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb00100a80 0x7fbb001984b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:04.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 >> 172.21.15.16:0/3190848956 conn(0x7fbb000fc7a0 msgr2=0x7fbb0006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:04.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 shutdown_connections 2024-09-15T22:43:04.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:04.975+0000 7fbb06b43640 1 -- 172.21.15.16:0/3190848956 wait complete. 2024-09-15T22:43:05.784 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:43:05.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:06.044 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:06.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:06 smithi059 ceph-mon[34872]: pgmap v143: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:43:06.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:06 smithi130 ceph-mon[35742]: pgmap v143: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:43:06.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:06 smithi016 ceph-mon[31492]: pgmap v143: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 -- 172.21.15.16:0/3325428964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94106f10 msgr2=0x7efd9410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 --2- 172.21.15.16:0/3325428964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94106f10 0x7efd9410ea90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009900 tx=0x7efd8c02a410 comp rx=0 tx=0).stop 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 -- 172.21.15.16:0/3325428964 shutdown_connections 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 --2- 172.21.15.16:0/3325428964 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94106f10 0x7efd9410ea90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 --2- 172.21.15.16:0/3325428964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd94102790 0x7efd941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 --2- 172.21.15.16:0/3325428964 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd94101de0 0x7efd941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 -- 172.21.15.16:0/3325428964 >> 172.21.15.16:0/3325428964 conn(0x7efd940fd8c0 msgr2=0x7efd940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.658+0000 7efd997b3640 1 -- 172.21.15.16:0/3325428964 shutdown_connections 2024-09-15T22:43:06.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.659+0000 7efd997b3640 1 -- 172.21.15.16:0/3325428964 wait complete. 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.659+0000 7efd997b3640 1 Processor -- start 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.659+0000 7efd997b3640 1 -- start start 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd997b3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd94101de0 0x7efd940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd997b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94102790 0x7efd94074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd997b3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd94106f10 0x7efd94074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd997b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd94111f70 con 0x7efd94102790 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd997b3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd94111df0 con 0x7efd94101de0 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd997b3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efd941120f0 con 0x7efd94106f10 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94102790 0x7efd94074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94102790 0x7efd94074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54142/0 (socket says 172.21.15.16:54142) 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 -- 172.21.15.16:0/1053723617 learned_addr learned my addr 172.21.15.16:0/1053723617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:06.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd937fe640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd94106f10 0x7efd94074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd92ffd640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd94101de0 0x7efd940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 -- 172.21.15.16:0/1053723617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd94106f10 msgr2=0x7efd94074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd94106f10 0x7efd94074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 -- 172.21.15.16:0/1053723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd94101de0 msgr2=0x7efd940744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd94101de0 0x7efd940744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 -- 172.21.15.16:0/1053723617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd941a6890 con 0x7efd94102790 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd927fc640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94102790 0x7efd94074a30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d950 tx=0x7efd7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd8bfff640 1 -- 172.21.15.16:0/1053723617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c017070 con 0x7efd94102790 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd941a6b80 con 0x7efd94102790 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd8bfff640 1 -- 172.21.15.16:0/1053723617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7c009c30 con 0x7efd94102790 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.660+0000 7efd8bfff640 1 -- 172.21.15.16:0/1053723617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00f6b0 con 0x7efd94102790 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.661+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941a70c0 con 0x7efd94102790 2024-09-15T22:43:06.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.661+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd941021c0 con 0x7efd94102790 2024-09-15T22:43:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.663+0000 7efd8bfff640 1 -- 172.21.15.16:0/1053723617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efd7c0109d0 con 0x7efd94102790 2024-09-15T22:43:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.663+0000 7efd8bfff640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd800a33b0 0x7efd800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.664+0000 7efd92ffd640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd800a33b0 0x7efd800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.664+0000 7efd8bfff640 1 -- 172.21.15.16:0/1053723617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7efd7c0cfb80 con 0x7efd94102790 2024-09-15T22:43:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.664+0000 7efd92ffd640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd800a33b0 0x7efd800a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd84005fd0 tx=0x7efd84009680 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:06.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.667+0000 7efd8bfff640 1 -- 172.21.15.16:0/1053723617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd7c009de0 con 0x7efd94102790 2024-09-15T22:43:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.877+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd94075a20 con 0x7efd94102790 2024-09-15T22:43:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.878+0000 7efd8bfff640 1 -- 172.21.15.16:0/1053723617 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7efd94075a20 con 0x7efd94102790 2024-09-15T22:43:06.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:43:06.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:43:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd800a33b0 msgr2=0x7efd800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd800a33b0 0x7efd800a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd84005fd0 tx=0x7efd84009680 comp rx=0 tx=0).stop 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94102790 msgr2=0x7efd94074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94102790 0x7efd94074a30 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00d950 tx=0x7efd7c00de10 comp rx=0 tx=0).stop 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 shutdown_connections 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd800a33b0 0x7efd800a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd94106f10 0x7efd94074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd94102790 0x7efd94074a30 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 --2- 172.21.15.16:0/1053723617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd94101de0 0x7efd940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.880+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 >> 172.21.15.16:0/1053723617 conn(0x7efd940fd8c0 msgr2=0x7efd940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.881+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 shutdown_connections 2024-09-15T22:43:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:06.881+0000 7efd997b3640 1 -- 172.21.15.16:0/1053723617 wait complete. 2024-09-15T22:43:07.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:07 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1053723617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:07.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:07 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1053723617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:07.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:43:07.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:07 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1053723617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:08.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:08 smithi016 ceph-mon[31492]: pgmap v144: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.503+0000 7f2937fd7640 1 -- 172.21.15.16:0/4253564233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930100e20 msgr2=0x7f2930101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.503+0000 7f2937fd7640 1 --2- 172.21.15.16:0/4253564233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930100e20 0x7f2930101200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f292c009900 tx=0x7f292c02a480 comp rx=0 tx=0).stop 2024-09-15T22:43:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 -- 172.21.15.16:0/4253564233 shutdown_connections 2024-09-15T22:43:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 --2- 172.21.15.16:0/4253564233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2930102ef0 0x7f293010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 --2- 172.21.15.16:0/4253564233 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2930102570 0x7f29301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 --2- 172.21.15.16:0/4253564233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930100e20 0x7f2930101200 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 -- 172.21.15.16:0/4253564233 >> 172.21.15.16:0/4253564233 conn(0x7f29300fc7d0 msgr2=0x7f29300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 -- 172.21.15.16:0/4253564233 shutdown_connections 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 -- 172.21.15.16:0/4253564233 wait complete. 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 Processor -- start 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.504+0000 7f2937fd7640 1 -- start start 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2937fd7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2930100e20 0x7f29300fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2937fd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930102570 0x7f29300fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2937fd7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2930102ef0 0x7f29300fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2937fd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2930110f50 con 0x7f2930102570 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2937fd7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2930110dd0 con 0x7f2930102ef0 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2937fd7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29301110d0 con 0x7f2930100e20 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930102570 0x7f29300fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930102570 0x7f29300fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54154/0 (socket says 172.21.15.16:54154) 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 -- 172.21.15.16:0/948276908 learned_addr learned my addr 172.21.15.16:0/948276908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935d73640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2930100e20 0x7f29300fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2936574640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2930102ef0 0x7f29300fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 -- 172.21.15.16:0/948276908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2930100e20 msgr2=0x7f29300fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2930100e20 0x7f29300fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 -- 172.21.15.16:0/948276908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2930102ef0 msgr2=0x7f29300fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2930102ef0 0x7f29300fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 -- 172.21.15.16:0/948276908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29301a6a20 con 0x7f2930102570 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2936574640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2930102ef0 0x7f29300fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2935572640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930102570 0x7f29300fd1c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f292000d950 tx=0x7f292000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2926ffd640 1 -- 172.21.15.16:0/948276908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920017030 con 0x7f2930102570 2024-09-15T22:43:08.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.505+0000 7f2926ffd640 1 -- 172.21.15.16:0/948276908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2920004020 con 0x7f2930102570 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.506+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29301a6d10 con 0x7f2930102570 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.506+0000 7f2926ffd640 1 -- 172.21.15.16:0/948276908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200047a0 con 0x7f2930102570 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.506+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29301a71f0 con 0x7f2930102570 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f2926ffd640 1 -- 172.21.15.16:0/948276908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f29200041d0 con 0x7f2930102570 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 Processor -- start 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 -- start start 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2eb40 0x55762cb2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2d5d0 0x55762cb2da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb3a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55762ca932f0 con 0x55762cb2eb40 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55762ca3acf0 con 0x55762cb2d5d0 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.508+0000 7f293a3b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55762ca8f2b0 con 0x55762cb38370 2024-09-15T22:43:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2eb40 0x55762cb2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2eb40 0x55762cb2d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54170/0 (socket says 172.21.15.16:54170) 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 -- 172.21.15.16:0/3027245676 learned_addr learned my addr 172.21.15.16:0/3027245676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2935d73640 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2d5d0 0x55762cb2da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2935572640 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb3a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 -- 172.21.15.16:0/3027245676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 msgr2=0x55762cb3a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb3a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 -- 172.21.15.16:0/3027245676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2d5d0 msgr2=0x55762cb2da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2d5d0 0x55762cb2da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 -- 172.21.15.16:0/3027245676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762ca1f3a0 con 0x55762cb2eb40 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2935572640 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb3a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2926ffd640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f291c0a33b0 0x7f291c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2936574640 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2eb40 0x55762cb2d090 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ea0 tx=0x7f2928009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc4484921bc9c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2935d73640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f291c0a33b0 0x7f291c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f2926ffd640 1 -- 172.21.15.16:0/948276908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f29200cfad0 con 0x7f2930102570 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.509+0000 7f291bfff640 1 -- 172.21.15.16:0/3027245676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928016070 con 0x55762cb2eb40 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f291bfff640 1 -- 172.21.15.16:0/3027245676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292800ee80 con 0x55762cb2eb40 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f291bfff640 1 -- 172.21.15.16:0/3027245676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29280117a0 con 0x55762cb2eb40 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 -- 172.21.15.16:0/3027245676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2eb40 msgr2=0x55762cb2d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f2935d73640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f291c0a33b0 0x7f291c0a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f292c009230 tx=0x7f292c0058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2eb40 0x55762cb2d090 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2928009ea0 tx=0x7f2928009f90 comp rx=0 tx=0).stop 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 -- 172.21.15.16:0/3027245676 shutdown_connections 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb3a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2d5d0 0x55762cb2da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3027245676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2eb40 0x55762cb2d090 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 -- 172.21.15.16:0/3027245676 >> 172.21.15.16:0/3027245676 conn(0x55762caf6310 msgr2=0x55762cb29c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.510+0000 7f293a3b0740 1 -- 172.21.15.16:0/3027245676 shutdown_connections 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 -- 172.21.15.16:0/3027245676 wait complete. 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 -- start start 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 Processor -- start 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb37d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb35000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55762cb3b070 con 0x55762cb38370 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55762ca4e060 con 0x55762cb2eb40 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.511+0000 7f293a3b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55762cb3b1f0 con 0x55762cb2d5d0 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f293a3b0740 10 client.0 _ll_register_callbacks cb 0x55762c7a18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb355d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54184/0 (socket says 172.21.15.16:54184) 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 -- 172.21.15.16:0/618080185 learned_addr learned my addr 172.21.15.16:0/618080185 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2935572640 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb35000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2935d73640 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb37d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 -- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 msgr2=0x55762cb37d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb37d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 -- 172.21.15.16:0/618080185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 msgr2=0x55762cb35000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb35000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 -- 172.21.15.16:0/618080185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762c8043c0 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f2936574640 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb355d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2928009370 tx=0x7f2928011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9048af1ce6a0d4ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f29197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f29197fa640 1 -- 172.21.15.16:0/618080185 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2928028060 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f29197fa640 1 -- 172.21.15.16:0/618080185 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f29280099e0 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.512+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55762caa0860 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f29197fa640 1 -- 172.21.15.16:0/618080185 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f292802f050 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 10 client.16392 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55762cb46af0 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f29197fa640 1 -- 172.21.15.16:0/618080185 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55762caa0860 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f29197fa640 1 -- 172.21.15.16:0/618080185 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f292800f040 con 0x55762cb38370 2024-09-15T22:43:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 10 client.16392 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 10 client.16392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 20 client.16392 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cb47ef0 0x55762cb4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 4 client.16392 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55762ca9ad70 con 0x55762cb47ef0 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.513+0000 7f2935d73640 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cb47ef0 0x55762cb4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.514+0000 7f2935d73640 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cb47ef0 0x55762cb4a2e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c2cacc301531e83 server_cookie=b5de2a2641ccaa5f in_seq=0 out_seq=0 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.514+0000 7f29197fa640 10 client.16392 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.514+0000 7f29197fa640 1 -- 172.21.15.16:0/618080185 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55762ca9ad70 con 0x55762cb47ef0 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.514+0000 7f29197fa640 10 client.16392 handle_command_reply: tid=0 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.514+0000 7f293a3b0740 1 client.16392 shutdown 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 msgr2=0x55762cb355d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb355d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2928009370 tx=0x7f2928011c80 comp rx=0 tx=0).stop 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 shutdown_connections 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cb47ef0 0x55762cb4a2e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb355d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb35000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f29197fa640 0 client.16392 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 --2- 172.21.15.16:0/618080185 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb37d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 >> 172.21.15.16:0/618080185 conn(0x55762caf6310 msgr2=0x55762cb2f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 shutdown_connections 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 1 -- 172.21.15.16:0/618080185 wait complete. 2024-09-15T22:43:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.515+0000 7f293a3b0740 20 client.16392 trim_cache size 0 max 16384 2024-09-15T22:43:08.539 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:08 smithi130 ceph-mon[35742]: pgmap v144: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 Processor -- start 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 -- start start 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb30120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb30660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb4d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29301b3a90 con 0x55762cb38370 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29301b1360 con 0x55762cb2eb40 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.528+0000 7f293a3b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29301af1b0 con 0x55762cb2d5d0 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2936574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb4d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935572640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb30660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2936574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb4d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54194/0 (socket says 172.21.15.16:54194) 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2936574640 1 -- 172.21.15.16:0/1135681365 learned_addr learned my addr 172.21.15.16:0/1135681365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935d73640 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb30120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935572640 1 -- 172.21.15.16:0/1135681365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 msgr2=0x55762cb30120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935572640 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb30120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935572640 1 -- 172.21.15.16:0/1135681365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 msgr2=0x55762cb4d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935572640 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb4d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935572640 1 -- 172.21.15.16:0/1135681365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762cb2a960 con 0x55762cb2eb40 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2936574640 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f2935572640 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb30660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29200cce60 tx=0x7f2920000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a675f3047cdeef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f291b7fe640 1 -- 172.21.15.16:0/1135681365 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292009b350 con 0x55762cb2eb40 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.529+0000 7f291b7fe640 1 -- 172.21.15.16:0/1135681365 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292009c0f0 con 0x55762cb2eb40 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f291b7fe640 1 -- 172.21.15.16:0/1135681365 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29200cd0e0 con 0x55762cb2eb40 2024-09-15T22:43:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 -- 172.21.15.16:0/1135681365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 msgr2=0x55762cb30660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb30660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f29200cce60 tx=0x7f2920000bc0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 -- 172.21.15.16:0/1135681365 shutdown_connections 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb38370 0x55762cb4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cb30660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 --2- 172.21.15.16:0/1135681365 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb2d5d0 0x55762cb30120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 -- 172.21.15.16:0/1135681365 >> 172.21.15.16:0/1135681365 conn(0x55762caf6310 msgr2=0x55762cb338d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 -- 172.21.15.16:0/1135681365 shutdown_connections 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.530+0000 7f293a3b0740 1 -- 172.21.15.16:0/1135681365 wait complete. 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 -- start start 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 Processor -- start 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2d5d0 0x55762cba9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cbaa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb4a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29301af820 con 0x55762cb2d5d0 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29301b0c90 con 0x55762cb2eb40 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29301afef0 con 0x55762cb38370 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f293a3b0740 10 client.0 _ll_register_callbacks cb 0x7f292c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f2935d73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2d5d0 0x55762cba9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f2935d73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2d5d0 0x55762cba9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54198/0 (socket says 172.21.15.16:54198) 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f2935d73640 1 -- 172.21.15.16:0/3054758717 learned_addr learned my addr 172.21.15.16:0/3054758717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.531+0000 7f2935572640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cbaa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2936574640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb4a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2935d73640 1 -- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 msgr2=0x55762cb4a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2935d73640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb4a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2935d73640 1 -- 172.21.15.16:0/3054758717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 msgr2=0x55762cbaa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:08.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2935d73640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cbaa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2935d73640 1 -- 172.21.15.16:0/3054758717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762cb4b0b0 con 0x55762cb2d5d0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2935572640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cbaa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f2935d73640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2d5d0 0x55762cba9b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f292c02a9a0 tx=0x7f292c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3262ecaaafb45fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f29197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f29197fa640 1 -- 172.21.15.16:0/3054758717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c057060 con 0x55762cb2d5d0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f29197fa640 1 -- 172.21.15.16:0/3054758717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f292c003a00 con 0x55762cb2d5d0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f292009c0f0 con 0x55762cb2d5d0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f29197fa640 1 -- 172.21.15.16:0/3054758717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f292c05e050 con 0x55762cb2d5d0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f293a3b0740 10 client.16398 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.532+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55762cbb23a0 con 0x55762cb2d5d0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f29197fa640 1 -- 172.21.15.16:0/3054758717 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292009c0f0 con 0x55762cb2d5d0 2024-09-15T22:43:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f29197fa640 1 -- 172.21.15.16:0/3054758717 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f292c044040 con 0x55762cb2d5d0 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f293a3b0740 10 client.16398 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f293a3b0740 10 client.16398 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f293a3b0740 20 client.16398 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cbb2a70 0x55762cbb4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f293a3b0740 4 client.16398 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55762ca17170 con 0x55762cbb2a70 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.533+0000 7f2935572640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cbb2a70 0x55762cbb4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f29197fa640 10 client.16398 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f2935572640 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cbb2a70 0x55762cbb4e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa8c7bf0d00bf9cd server_cookie=b7364dddbf90f92 in_seq=0 out_seq=0 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f29197fa640 1 -- 172.21.15.16:0/3054758717 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55762ca17170 con 0x55762cbb2a70 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f29197fa640 10 client.16398 handle_command_reply: tid=0 2024-09-15T22:43:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 client.16398 shutdown 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2d5d0 msgr2=0x55762cba9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2d5d0 0x55762cba9b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f292c02a9a0 tx=0x7f292c03f040 comp rx=0 tx=0).stop 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 shutdown_connections 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55762cbb2a70 0x55762cbb4e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55762cb38370 0x55762cb4a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55762cb2eb40 0x55762cbaa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f29197fa640 0 client.16398 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 --2- 172.21.15.16:0/3054758717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55762cb2d5d0 0x55762cba9b80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 >> 172.21.15.16:0/3054758717 conn(0x55762caf6310 msgr2=0x55762cb36860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:08.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 shutdown_connections 2024-09-15T22:43:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 1 -- 172.21.15.16:0/3054758717 wait complete. 2024-09-15T22:43:08.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.534+0000 7f293a3b0740 20 client.16398 trim_cache size 0 max 16384 2024-09-15T22:43:08.551 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f291c0a33b0 msgr2=0x7f291c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f291c0a33b0 0x7f291c0a5870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f292c009230 tx=0x7f292c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930102570 msgr2=0x7f29300fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930102570 0x7f29300fd1c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f292000d950 tx=0x7f292000de10 comp rx=0 tx=0).stop 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 shutdown_connections 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2930102ef0 0x7f29300fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2930102570 0x7f29300fd1c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f291c0a33b0 0x7f291c0a5870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 --2- 172.21.15.16:0/948276908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2930100e20 0x7f29300fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 >> 172.21.15.16:0/948276908 conn(0x7f29300fc7d0 msgr2=0x7f293006a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 shutdown_connections 2024-09-15T22:43:08.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:08.535+0000 7f2937fd7640 1 -- 172.21.15.16:0/948276908 wait complete. 2024-09-15T22:43:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:08 smithi059 ceph-mon[34872]: pgmap v144: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:09.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:43:09.657 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 -- 172.21.15.16:0/1920879148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378102e00 msgr2=0x7fe3781031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 --2- 172.21.15.16:0/1920879148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378102e00 0x7fe3781031e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe374009900 tx=0x7fe37402a480 comp rx=0 tx=0).stop 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 -- 172.21.15.16:0/1920879148 shutdown_connections 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 --2- 172.21.15.16:0/1920879148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37810c1a0 0x7fe37810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 --2- 172.21.15.16:0/1920879148 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe378103720 0x7fe37810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 --2- 172.21.15.16:0/1920879148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378102e00 0x7fe3781031e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 -- 172.21.15.16:0/1920879148 >> 172.21.15.16:0/1920879148 conn(0x7fe3780fc7a0 msgr2=0x7fe3780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 -- 172.21.15.16:0/1920879148 shutdown_connections 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.239+0000 7fe37eb07640 1 -- 172.21.15.16:0/1920879148 wait complete. 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 Processor -- start 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 -- start start 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe378102e00 0x7fe378198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378103720 0x7fe378198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37810c1a0 0x7fe37819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe378111f70 con 0x7fe378103720 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe378111df0 con 0x7fe37810c1a0 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37eb07640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781120f0 con 0x7fe378102e00 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378103720 0x7fe378198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378103720 0x7fe378198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54230/0 (socket says 172.21.15.16:54230) 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe36ffff640 1 -- 172.21.15.16:0/4174249524 learned_addr learned my addr 172.21.15.16:0/4174249524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.240+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37810c1a0 0x7fe37819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe378102e00 0x7fe378198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36ffff640 1 -- 172.21.15.16:0/4174249524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe378102e00 msgr2=0x7fe378198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36ffff640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe378102e00 0x7fe378198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36ffff640 1 -- 172.21.15.16:0/4174249524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37810c1a0 msgr2=0x7fe37819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36ffff640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37810c1a0 0x7fe37819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36ffff640 1 -- 172.21.15.16:0/4174249524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37819d3c0 con 0x7fe378103720 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36ffff640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378103720 0x7fe378198950 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e9f0 tx=0x7fe36800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36dffb640 1 -- 172.21.15.16:0/4174249524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36800cd80 con 0x7fe378103720 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36dffb640 1 -- 172.21.15.16:0/4174249524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe368010910 con 0x7fe378103720 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37819d6b0 con 0x7fe378103720 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe36dffb640 1 -- 172.21.15.16:0/4174249524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368018990 con 0x7fe378103720 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.241+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3781a4f90 con 0x7fe378103720 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.244+0000 7fe36dffb640 1 -- 172.21.15.16:0/4174249524 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe368010ac0 con 0x7fe378103720 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.244+0000 7fe380ee0740 1 Processor -- start 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.244+0000 7fe380ee0740 1 -- start start 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae62090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae625d0 0x5587fae62a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae73b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe380ee0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587fadc82f0 con 0x5587fae625d0 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe380ee0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5587fad6fcf0 con 0x5587fae63b40 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe380ee0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5587fadc42b0 con 0x5587fae71780 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae625d0 0x5587fae62a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae625d0 0x5587fae62a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54244/0 (socket says 172.21.15.16:54244) 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 -- 172.21.15.16:0/2329358905 learned_addr learned my addr 172.21.15.16:0/2329358905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae62090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe36dffb640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3600a33b0 0x7fe3600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe36ffff640 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae73b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 -- 172.21.15.16:0/2329358905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 msgr2=0x5587fae73b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae73b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 -- 172.21.15.16:0/2329358905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 msgr2=0x5587fae62090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae62090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37c8a3640 1 -- 172.21.15.16:0/2329358905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587fad543a0 con 0x5587fae625d0 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe36ffff640 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae73b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae62090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3600a33b0 0x7fe3600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.245+0000 7fe36dffb640 1 -- 172.21.15.16:0/4174249524 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe3680d03f0 con 0x7fe378103720 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.246+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae625d0 0x5587fae62a10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe374009900 tx=0x7fe374005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b36453ac31ea8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.246+0000 7fe35effd640 1 -- 172.21.15.16:0/2329358905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374040070 con 0x5587fae625d0 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.246+0000 7fe35effd640 1 -- 172.21.15.16:0/2329358905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3740318f0 con 0x5587fae625d0 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.246+0000 7fe35effd640 1 -- 172.21.15.16:0/2329358905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374035040 con 0x5587fae625d0 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.246+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3600a33b0 0x7fe3600a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe370005fd0 tx=0x7fe370004a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 -- 172.21.15.16:0/2329358905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae625d0 msgr2=0x5587fae62a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae625d0 0x5587fae62a10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe374009900 tx=0x7fe374005e00 comp rx=0 tx=0).stop 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 -- 172.21.15.16:0/2329358905 shutdown_connections 2024-09-15T22:43:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae73b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae625d0 0x5587fae62a10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 --2- 172.21.15.16:0/2329358905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae62090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 -- 172.21.15.16:0/2329358905 >> 172.21.15.16:0/2329358905 conn(0x5587fae2b310 msgr2=0x5587fae5ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 -- 172.21.15.16:0/2329358905 shutdown_connections 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 -- 172.21.15.16:0/2329358905 wait complete. 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 -- start start 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.247+0000 7fe380ee0740 1 Processor -- start 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae710f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae6e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5587fae74420 con 0x5587fae71780 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5587fad83060 con 0x5587fae63b40 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5587fae745a0 con 0x5587fae625d0 2024-09-15T22:43:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 10 client.0 _ll_register_callbacks cb 0x5587faad68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54256/0 (socket says 172.21.15.16:54256) 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 -- 172.21.15.16:0/12565809 learned_addr learned my addr 172.21.15.16:0/12565809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe36ffff640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae6e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 -- 172.21.15.16:0/12565809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 msgr2=0x5587fae710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 -- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 msgr2=0x5587fae6e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae6e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 -- 172.21.15.16:0/12565809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587fab393c0 con 0x5587fae71780 2024-09-15T22:43:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe36ffff640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae6e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6e900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe370009ed0 tx=0x7fe370004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823ffcb664d27fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe357fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe357fff640 1 -- 172.21.15.16:0/12565809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37001ed70 con 0x5587fae71780 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe357fff640 1 -- 172.21.15.16:0/12565809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe370028470 con 0x5587fae71780 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.248+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3740318f0 con 0x5587fae71780 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe357fff640 1 -- 172.21.15.16:0/12565809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe370013e50 con 0x5587fae71780 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 10 client.16410 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5587fae7fe50 con 0x5587fae71780 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe357fff640 1 -- 172.21.15.16:0/12565809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3740318f0 con 0x5587fae71780 2024-09-15T22:43:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe357fff640 1 -- 172.21.15.16:0/12565809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe37000f040 con 0x5587fae71780 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 10 client.16410 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 10 client.16410 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 20 client.16410 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587fae81310 0x5587fae83740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 4 client.16410 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587fadd5860 con 0x5587fae81310 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.249+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587fae81310 0x5587fae83740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.250+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587fae81310 0x5587fae83740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ad93d3a8d225841 server_cookie=2f82842e855eb4f9 in_seq=0 out_seq=0 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.250+0000 7fe357fff640 10 client.16410 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe357fff640 1 -- 172.21.15.16:0/12565809 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587fadd5860 con 0x5587fae81310 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe357fff640 10 client.16410 handle_command_reply: tid=0 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 client.16410 shutdown 2024-09-15T22:43:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 msgr2=0x5587fae6e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6e900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe370009ed0 tx=0x7fe370004ab0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 shutdown_connections 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587fae81310 0x5587fae83740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6e900 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae6e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 --2- 172.21.15.16:0/12565809 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 >> 172.21.15.16:0/12565809 conn(0x5587fae2b310 msgr2=0x5587fae64930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe357fff640 0 client.16410 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 shutdown_connections 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 1 -- 172.21.15.16:0/12565809 wait complete. 2024-09-15T22:43:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.251+0000 7fe380ee0740 20 client.16410 trim_cache size 0 max 16384 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.264+0000 7fe380ee0740 1 Processor -- start 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.264+0000 7fe380ee0740 1 -- start start 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae625d0 0x5587fae7d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae63b40 0x5587fae6dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae5e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe380ee0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781af570 con 0x5587fae63b40 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe380ee0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ace40 con 0x5587fae625d0 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe380ee0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781aac90 con 0x5587fae71780 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae63b40 0x5587fae6dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae63b40 0x5587fae6dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54262/0 (socket says 172.21.15.16:54262) 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 -- 172.21.15.16:0/4054529988 learned_addr learned my addr 172.21.15.16:0/4054529988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae625d0 0x5587fae7d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae5e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 -- 172.21.15.16:0/4054529988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 msgr2=0x5587fae5e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae5e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 -- 172.21.15.16:0/4054529988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae625d0 msgr2=0x5587fae7d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae625d0 0x5587fae7d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 -- 172.21.15.16:0/4054529988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587fae63760 con 0x5587fae63b40 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae625d0 0x5587fae7d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.265+0000 7fe36ffff640 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae63b40 0x5587fae6dc60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe3680cd7e0 tx=0x7fe3680d2840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf97f419668605 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe35e7fc640 1 -- 172.21.15.16:0/4054529988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680a06c0 con 0x5587fae63b40 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe35e7fc640 1 -- 172.21.15.16:0/4054529988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3680a4f80 con 0x5587fae63b40 2024-09-15T22:43:10.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe35e7fc640 1 -- 172.21.15.16:0/4054529988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680cda00 con 0x5587fae63b40 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 -- 172.21.15.16:0/4054529988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae63b40 msgr2=0x5587fae6dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae63b40 0x5587fae6dc60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe3680cd7e0 tx=0x7fe3680d2840 comp rx=0 tx=0).stop 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 -- 172.21.15.16:0/4054529988 shutdown_connections 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae71780 0x5587fae5e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae63b40 0x5587fae6dc60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 --2- 172.21.15.16:0/4054529988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae625d0 0x5587fae7d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 -- 172.21.15.16:0/4054529988 >> 172.21.15.16:0/4054529988 conn(0x5587fae2b310 msgr2=0x5587fae6cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.266+0000 7fe380ee0740 1 -- 172.21.15.16:0/4054529988 shutdown_connections 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 -- 172.21.15.16:0/4054529988 wait complete. 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 -- start start 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 Processor -- start 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae79b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae7a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ab300 con 0x5587fae71780 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ac770 con 0x5587fae63b40 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ab9d0 con 0x5587fae625d0 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe380ee0740 10 client.0 _ll_register_callbacks cb 0x7fe370005bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe37d0a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe37d0a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54268/0 (socket says 172.21.15.16:54268) 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.267+0000 7fe37d0a4640 1 -- 172.21.15.16:0/1857359342 learned_addr learned my addr 172.21.15.16:0/1857359342 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe36ffff640 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae7a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae79b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe37d0a4640 1 -- 172.21.15.16:0/1857359342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 msgr2=0x5587fae79b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae79b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe37d0a4640 1 -- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 msgr2=0x5587fae7a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae7a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe37d0a4640 1 -- 172.21.15.16:0/1857359342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587fae6b5f0 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe37d0a4640 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6ae20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe37001fa30 tx=0x7fe370028720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521160d29d930f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe357fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe357fff640 1 -- 172.21.15.16:0/1857359342 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37001e7b0 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe357fff640 1 -- 172.21.15.16:0/1857359342 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe370028b20 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3680a4f80 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe357fff640 1 -- 172.21.15.16:0/1857359342 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3700134c0 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe380ee0740 10 client.16416 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.268+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5587faeebba0 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe357fff640 1 -- 172.21.15.16:0/1857359342 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3680a4f80 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe357fff640 1 -- 172.21.15.16:0/1857359342 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3700359e0 con 0x5587fae71780 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe380ee0740 10 client.16416 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe380ee0740 10 client.16416 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:43:10.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe380ee0740 20 client.16416 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe380ee0740 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587faeec260 0x5587faeee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe380ee0740 4 client.16416 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587fadcfd70 con 0x5587faeec260 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.269+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587faeec260 0x5587faeee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe37c8a3640 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587faeec260 0x5587faeee680 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eab107d5ee3288bb server_cookie=284c86274ec568e8 in_seq=0 out_seq=0 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe357fff640 10 client.16416 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe357fff640 1 -- 172.21.15.16:0/1857359342 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587fadcfd70 con 0x5587faeec260 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe357fff640 10 client.16416 handle_command_reply: tid=0 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 client.16416 shutdown 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 msgr2=0x5587fae6ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6ae20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe37001fa30 tx=0x7fe370028720 comp rx=0 tx=0).stop 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 shutdown_connections 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5587faeec260 0x5587faeee680 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5587fae71780 0x5587fae6ae20 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5587fae63b40 0x5587fae7a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 --2- 172.21.15.16:0/1857359342 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5587fae625d0 0x5587fae79b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 >> 172.21.15.16:0/1857359342 conn(0x5587fae2b310 msgr2=0x5587fae6c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe357fff640 0 client.16416 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 shutdown_connections 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 1 -- 172.21.15.16:0/1857359342 wait complete. 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.270+0000 7fe380ee0740 20 client.16416 trim_cache size 0 max 16384 2024-09-15T22:43:10.284 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:43:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3600a33b0 msgr2=0x7fe3600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3600a33b0 0x7fe3600a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe370005fd0 tx=0x7fe370004a40 comp rx=0 tx=0).stop 2024-09-15T22:43:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378103720 msgr2=0x7fe378198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378103720 0x7fe378198950 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e9f0 tx=0x7fe36800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 shutdown_connections 2024-09-15T22:43:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37810c1a0 0x7fe37819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe378103720 0x7fe378198950 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3600a33b0 0x7fe3600a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 --2- 172.21.15.16:0/4174249524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe378102e00 0x7fe378198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 >> 172.21.15.16:0/4174249524 conn(0x7fe3780fc7a0 msgr2=0x7fe3780fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 shutdown_connections 2024-09-15T22:43:10.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:10.271+0000 7fe37eb07640 1 -- 172.21.15.16:0/4174249524 wait complete. 2024-09-15T22:43:10.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:10 smithi016 ceph-mon[31492]: pgmap v145: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:43:10.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:10 smithi130 ceph-mon[35742]: pgmap v145: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:43:10.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:10 smithi059 ceph-mon[34872]: pgmap v145: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:43:11.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:43:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 -- 172.21.15.16:0/301927825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103720 msgr2=0x7ff840107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 --2- 172.21.15.16:0/301927825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103720 0x7ff840107b60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff830009930 tx=0x7ff83002a420 comp rx=0 tx=0).stop 2024-09-15T22:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 -- 172.21.15.16:0/301927825 shutdown_connections 2024-09-15T22:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 --2- 172.21.15.16:0/301927825 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8401082a0 0x7ff84010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 --2- 172.21.15.16:0/301927825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840103720 0x7ff840107b60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 --2- 172.21.15.16:0/301927825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff840102d70 0x7ff840103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 -- 172.21.15.16:0/301927825 >> 172.21.15.16:0/301927825 conn(0x7ff8400fc7d0 msgr2=0x7ff8400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 -- 172.21.15.16:0/301927825 shutdown_connections 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 -- 172.21.15.16:0/301927825 wait complete. 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 Processor -- start 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.073+0000 7ff8485e8640 1 -- start start 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8485e8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102d70 0x7ff8401985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8485e8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff840103720 0x7ff840198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8485e8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8401082a0 0x7ff84019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8485e8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff840112060 con 0x7ff840102d70 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8485e8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff840111ee0 con 0x7ff840103720 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8485e8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401121e0 con 0x7ff8401082a0 2024-09-15T22:43:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102d70 0x7ff8401985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102d70 0x7ff8401985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49554/0 (socket says 172.21.15.16:49554) 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 -- 172.21.15.16:0/2002995336 learned_addr learned my addr 172.21.15.16:0/2002995336 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846b85640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8401082a0 0x7ff84019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff845b83640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff840103720 0x7ff840198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 -- 172.21.15.16:0/2002995336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8401082a0 msgr2=0x7ff84019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8401082a0 0x7ff84019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 -- 172.21.15.16:0/2002995336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff840103720 msgr2=0x7ff840198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff840103720 0x7ff840198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 -- 172.21.15.16:0/2002995336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84019d590 con 0x7ff840102d70 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff846384640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102d70 0x7ff8401985e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c890 tx=0x7ff83c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8377fe640 1 -- 172.21.15.16:0/2002995336 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c004220 con 0x7ff840102d70 2024-09-15T22:43:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8377fe640 1 -- 172.21.15.16:0/2002995336 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c0043c0 con 0x7ff840102d70 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.074+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84019d880 con 0x7ff840102d70 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.075+0000 7ff8377fe640 1 -- 172.21.15.16:0/2002995336 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0049e0 con 0x7ff840102d70 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.075+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8401a5160 con 0x7ff840102d70 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.077+0000 7ff8377fe640 1 -- 172.21.15.16:0/2002995336 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff83c0079b0 con 0x7ff840102d70 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.077+0000 7ff84a9c1740 1 Processor -- start 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.077+0000 7ff84a9c1740 1 -- start start 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.077+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555800996b40 0x555800995090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x555800995a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558009a0370 0x5558009a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5558008fb2f0 con 0x5558009a0370 2024-09-15T22:43:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5558008a2cf0 con 0x5558009955d0 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5558008f72b0 con 0x555800996b40 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558009a0370 0x5558009a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558009a0370 0x5558009a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49562/0 (socket says 172.21.15.16:49562) 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 -- 172.21.15.16:0/2716679425 learned_addr learned my addr 172.21.15.16:0/2716679425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff846b85640 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555800996b40 0x555800995090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff846384640 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x555800995a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff8377fe640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff82c0a3430 0x7ff82c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff846b85640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff82c0a3430 0x7ff82c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 -- 172.21.15.16:0/2716679425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555800996b40 msgr2=0x555800995090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555800996b40 0x555800995090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 -- 172.21.15.16:0/2716679425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 msgr2=0x555800995a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x555800995a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff8377fe640 1 -- 172.21.15.16:0/2002995336 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff83c0d0100 con 0x7ff840102d70 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.078+0000 7ff845b83640 1 -- 172.21.15.16:0/2716679425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558008873a0 con 0x5558009a0370 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.079+0000 7ff845b83640 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558009a0370 0x5558009a2760 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff830005ab0 tx=0x7ff830032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb2b13cba5cf8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.079+0000 7ff846b85640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff82c0a3430 0x7ff82c0a58f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff838004ab0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.079+0000 7ff834ff9640 1 -- 172.21.15.16:0/2716679425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83003f070 con 0x5558009a0370 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.079+0000 7ff834ff9640 1 -- 172.21.15.16:0/2716679425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8300317b0 con 0x5558009a0370 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.079+0000 7ff834ff9640 1 -- 172.21.15.16:0/2716679425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830035070 con 0x5558009a0370 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.079+0000 7ff84a9c1740 1 -- 172.21.15.16:0/2716679425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558009a0370 msgr2=0x5558009a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.079+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558009a0370 0x5558009a2760 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff830005ab0 tx=0x7ff830032cb0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- 172.21.15.16:0/2716679425 shutdown_connections 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558009a0370 0x5558009a2760 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x555800995a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/2716679425 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555800996b40 0x555800995090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- 172.21.15.16:0/2716679425 >> 172.21.15.16:0/2716679425 conn(0x55580095e310 msgr2=0x555800991c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- 172.21.15.16:0/2716679425 shutdown_connections 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- 172.21.15.16:0/2716679425 wait complete. 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- start start 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 Processor -- start 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x55580099d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x55580099d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5558009a3010 con 0x555800996b40 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5558008b6060 con 0x5558009a0370 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5558009a3190 con 0x5558009955d0 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff84a9c1740 10 client.0 _ll_register_callbacks cb 0x5558006098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.080+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x55580099d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x55580099d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49578/0 (socket says 172.21.15.16:49578) 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 -- 172.21.15.16:0/4162908551 learned_addr learned my addr 172.21.15.16:0/4162908551 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff846b85640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x55580099d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff846384640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 -- 172.21.15.16:0/4162908551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 msgr2=0x55580099fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 -- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 msgr2=0x55580099d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x55580099d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 -- 172.21.15.16:0/4162908551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55580066c3c0 con 0x555800996b40 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff846384640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff846b85640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x55580099d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff845b83640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x55580099d060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff830006010 tx=0x7ff8300315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2949c63caff7757 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff829ffb640 1 -- 172.21.15.16:0/4162908551 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830035070 con 0x555800996b40 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff829ffb640 1 -- 172.21.15.16:0/4162908551 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff8300035f0 con 0x555800996b40 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8300317b0 con 0x555800996b40 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff829ffb640 1 -- 172.21.15.16:0/4162908551 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff830055050 con 0x555800996b40 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff84a9c1740 10 client.16428 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.081+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558009aeb00 con 0x555800996b40 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff829ffb640 1 -- 172.21.15.16:0/4162908551 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8300317b0 con 0x555800996b40 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff829ffb640 1 -- 172.21.15.16:0/4162908551 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff83003b910 con 0x555800996b40 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff84a9c1740 10 client.16428 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff84a9c1740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff84a9c1740 20 client.16428 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x5558009b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff84a9c1740 4 client.16428 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555800908860 con 0x5558009afe50 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff846384640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x5558009b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff846384640 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x5558009b2240 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b58a366c3491a5 server_cookie=f32b3c9e6a886ce2 in_seq=0 out_seq=0 2024-09-15T22:43:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.082+0000 7ff829ffb640 10 client.16428 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff829ffb640 1 -- 172.21.15.16:0/4162908551 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555800908860 con 0x5558009afe50 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff829ffb640 10 client.16428 handle_command_reply: tid=0 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 client.16428 shutdown 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 msgr2=0x55580099d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x55580099d060 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff830006010 tx=0x7ff8300315b0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 shutdown_connections 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x5558009b2240 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x55580099d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x55580099d060 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff829ffb640 0 client.16428 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4162908551 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 >> 172.21.15.16:0/4162908551 conn(0x55580095e310 msgr2=0x555800997960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 shutdown_connections 2024-09-15T22:43:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.083+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4162908551 wait complete. 2024-09-15T22:43:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.084+0000 7ff84a9c1740 20 client.16428 trim_cache size 0 max 16384 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 Processor -- start 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 -- start start 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x5558009b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401b1820 con 0x555800996b40 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401af0f0 con 0x5558009a0370 2024-09-15T22:43:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401acf40 con 0x5558009955d0 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49586/0 (socket says 172.21.15.16:49586) 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff845b83640 1 -- 172.21.15.16:0/4036302993 learned_addr learned my addr 172.21.15.16:0/4036302993 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff846384640 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.097+0000 7ff846b85640 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x5558009b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff845b83640 1 -- 172.21.15.16:0/4036302993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 msgr2=0x55580099c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff845b83640 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff845b83640 1 -- 172.21.15.16:0/4036302993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 msgr2=0x5558009b57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff845b83640 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x5558009b57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff845b83640 1 -- 172.21.15.16:0/4036302993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55580099db60 con 0x555800996b40 2024-09-15T22:43:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff846384640 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff846b85640 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x5558009b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff845b83640 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b5260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff83003be80 tx=0x7ff830032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15762bca2c919fb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff82bfff640 1 -- 172.21.15.16:0/4036302993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830035070 con 0x555800996b40 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff82bfff640 1 -- 172.21.15.16:0/4036302993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83004c070 con 0x555800996b40 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.098+0000 7ff82bfff640 1 -- 172.21.15.16:0/4036302993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83003f040 con 0x555800996b40 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4036302993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 msgr2=0x5558009b5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b5260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff83003be80 tx=0x7ff830032fc0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4036302993 shutdown_connections 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009a0370 0x5558009b57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b5260 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/4036302993 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009955d0 0x55580099c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4036302993 >> 172.21.15.16:0/4036302993 conn(0x55580095e310 msgr2=0x55580099b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4036302993 shutdown_connections 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 -- 172.21.15.16:0/4036302993 wait complete. 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.099+0000 7ff84a9c1740 1 -- start start 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 1 Processor -- start 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x5558009b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009a0370 0x5558009b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401ad5b0 con 0x555800996b40 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401aea20 con 0x5558009955d0 2024-09-15T22:43:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8401adc80 con 0x5558009a0370 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff84a9c1740 10 client.0 _ll_register_callbacks cb 0x7ff83c0980f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49602/0 (socket says 172.21.15.16:49602) 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 -- 172.21.15.16:0/1453722535 learned_addr learned my addr 172.21.15.16:0/1453722535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff846b85640 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009a0370 0x5558009b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff846384640 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x5558009b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 -- 172.21.15.16:0/1453722535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009a0370 msgr2=0x5558009b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009a0370 0x5558009b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 -- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 msgr2=0x5558009b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x5558009b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 -- 172.21.15.16:0/1453722535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558009b36a0 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff845b83640 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b2b20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff830031d10 tx=0x7ff8300315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff262ecc4b041ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.100+0000 7ff829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff829ffb640 1 -- 172.21.15.16:0/1453722535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830066060 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff829ffb640 1 -- 172.21.15.16:0/1453722535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7ff83003b630 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff83004c070 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff829ffb640 1 -- 172.21.15.16:0/1453722535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff83006d050 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 10 client.16434 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555800a1a5a0 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff829ffb640 1 -- 172.21.15.16:0/1453722535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83004c070 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff829ffb640 1 -- 172.21.15.16:0/1453722535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff830051420 con 0x555800996b40 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 10 client.16434 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:12.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 20 client.16434 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x555800a1cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 4 client.16434 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.101+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555800902d70 con 0x5558009afe50 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff846384640 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x555800a1cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff846384640 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x555800a1cc80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c673bf4f985e5788 server_cookie=3d5df4add15e868b in_seq=0 out_seq=0 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff829ffb640 10 client.16434 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff829ffb640 1 -- 172.21.15.16:0/1453722535 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555800902d70 con 0x5558009afe50 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff829ffb640 10 client.16434 handle_command_reply: tid=0 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff84a9c1740 1 client.16434 shutdown 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 msgr2=0x5558009b2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b2b20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff830031d10 tx=0x7ff8300315b0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 shutdown_connections 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.102+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5558009afe50 0x555800a1cc80 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5558009a0370 0x5558009b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555800996b40 0x5558009b2b20 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff84a9c1740 1 --2- 172.21.15.16:0/1453722535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5558009955d0 0x5558009b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 >> 172.21.15.16:0/1453722535 conn(0x55580095e310 msgr2=0x5558009ab420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff829ffb640 0 client.16434 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 shutdown_connections 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff84a9c1740 1 -- 172.21.15.16:0/1453722535 wait complete. 2024-09-15T22:43:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff84a9c1740 20 client.16434 trim_cache size 0 max 16384 2024-09-15T22:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:43:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff82c0a3430 msgr2=0x7ff82c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff82c0a3430 0x7ff82c0a58f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff838004ab0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102d70 msgr2=0x7ff8401985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102d70 0x7ff8401985e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c890 tx=0x7ff83c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 shutdown_connections 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff82c0a3430 0x7ff82c0a58f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8401082a0 0x7ff84019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff840103720 0x7ff840198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 --2- 172.21.15.16:0/2002995336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff840102d70 0x7ff8401985e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 >> 172.21.15.16:0/2002995336 conn(0x7ff8400fc7d0 msgr2=0x7ff840105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 shutdown_connections 2024-09-15T22:43:12.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:12.103+0000 7ff8485e8640 1 -- 172.21.15.16:0/2002995336 wait complete. 2024-09-15T22:43:12.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:12 smithi016 ceph-mon[31492]: pgmap v146: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:12.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:12 smithi130 ceph-mon[35742]: pgmap v146: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:12.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:12 smithi059 ceph-mon[34872]: pgmap v146: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:43:13.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:43:13.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:43:14.131 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:14.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:14 smithi016 ceph-mon[31492]: pgmap v147: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:43:14.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:14 smithi130 ceph-mon[35742]: pgmap v147: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:43:14.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 -- 172.21.15.16:0/836698806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24102790 msgr2=0x7fde241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 --2- 172.21.15.16:0/836698806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24102790 0x7fde241067e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009900 tx=0x7fde0c02a410 comp rx=0 tx=0).stop 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 -- 172.21.15.16:0/836698806 shutdown_connections 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 --2- 172.21.15.16:0/836698806 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde24106f10 0x7fde2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 --2- 172.21.15.16:0/836698806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24102790 0x7fde241067e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 --2- 172.21.15.16:0/836698806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24101de0 0x7fde241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 -- 172.21.15.16:0/836698806 >> 172.21.15.16:0/836698806 conn(0x7fde240fd8c0 msgr2=0x7fde240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 -- 172.21.15.16:0/836698806 shutdown_connections 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.601+0000 7fde2a513640 1 -- 172.21.15.16:0/836698806 wait complete. 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 Processor -- start 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 -- start start 2024-09-15T22:43:14.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101de0 0x7fde241984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde24102790 0x7fde241989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24106f10 0x7fde2419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde24111f70 con 0x7fde24101de0 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde24111df0 con 0x7fde24106f10 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde2a513640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fde241120f0 con 0x7fde24102790 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101de0 0x7fde241984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101de0 0x7fde241984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49614/0 (socket says 172.21.15.16:49614) 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde23fff640 1 -- 172.21.15.16:0/2908117361 learned_addr learned my addr 172.21.15.16:0/2908117361 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde1bfff640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde24102790 0x7fde241989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.602+0000 7fde28ab0640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24106f10 0x7fde2419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde23fff640 1 -- 172.21.15.16:0/2908117361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde24102790 msgr2=0x7fde241989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde23fff640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde24102790 0x7fde241989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde23fff640 1 -- 172.21.15.16:0/2908117361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24106f10 msgr2=0x7fde2419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde23fff640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24106f10 0x7fde2419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde23fff640 1 -- 172.21.15.16:0/2908117361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2419d460 con 0x7fde24101de0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde28ab0640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24106f10 0x7fde2419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde23fff640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101de0 0x7fde241984b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fde1400c970 tx=0x7fde1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde21ffb640 1 -- 172.21.15.16:0/2908117361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14007bb0 con 0x7fde24101de0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde21ffb640 1 -- 172.21.15.16:0/2908117361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde14007d50 con 0x7fde24101de0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2419d750 con 0x7fde24101de0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde21ffb640 1 -- 172.21.15.16:0/2908117361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14018800 con 0x7fde24101de0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.603+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241a5030 con 0x7fde24101de0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.606+0000 7fde21ffb640 1 -- 172.21.15.16:0/2908117361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fde1400f450 con 0x7fde24101de0 2024-09-15T22:43:14.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.606+0000 7fde2c8ec740 1 Processor -- start 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.606+0000 7fde2c8ec740 1 -- start start 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e60f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e6e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d03dc6280 con 0x563d03e61480 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563d03d6db10 con 0x563d03e6bd40 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563d03dc2240 con 0x563d03e60b60 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49626/0 (socket says 172.21.15.16:49626) 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 -- 172.21.15.16:0/4221921472 learned_addr learned my addr 172.21.15.16:0/4221921472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde28ab0640 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e60f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde1bfff640 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e6e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde21ffb640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fde100a33b0 0x7fde100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 -- 172.21.15.16:0/4221921472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 msgr2=0x563d03e60f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e60f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 -- 172.21.15.16:0/4221921472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 msgr2=0x563d03e6e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e6e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde23fff640 1 -- 172.21.15.16:0/4221921472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d03d7a1f0 con 0x563d03e61480 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde1bfff640 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e6e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.607+0000 7fde28ab0640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fde100a33b0 0x7fde100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.608+0000 7fde21ffb640 1 -- 172.21.15.16:0/2908117361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fde140d0100 con 0x7fde24101de0 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.608+0000 7fde23fff640 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6b800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fde140cd460 tx=0x7fde140669c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1860caf213d4da9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.608+0000 7fde1a7fc640 1 -- 172.21.15.16:0/4221921472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1409be50 con 0x563d03e61480 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.608+0000 7fde1a7fc640 1 -- 172.21.15.16:0/4221921472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1406d0d0 con 0x563d03e61480 2024-09-15T22:43:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.608+0000 7fde1a7fc640 1 -- 172.21.15.16:0/4221921472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14067160 con 0x563d03e61480 2024-09-15T22:43:14.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4221921472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 msgr2=0x563d03e6b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6b800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fde140cd460 tx=0x7fde140669c0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde28ab0640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fde100a33b0 0x7fde100a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005fd0 tx=0x7fde1c004a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4221921472 shutdown_connections 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e6e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6b800 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4221921472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e60f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4221921472 >> 172.21.15.16:0/4221921472 conn(0x563d03e5d0f0 msgr2=0x563d03e5d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4221921472 shutdown_connections 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4221921472 wait complete. 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 -- start start 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.609+0000 7fde2c8ec740 1 Processor -- start 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e61480 0x563d03e690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e69600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563d03e6e8b0 con 0x563d03e6bd40 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563d03d80f10 con 0x563d03e61480 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563d03e6ea30 con 0x563d03e60b60 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 10 client.0 _ll_register_callbacks cb 0x563d03d49a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e69600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49628/0 (socket says 172.21.15.16:49628) 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 -- 172.21.15.16:0/2287982967 learned_addr learned my addr 172.21.15.16:0/2287982967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde23fff640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde1bfff640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e61480 0x563d03e690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 -- 172.21.15.16:0/2287982967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 msgr2=0x563d03e68b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e68b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 -- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e61480 msgr2=0x563d03e690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e61480 0x563d03e690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 -- 172.21.15.16:0/2287982967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d03b376a0 con 0x563d03e6bd40 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde1bfff640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e61480 0x563d03e690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde28ab0640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e69600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009b60 tx=0x7fde1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92ebc3a51d7d816 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde037fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde037fe640 1 -- 172.21.15.16:0/2287982967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c02b060 con 0x563d03e6bd40 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde037fe640 1 -- 172.21.15.16:0/2287982967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fde1c018070 con 0x563d03e6bd40 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.610+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1406d0d0 con 0x563d03e6bd40 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde037fe640 1 -- 172.21.15.16:0/2287982967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde1c032050 con 0x563d03e6bd40 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 10 client.16446 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d03e7a480 con 0x563d03e6bd40 2024-09-15T22:43:14.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde037fe640 1 -- 172.21.15.16:0/2287982967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1406d0d0 con 0x563d03e6bd40 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde037fe640 1 -- 172.21.15.16:0/2287982967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fde1c00f040 con 0x563d03e6bd40 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 10 client.16446 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 20 client.16446 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03e7b940 0x563d03e7dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 4 client.16446 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d03dd37f0 con 0x563d03e7b940 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.611+0000 7fde23fff640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03e7b940 0x563d03e7dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.612+0000 7fde23fff640 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03e7b940 0x563d03e7dd90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82c090b655edda4 server_cookie=5c81830a037f6460 in_seq=0 out_seq=0 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.612+0000 7fde037fe640 10 client.16446 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.612+0000 7fde037fe640 1 -- 172.21.15.16:0/2287982967 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d03dd37f0 con 0x563d03e7b940 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.612+0000 7fde037fe640 10 client.16446 handle_command_reply: tid=0 2024-09-15T22:43:14.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.612+0000 7fde2c8ec740 1 client.16446 shutdown 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 msgr2=0x563d03e69600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e69600 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009b60 tx=0x7fde1c004ab0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 shutdown_connections 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03e7b940 0x563d03e7dd90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e69600 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e61480 0x563d03e690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/2287982967 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e68b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 >> 172.21.15.16:0/2287982967 conn(0x563d03e5d0f0 msgr2=0x563d03e62e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde037fe640 0 client.16446 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 shutdown_connections 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 1 -- 172.21.15.16:0/2287982967 wait complete. 2024-09-15T22:43:14.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.613+0000 7fde2c8ec740 20 client.16446 trim_cache size 0 max 16384 2024-09-15T22:43:14.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 Processor -- start 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 -- start start 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e74060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b1800 con 0x563d03e61480 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde241af0d0 con 0x563d03e6bd40 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fde241acf20 con 0x563d03e60b60 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.630+0000 7fde1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49630/0 (socket says 172.21.15.16:49630) 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 -- 172.21.15.16:0/1337245042 learned_addr learned my addr 172.21.15.16:0/1337245042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde23fff640 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde28ab0640 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e74060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 -- 172.21.15.16:0/1337245042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 msgr2=0x563d03e635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 -- 172.21.15.16:0/1337245042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 msgr2=0x563d03e74060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e74060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 -- 172.21.15.16:0/1337245042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d03e58830 con 0x563d03e61480 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde28ab0640 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e74060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde1bfff640 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6a9e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009900 tx=0x7fde0c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deada3acb8f64be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde19ffb640 1 -- 172.21.15.16:0/1337245042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c035040 con 0x563d03e61480 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde19ffb640 1 -- 172.21.15.16:0/1337245042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0c005020 con 0x563d03e61480 2024-09-15T22:43:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.631+0000 7fde19ffb640 1 -- 172.21.15.16:0/1337245042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0c004bc0 con 0x563d03e61480 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 -- 172.21.15.16:0/1337245042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 msgr2=0x563d03e6a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6a9e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009900 tx=0x7fde0c031630 comp rx=0 tx=0).stop 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 -- 172.21.15.16:0/1337245042 shutdown_connections 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e6bd40 0x563d03e74060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e61480 0x563d03e6a9e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/1337245042 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e60b60 0x563d03e635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 -- 172.21.15.16:0/1337245042 >> 172.21.15.16:0/1337245042 conn(0x563d03e5d0f0 msgr2=0x563d03e6a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 -- 172.21.15.16:0/1337245042 shutdown_connections 2024-09-15T22:43:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 -- 172.21.15.16:0/1337245042 wait complete. 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 -- start start 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.632+0000 7fde2c8ec740 1 Processor -- start 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e60b60 0x563d03ee30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e61480 0x563d03ee3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde2c8ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde241ad590 con 0x563d03e6bd40 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fde241aea00 con 0x563d03e60b60 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde2c8ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fde241adc60 con 0x563d03e61480 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde2c8ec740 10 client.0 _ll_register_callbacks cb 0x7fde1c009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e833d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49640/0 (socket says 172.21.15.16:49640) 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 -- 172.21.15.16:0/4099590505 learned_addr learned my addr 172.21.15.16:0/4099590505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde1bfff640 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e61480 0x563d03ee3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde23fff640 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e60b60 0x563d03ee30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 -- 172.21.15.16:0/4099590505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e61480 msgr2=0x563d03ee3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e61480 0x563d03ee3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 -- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e60b60 msgr2=0x563d03ee30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e60b60 0x563d03ee30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 -- 172.21.15.16:0/4099590505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d03e612b0 con 0x563d03e6bd40 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde28ab0640 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e833d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde1c01cad0 tx=0x7fde1c013da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991728c91a930dad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.633+0000 7fde037fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde037fe640 1 -- 172.21.15.16:0/4099590505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c02b020 con 0x563d03e6bd40 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde037fe640 1 -- 172.21.15.16:0/4099590505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fde1c01e6f0 con 0x563d03e6bd40 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde0c005020 con 0x563d03e6bd40 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde037fe640 1 -- 172.21.15.16:0/4099590505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fde1c03e050 con 0x563d03e6bd40 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 10 client.16452 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563d03eea6c0 con 0x563d03e6bd40 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde037fe640 1 -- 172.21.15.16:0/4099590505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde0c005020 con 0x563d03e6bd40 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde037fe640 1 -- 172.21.15.16:0/4099590505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fde1c026970 con 0x563d03e6bd40 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 10 client.16452 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 20 client.16452 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03eeae40 0x563d03eed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 4 client.16452 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.634+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d03dcdd00 con 0x563d03eeae40 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.635+0000 7fde23fff640 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03eeae40 0x563d03eed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.635+0000 7fde23fff640 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03eeae40 0x563d03eed260 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d3ca6899cdad6d server_cookie=214e05888d63186b in_seq=0 out_seq=0 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.635+0000 7fde037fe640 10 client.16452 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde037fe640 1 -- 172.21.15.16:0/4099590505 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d03dcdd00 con 0x563d03eeae40 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde037fe640 10 client.16452 handle_command_reply: tid=0 2024-09-15T22:43:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 client.16452 shutdown 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 msgr2=0x563d03e833d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e833d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fde1c01cad0 tx=0x7fde1c013da0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 shutdown_connections 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563d03eeae40 0x563d03eed260 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563d03e6bd40 0x563d03e833d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563d03e61480 0x563d03ee3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 --2- 172.21.15.16:0/4099590505 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d03e60b60 0x563d03ee30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 >> 172.21.15.16:0/4099590505 conn(0x563d03e5d0f0 msgr2=0x563d03e5c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde037fe640 0 client.16452 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 shutdown_connections 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.636+0000 7fde2c8ec740 1 -- 172.21.15.16:0/4099590505 wait complete. 2024-09-15T22:43:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2c8ec740 20 client.16452 trim_cache size 0 max 16384 2024-09-15T22:43:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:43:14.644 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:43:14.644 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "3d32d9de-8fe5-4272-89a6-f6f28e3b2fa8", 2024-09-15T22:43:14.644 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:43:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:43:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fde100a33b0 msgr2=0x7fde100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fde100a33b0 0x7fde100a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005fd0 tx=0x7fde1c004a40 comp rx=0 tx=0).stop 2024-09-15T22:43:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101de0 msgr2=0x7fde241984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101de0 0x7fde241984b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fde1400c970 tx=0x7fde1400ce30 comp rx=0 tx=0).stop 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 shutdown_connections 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24106f10 0x7fde2419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fde100a33b0 0x7fde100a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fde24102790 0x7fde241989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 --2- 172.21.15.16:0/2908117361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde24101de0 0x7fde241984b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 >> 172.21.15.16:0/2908117361 conn(0x7fde240fd8c0 msgr2=0x7fde240fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 shutdown_connections 2024-09-15T22:43:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:14.637+0000 7fde2a513640 1 -- 172.21.15.16:0/2908117361 wait complete. 2024-09-15T22:43:14.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:14 smithi059 ceph-mon[34872]: pgmap v147: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:43:15.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:43:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:16.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:16 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:43:16.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:16 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:16.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:16 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:43:16.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:16 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:16.303 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:16 smithi016 ceph-mon[31492]: pgmap v148: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:43:16.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 -- 172.21.15.16:0/4244341824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604102ef0 msgr2=0x7fa60410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 --2- 172.21.15.16:0/4244341824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604102ef0 0x7fa60410da40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009ca0 tx=0x7fa5fc02a950 comp rx=0 tx=0).stop 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 -- 172.21.15.16:0/4244341824 shutdown_connections 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 --2- 172.21.15.16:0/4244341824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604102ef0 0x7fa60410da40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 --2- 172.21.15.16:0/4244341824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa604102570 0x7fa6041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 --2- 172.21.15.16:0/4244341824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa604100e20 0x7fa604101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 -- 172.21.15.16:0/4244341824 >> 172.21.15.16:0/4244341824 conn(0x7fa6040fc7d0 msgr2=0x7fa6040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 -- 172.21.15.16:0/4244341824 shutdown_connections 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.300+0000 7fa60b957640 1 -- 172.21.15.16:0/4244341824 wait complete. 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 Processor -- start 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 -- start start 2024-09-15T22:43:16.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100e20 0x7fa6040fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa604102570 0x7fa6040fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa604102ef0 0x7fa6040fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa604110f80 con 0x7fa604100e20 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa604110e00 con 0x7fa604102ef0 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa60b957640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa604111100 con 0x7fa604102570 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa6096f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100e20 0x7fa6040fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa6096f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100e20 0x7fa6040fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49664/0 (socket says 172.21.15.16:49664) 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa6096f3640 1 -- 172.21.15.16:0/2352128498 learned_addr learned my addr 172.21.15.16:0/2352128498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa604102570 0x7fa6040fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa609ef4640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa604102ef0 0x7fa6040fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa6096f3640 1 -- 172.21.15.16:0/2352128498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa604102570 msgr2=0x7fa6040fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.301+0000 7fa6096f3640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa604102570 0x7fa6040fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa6096f3640 1 -- 172.21.15.16:0/2352128498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa604102ef0 msgr2=0x7fa6040fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa6096f3640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa604102ef0 0x7fa6040fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa6096f3640 1 -- 172.21.15.16:0/2352128498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6041a68b0 con 0x7fa604100e20 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa604102570 0x7fa6040fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa609ef4640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa604102ef0 0x7fa6040fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa6096f3640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100e20 0x7fa6040fcc80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d770 tx=0x7fa60000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/2352128498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600017070 con 0x7fa604100e20 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/2352128498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa600005500 con 0x7fa604100e20 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/2352128498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600004e00 con 0x7fa604100e20 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6041a6ba0 con 0x7fa604100e20 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.302+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6041a7090 con 0x7fa604100e20 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.304+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/2352128498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa600004fa0 con 0x7fa604100e20 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 Processor -- start 2024-09-15T22:43:16.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 -- start start 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b085fd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fddc0 0x564b08608140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0860aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b085632e0 con 0x564b085fd4a0 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b0850ace0 con 0x564b085fddc0 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa60dd30740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564b0855f2a0 con 0x564b08608680 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa5fa7fc640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa5f00a33b0 0x7fa5f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa609ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b085fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa609ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b085fd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49678/0 (socket says 172.21.15.16:49678) 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa609ef4640 1 -- 172.21.15.16:0/1426583866 learned_addr learned my addr 172.21.15.16:0/1426583866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fddc0 0x564b08608140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.305+0000 7fa5fa7fc640 1 -- 172.21.15.16:0/2352128498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa6000cfc10 con 0x7fa604100e20 2024-09-15T22:43:16.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa609ef4640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa5f00a33b0 0x7fa5f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa608ef2640 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0860aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa6096f3640 1 -- 172.21.15.16:0/1426583866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 msgr2=0x564b0860aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0860aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa6096f3640 1 -- 172.21.15.16:0/1426583866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 msgr2=0x564b085fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b085fd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa6096f3640 1 -- 172.21.15.16:0/1426583866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b084ef440 con 0x564b085fddc0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa609ef4640 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b085fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fddc0 0x564b08608140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6000ccfa0 tx=0x7fa60006e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62acaa4d61104eb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa609ef4640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa5f00a33b0 0x7fa5f00a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009430 tx=0x7fa5fc02a950 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa5ef7fe640 1 -- 172.21.15.16:0/1426583866 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600020030 con 0x564b085fddc0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa5ef7fe640 1 -- 172.21.15.16:0/1426583866 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa60009bb30 con 0x564b085fddc0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.306+0000 7fa5ef7fe640 1 -- 172.21.15.16:0/1426583866 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000981a0 con 0x564b085fddc0 2024-09-15T22:43:16.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 -- 172.21.15.16:0/1426583866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fddc0 msgr2=0x564b08608140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fddc0 0x564b08608140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa6000ccfa0 tx=0x7fa60006e960 comp rx=0 tx=0).stop 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 -- 172.21.15.16:0/1426583866 shutdown_connections 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0860aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fddc0 0x564b08608140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1426583866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b085fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 -- 172.21.15.16:0/1426583866 >> 172.21.15.16:0/1426583866 conn(0x564b085f9a30 msgr2=0x564b085f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 -- 172.21.15.16:0/1426583866 shutdown_connections 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 -- 172.21.15.16:0/1426583866 wait complete. 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 -- start start 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.307+0000 7fa60dd30740 1 Processor -- start 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b08607ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b086052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b08608680 0x564b08605800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa60dd30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564b0860b1f0 con 0x564b085fd4a0 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa60dd30740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b0851e050 con 0x564b08608680 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa60dd30740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564b0860b370 con 0x564b085fddc0 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa60dd30740 10 client.0 _ll_register_callbacks cb 0x564b0855e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b08607ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b08607ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49682/0 (socket says 172.21.15.16:49682) 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 -- 172.21.15.16:0/2948680323 learned_addr learned my addr 172.21.15.16:0/2948680323 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b086052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa609ef4640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b08608680 0x564b08605800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 -- 172.21.15.16:0/2948680323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 msgr2=0x564b086052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b086052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 -- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b08608680 msgr2=0x564b08605800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b08608680 0x564b08605800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 -- 172.21.15.16:0/2948680323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b082d43c0 con 0x564b085fd4a0 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b086052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa6096f3640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b08607ff0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6000cff00 tx=0x7fa600009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc416ec4869bf44e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.308+0000 7fa5ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa5ecff9640 1 -- 172.21.15.16:0/2948680323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60009ce60 con 0x564b085fd4a0 2024-09-15T22:43:16.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa5ecff9640 1 -- 172.21.15.16:0/2948680323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa6000981a0 con 0x564b085fd4a0 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa60009bb30 con 0x564b085fd4a0 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa5ecff9640 1 -- 172.21.15.16:0/2948680323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa600020040 con 0x564b085fd4a0 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 10 client.16464 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b08616bc0 con 0x564b085fd4a0 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa5ecff9640 1 -- 172.21.15.16:0/2948680323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa60009bb30 con 0x564b085fd4a0 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa5ecff9640 1 -- 172.21.15.16:0/2948680323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa6000cd2b0 con 0x564b085fd4a0 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 10 client.16464 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 20 client.16464 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08617f10 0x564b0861a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 4 client.16464 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.309+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b08570850 con 0x564b08617f10 2024-09-15T22:43:16.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.310+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08617f10 0x564b0861a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.310+0000 7fa5ecff9640 10 client.16464 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.310+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08617f10 0x564b0861a300 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d754d64ea856ad7 server_cookie=54fe239e9db1838 in_seq=0 out_seq=0 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa5ecff9640 1 -- 172.21.15.16:0/2948680323 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b08570850 con 0x564b08617f10 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa5ecff9640 10 client.16464 handle_command_reply: tid=0 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 client.16464 shutdown 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 msgr2=0x564b08607ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b08607ff0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6000cff00 tx=0x7fa600009c30 comp rx=0 tx=0).stop 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 shutdown_connections 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08617f10 0x564b0861a300 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b08608680 0x564b08605800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b086052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2948680323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fd4a0 0x564b08607ff0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 >> 172.21.15.16:0/2948680323 conn(0x564b085f9a30 msgr2=0x564b085ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:16.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa5ecff9640 0 client.16464 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 shutdown_connections 2024-09-15T22:43:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 1 -- 172.21.15.16:0/2948680323 wait complete. 2024-09-15T22:43:16.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.311+0000 7fa60dd30740 20 client.16464 trim_cache size 0 max 16384 2024-09-15T22:43:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.328+0000 7fa60dd30740 1 Processor -- start 2024-09-15T22:43:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.328+0000 7fa60dd30740 1 -- start start 2024-09-15T22:43:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b086007a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fddc0 0x564b0861cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0861d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa60dd30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b3910 con 0x564b085fddc0 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa60dd30740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b11e0 con 0x564b085fd4a0 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa60dd30740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041af030 con 0x564b08608680 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fddc0 0x564b0861cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fddc0 0x564b0861cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49688/0 (socket says 172.21.15.16:49688) 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 -- 172.21.15.16:0/2670940693 learned_addr learned my addr 172.21.15.16:0/2670940693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa609ef4640 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0861d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa6096f3640 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b086007a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 -- 172.21.15.16:0/2670940693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 msgr2=0x564b0861d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0861d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 -- 172.21.15.16:0/2670940693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 msgr2=0x564b086007a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b086007a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 -- 172.21.15.16:0/2670940693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b085fdbf0 con 0x564b085fddc0 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.329+0000 7fa608ef2640 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fddc0 0x564b0861cf40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x564b08604b30 tx=0x7fa5f400d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fbb2d93d480b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa5eeffd640 1 -- 172.21.15.16:0/2670940693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4005050 con 0x564b085fddc0 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa5eeffd640 1 -- 172.21.15.16:0/2670940693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f4004730 con 0x564b085fddc0 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa5eeffd640 1 -- 172.21.15.16:0/2670940693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4019590 con 0x564b085fddc0 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 -- 172.21.15.16:0/2670940693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fddc0 msgr2=0x564b0861cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fddc0 0x564b0861cf40 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x564b08604b30 tx=0x7fa5f400d3a0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 -- 172.21.15.16:0/2670940693 shutdown_connections 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b08608680 0x564b0861d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b085fddc0 0x564b0861cf40 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 --2- 172.21.15.16:0/2670940693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b086007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 -- 172.21.15.16:0/2670940693 >> 172.21.15.16:0/2670940693 conn(0x564b085f9a30 msgr2=0x564b08600090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.330+0000 7fa60dd30740 1 -- 172.21.15.16:0/2670940693 shutdown_connections 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 -- 172.21.15.16:0/2670940693 wait complete. 2024-09-15T22:43:16.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 -- start start 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 Processor -- start 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b0861d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b0861d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b08608680 0x564b0861a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041af6a0 con 0x564b08608680 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041b0b10 con 0x564b085fd4a0 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa6041afd70 con 0x564b085fddc0 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.331+0000 7fa60dd30740 10 client.0 _ll_register_callbacks cb 0x7fa5fc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b08608680 0x564b0861a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b08608680 0x564b0861a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49704/0 (socket says 172.21.15.16:49704) 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 -- 172.21.15.16:0/1307502028 learned_addr learned my addr 172.21.15.16:0/1307502028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b0861d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa608ef2640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b0861d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 -- 172.21.15.16:0/1307502028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 msgr2=0x564b0861d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b0861d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 -- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 msgr2=0x564b0861d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b0861d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 -- 172.21.15.16:0/1307502028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b08608360 con 0x564b08608680 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b0861d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa609ef4640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b08608680 0x564b0861a840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc02f720 tx=0x7fa5fc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3aa2da484aad69 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa5ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa5ecff9640 1 -- 172.21.15.16:0/1307502028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc034040 con 0x564b08608680 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa5ecff9640 1 -- 172.21.15.16:0/1307502028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa5fc004450 con 0x564b08608680 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.332+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f4004730 con 0x564b08608680 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa5ecff9640 1 -- 172.21.15.16:0/1307502028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa5fc004650 con 0x564b08608680 2024-09-15T22:43:16.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 10 client.16470 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b08682300 con 0x564b08608680 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa5ecff9640 1 -- 172.21.15.16:0/1307502028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f4004730 con 0x564b08608680 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa5ecff9640 1 -- 172.21.15.16:0/1307502028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa5fc048040 con 0x564b08608680 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 10 client.16470 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 10 client.16470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 20 client.16470 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08682a50 0x564b08684e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 4 client.16470 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564b0856ad60 con 0x564b08682a50 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.333+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08682a50 0x564b08684e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa6096f3640 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08682a50 0x564b08684e70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae864d132c7ca18 server_cookie=682f44344d920624 in_seq=0 out_seq=0 2024-09-15T22:43:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa5ecff9640 10 client.16470 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa5ecff9640 1 -- 172.21.15.16:0/1307502028 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564b0856ad60 con 0x564b08682a50 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa5ecff9640 10 client.16470 handle_command_reply: tid=0 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 client.16470 shutdown 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b08608680 msgr2=0x564b0861a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b08608680 0x564b0861a840 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc02f720 tx=0x7fa5fc0095c0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 shutdown_connections 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564b08682a50 0x564b08684e70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564b08608680 0x564b0861a840 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564b085fddc0 0x564b0861d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa5ecff9640 0 client.16470 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 --2- 172.21.15.16:0/1307502028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b085fd4a0 0x564b0861d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.334+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 >> 172.21.15.16:0/1307502028 conn(0x564b085f9a30 msgr2=0x564b085f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 shutdown_connections 2024-09-15T22:43:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60dd30740 1 -- 172.21.15.16:0/1307502028 wait complete. 2024-09-15T22:43:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60dd30740 20 client.16470 trim_cache size 0 max 16384 2024-09-15T22:43:16.352 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:43:16.352 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:43:16.352 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:43:16.352 INFO:teuthology.orchestra.run.smithi016.stdout: "3d32d9de-8fe5-4272-89a6-f6f28e3b2fa8": { 2024-09-15T22:43:16.352 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:43:16.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "3d32d9de-8fe5-4272-89a6-f6f28e3b2fa8", 2024-09-15T22:43:16.353 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:43:16.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:43:16.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:43:16.353 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:43:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa5f00a33b0 msgr2=0x7fa5f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa5f00a33b0 0x7fa5f00a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009430 tx=0x7fa5fc02a950 comp rx=0 tx=0).stop 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100e20 msgr2=0x7fa6040fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100e20 0x7fa6040fcc80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa60000d770 tx=0x7fa60000dc30 comp rx=0 tx=0).stop 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 shutdown_connections 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa604102ef0 0x7fa6040fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa5f00a33b0 0x7fa5f00a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa604102570 0x7fa6040fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 --2- 172.21.15.16:0/2352128498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa604100e20 0x7fa6040fcc80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.335+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 >> 172.21.15.16:0/2352128498 conn(0x7fa6040fc7d0 msgr2=0x7fa60406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.336+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 shutdown_connections 2024-09-15T22:43:16.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:16.336+0000 7fa60b957640 1 -- 172.21.15.16:0/2352128498 wait complete. 2024-09-15T22:43:16.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:16 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:43:16.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:16 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:16.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:16 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:43:16.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:16 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:16.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:16 smithi130 ceph-mon[35742]: pgmap v148: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:43:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:16 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:43:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:16 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:16 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:43:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:16 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:16 smithi059 ceph-mon[34872]: pgmap v148: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:43:17.118 INFO:tasks.cephfs.filesystem:scrub status for tag:3d32d9de-8fe5-4272-89a6-f6f28e3b2fa8 - {'path': '/', 'tag': '3d32d9de-8fe5-4272-89a6-f6f28e3b2fa8', 'options': 'recursive,force'} 2024-09-15T22:43:18.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:18 smithi130 ceph-mon[35742]: pgmap v149: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-15T22:43:18.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:18 smithi016 ceph-mon[31492]: pgmap v149: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-15T22:43:18.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:18 smithi059 ceph-mon[34872]: pgmap v149: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-15T22:43:19.500 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:19 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:43:19.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:19 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:43:19.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:19 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:43:20.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:20 smithi130 ceph-mon[35742]: pgmap v150: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:43:20.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:20 smithi016 ceph-mon[31492]: pgmap v150: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:43:20.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:20 smithi059 ceph-mon[34872]: pgmap v150: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:43:22.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:22 smithi130 ceph-mon[35742]: pgmap v151: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:43:22.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:22 smithi016 ceph-mon[31492]: pgmap v151: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:43:22.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:22 smithi059 ceph-mon[34872]: pgmap v151: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:43:24.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:24 smithi130 ceph-mon[35742]: pgmap v152: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:43:24.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:24 smithi016 ceph-mon[31492]: pgmap v152: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:43:24.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:24 smithi059 ceph-mon[34872]: pgmap v152: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:43:26.496 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:26 smithi059 ceph-mon[34872]: pgmap v153: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:43:26.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:26 smithi130 ceph-mon[35742]: pgmap v153: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:43:26.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:26 smithi016 ceph-mon[31492]: pgmap v153: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:43:28.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:28 smithi130 ceph-mon[35742]: pgmap v154: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:43:28.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:28 smithi016 ceph-mon[31492]: pgmap v154: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:43:28.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:28 smithi059 ceph-mon[34872]: pgmap v154: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:43:30.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:30 smithi130 ceph-mon[35742]: pgmap v155: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:30.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:30 smithi016 ceph-mon[31492]: pgmap v155: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:30.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:30 smithi059 ceph-mon[34872]: pgmap v155: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:32.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:32 smithi130 ceph-mon[35742]: pgmap v156: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:32.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:32 smithi016 ceph-mon[31492]: pgmap v156: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:32.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:32 smithi059 ceph-mon[34872]: pgmap v156: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:34.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:34 smithi130 ceph-mon[35742]: pgmap v157: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:34.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:34 smithi016 ceph-mon[31492]: pgmap v157: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:34 smithi059 ceph-mon[34872]: pgmap v157: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:36.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:36 smithi130 ceph-mon[35742]: pgmap v158: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:36.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:36 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:36.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:36 smithi016 ceph-mon[31492]: pgmap v158: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:36.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:36 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:36.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:36 smithi059 ceph-mon[34872]: pgmap v158: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:36.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:36 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:37.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:37 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:37 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:37.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:37 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:43:37.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:37 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:37 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:37.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:37 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:43:37.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:37 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:37 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:37 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:43:38.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:38 smithi059 ceph-mon[34872]: pgmap v159: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:38 smithi130 ceph-mon[35742]: pgmap v159: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:39.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:38 smithi016 ceph-mon[31492]: pgmap v159: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:40.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:40 smithi059 ceph-mon[34872]: pgmap v160: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:41.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:40 smithi130 ceph-mon[35742]: pgmap v160: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:41.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:40 smithi016 ceph-mon[31492]: pgmap v160: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-15T22:43:42.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:42 smithi059 ceph-mon[34872]: pgmap v161: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:43.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:42 smithi130 ceph-mon[35742]: pgmap v161: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:42 smithi016 ceph-mon[31492]: pgmap v161: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:44.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:44 smithi059 ceph-mon[34872]: pgmap v162: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:45.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:44 smithi130 ceph-mon[35742]: pgmap v162: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:45.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:44 smithi016 ceph-mon[31492]: pgmap v162: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:46.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:46 smithi059 ceph-mon[34872]: pgmap v163: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:47.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:46 smithi130 ceph-mon[35742]: pgmap v163: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:47.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:46 smithi016 ceph-mon[31492]: pgmap v163: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:47.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:43:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 -- 172.21.15.16:0/938844845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc102d40 msgr2=0x7f7bdc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 --2- 172.21.15.16:0/938844845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc102d40 0x7f7bdc103120 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009900 tx=0x7f7bd002a480 comp rx=0 tx=0).stop 2024-09-15T22:43:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 -- 172.21.15.16:0/938844845 shutdown_connections 2024-09-15T22:43:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 --2- 172.21.15.16:0/938844845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bdc10c170 0x7f7bdc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 --2- 172.21.15.16:0/938844845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bdc103660 0x7f7bdc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 --2- 172.21.15.16:0/938844845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc102d40 0x7f7bdc103120 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 -- 172.21.15.16:0/938844845 >> 172.21.15.16:0/938844845 conn(0x7f7bdc0fc7a0 msgr2=0x7f7bdc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 -- 172.21.15.16:0/938844845 shutdown_connections 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.887+0000 7f7be3078640 1 -- 172.21.15.16:0/938844845 wait complete. 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 Processor -- start 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 -- start start 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bdc102d40 0x7f7bdc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bdc103660 0x7f7bdc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc10c170 0x7f7bdc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc111f70 con 0x7f7bdc10c170 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc111df0 con 0x7f7bdc102d40 2024-09-15T22:43:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be3078640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1120f0 con 0x7f7bdc103660 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc10c170 0x7f7bdc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc10c170 0x7f7bdc19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45364/0 (socket says 172.21.15.16:45364) 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 -- 172.21.15.16:0/40579973 learned_addr learned my addr 172.21.15.16:0/40579973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be0e14640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bdc102d40 0x7f7bdc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bdc103660 0x7f7bdc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 -- 172.21.15.16:0/40579973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bdc103660 msgr2=0x7f7bdc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bdc103660 0x7f7bdc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 -- 172.21.15.16:0/40579973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bdc102d40 msgr2=0x7f7bdc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bdc102d40 0x7f7bdc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be1615640 1 -- 172.21.15.16:0/40579973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc19d4a0 con 0x7f7bdc10c170 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.888+0000 7f7be0e14640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bdc102d40 0x7f7bdc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.889+0000 7f7be1615640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc10c170 0x7f7bdc19cdc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400cad0 tx=0x7f7bd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.889+0000 7f7bd9ffb640 1 -- 172.21.15.16:0/40579973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400ae20 con 0x7f7bdc10c170 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.889+0000 7f7bd9ffb640 1 -- 172.21.15.16:0/40579973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd4005020 con 0x7f7bdc10c170 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.889+0000 7f7bd9ffb640 1 -- 172.21.15.16:0/40579973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400f6f0 con 0x7f7bdc10c170 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.889+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc19d790 con 0x7f7bdc10c170 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.889+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc1a5070 con 0x7f7bdc10c170 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7bd9ffb640 1 -- 172.21.15.16:0/40579973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7bd40051d0 con 0x7f7bdc10c170 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 Processor -- start 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 -- start start 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074db9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db9dc0 0x55f074dc4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dc6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f074d1f2e0 con 0x55f074db9dc0 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f074cc6ce0 con 0x55f074dc4680 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.892+0000 7f7be5451740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f074d1b2a0 con 0x55f074db94a0 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db9dc0 0x55f074dc4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db9dc0 0x55f074dc4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45366/0 (socket says 172.21.15.16:45366) 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 -- 172.21.15.16:0/3315388438 learned_addr learned my addr 172.21.15.16:0/3315388438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074db9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7bd9ffb640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7bcc0a33b0 0x7f7bcc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dc6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be1615640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7bcc0a33b0 0x7f7bcc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 -- 172.21.15.16:0/3315388438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 msgr2=0x55f074db9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074db9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 -- 172.21.15.16:0/3315388438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 msgr2=0x55f074dc6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dc6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 -- 172.21.15.16:0/3315388438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f074cab440 con 0x55f074db9dc0 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dc6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7bd9ffb640 1 -- 172.21.15.16:0/40579973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7bd40d01c0 con 0x7f7bdc10c170 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be0e14640 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db9dc0 0x55f074dc4140 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0033000 tx=0x7f7bd0032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3684760a0ab1f81c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7be1615640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7bcc0a33b0 0x7f7bcc0a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bd409bde0 tx=0x7f7bd40bc040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.893+0000 7f7bc6ffd640 1 -- 172.21.15.16:0/3315388438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd003f070 con 0x55f074db9dc0 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7bc6ffd640 1 -- 172.21.15.16:0/3315388438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd00317b0 con 0x55f074db9dc0 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7bc6ffd640 1 -- 172.21.15.16:0/3315388438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0035070 con 0x55f074db9dc0 2024-09-15T22:43:47.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 -- 172.21.15.16:0/3315388438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db9dc0 msgr2=0x55f074dc4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db9dc0 0x55f074dc4140 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0033000 tx=0x7f7bd0032cb0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 -- 172.21.15.16:0/3315388438 shutdown_connections 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dc6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db9dc0 0x55f074dc4140 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 --2- 172.21.15.16:0/3315388438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074db9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 -- 172.21.15.16:0/3315388438 >> 172.21.15.16:0/3315388438 conn(0x55f074db5a30 msgr2=0x55f074db5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 -- 172.21.15.16:0/3315388438 shutdown_connections 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.894+0000 7f7be5451740 1 -- 172.21.15.16:0/3315388438 wait complete. 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 -- start start 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 Processor -- start 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dc3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dc1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dc1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f074dc7250 con 0x55f074dc4680 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f074cda050 con 0x55f074db9dc0 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f074dc73d0 con 0x55f074db94a0 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be5451740 10 client.0 _ll_register_callbacks cb 0x55f074d1a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dc1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dc1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45372/0 (socket says 172.21.15.16:45372) 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 -- 172.21.15.16:0/1208170241 learned_addr learned my addr 172.21.15.16:0/1208170241 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7be0e14640 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dc1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dc3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 -- 172.21.15.16:0/1208170241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 msgr2=0x55f074dc3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dc3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 -- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 msgr2=0x55f074dc1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.895+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dc1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bdbfff640 1 -- 172.21.15.16:0/1208170241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f074a903c0 con 0x55f074dc4680 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dc1760 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8013ad0 tx=0x7f7bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62d9edce17d78e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bbbfff640 1 -- 172.21.15.16:0/1208170241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8013080 con 0x55f074dc4680 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bbbfff640 1 -- 172.21.15.16:0/1208170241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7bc8013220 con 0x55f074dc4680 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bd00317b0 con 0x55f074dc4680 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bbbfff640 1 -- 172.21.15.16:0/1208170241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8026050 con 0x55f074dc4680 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 10 client.16482 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f074dd2c20 con 0x55f074dc4680 2024-09-15T22:43:47.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bbbfff640 1 -- 172.21.15.16:0/1208170241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd00317b0 con 0x55f074dc4680 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7bbbfff640 1 -- 172.21.15.16:0/1208170241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7bc801e020 con 0x55f074dc4680 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 10 client.16482 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 20 client.16482 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074dd40e0 0x55f074dd64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 4 client.16482 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.896+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f074d2c850 con 0x55f074dd40e0 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.897+0000 7f7be0e14640 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074dd40e0 0x55f074dd64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.897+0000 7f7be0e14640 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074dd40e0 0x55f074dd64d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=454afc8766e761b2 server_cookie=1393e39ed63a7eb2 in_seq=0 out_seq=0 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.897+0000 7f7bbbfff640 10 client.16482 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7bbbfff640 1 -- 172.21.15.16:0/1208170241 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f074d2c850 con 0x55f074dd40e0 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7bbbfff640 10 client.16482 handle_command_reply: tid=0 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 client.16482 shutdown 2024-09-15T22:43:47.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 msgr2=0x55f074dc1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dc1760 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8013ad0 tx=0x7f7bc80046c0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 shutdown_connections 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074dd40e0 0x55f074dd64d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dc1760 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dc1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 --2- 172.21.15.16:0/1208170241 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dc3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7bbbfff640 0 client.16482 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 >> 172.21.15.16:0/1208170241 conn(0x55f074db5a30 msgr2=0x55f074dbb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 shutdown_connections 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 1 -- 172.21.15.16:0/1208170241 wait complete. 2024-09-15T22:43:47.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.898+0000 7f7be5451740 20 client.16482 trim_cache size 0 max 16384 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.915+0000 7f7be5451740 1 Processor -- start 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.915+0000 7f7be5451740 1 -- start start 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be5451740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db94a0 0x55f074ddbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be5451740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db9dc0 0x55f074ddc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be5451740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dbe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be5451740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1b17a0 con 0x55f074db94a0 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be5451740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1af070 con 0x55f074dc4680 2024-09-15T22:43:47.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be5451740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1acec0 con 0x55f074db9dc0 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db94a0 0x55f074ddbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db94a0 0x55f074ddbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45388/0 (socket says 172.21.15.16:45388) 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 -- 172.21.15.16:0/991387160 learned_addr learned my addr 172.21.15.16:0/991387160 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be0e14640 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db9dc0 0x55f074ddc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 -- 172.21.15.16:0/991387160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db9dc0 msgr2=0x55f074ddc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db9dc0 0x55f074ddc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 -- 172.21.15.16:0/991387160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 msgr2=0x55f074dbe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dbe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 -- 172.21.15.16:0/991387160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f074dba330 con 0x55f074db94a0 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7be0e14640 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db9dc0 0x55f074ddc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db94a0 0x55f074ddbd00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009250 tx=0x7f7bc8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67c088e383f33a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.916+0000 7f7bc67fc640 1 -- 172.21.15.16:0/991387160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc802f070 con 0x55f074db94a0 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7bc67fc640 1 -- 172.21.15.16:0/991387160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc800f420 con 0x55f074db94a0 2024-09-15T22:43:47.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7bc67fc640 1 -- 172.21.15.16:0/991387160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8004200 con 0x55f074db94a0 2024-09-15T22:43:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 -- 172.21.15.16:0/991387160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db94a0 msgr2=0x55f074ddbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db94a0 0x55f074ddbd00 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009250 tx=0x7f7bc8005fd0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 -- 172.21.15.16:0/991387160 shutdown_connections 2024-09-15T22:43:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074dc4680 0x55f074dbe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db9dc0 0x55f074ddc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 --2- 172.21.15.16:0/991387160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074db94a0 0x55f074ddbd00 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 -- 172.21.15.16:0/991387160 >> 172.21.15.16:0/991387160 conn(0x55f074db5a30 msgr2=0x55f074dc32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 -- 172.21.15.16:0/991387160 shutdown_connections 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.917+0000 7f7be5451740 1 -- 172.21.15.16:0/991387160 wait complete. 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 -- start start 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 Processor -- start 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dcac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dcb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1ad530 con 0x55f074dc4680 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1ae9a0 con 0x55f074db9dc0 2024-09-15T22:43:47.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc1adc00 con 0x55f074db94a0 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be5451740 10 client.0 _ll_register_callbacks cb 0x7f7bc8009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dcb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7bdbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dcb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45402/0 (socket says 172.21.15.16:45402) 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7bdbfff640 1 -- 172.21.15.16:0/3642591050 learned_addr learned my addr 172.21.15.16:0/3642591050 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.918+0000 7f7be0e14640 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dcac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bdbfff640 1 -- 172.21.15.16:0/3642591050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 msgr2=0x55f074dca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bdbfff640 1 -- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 msgr2=0x55f074dcac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:47.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dcac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bdbfff640 1 -- 172.21.15.16:0/3642591050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f074e423c0 con 0x55f074dc4680 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bdbfff640 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dcb160 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800f650 tx=0x7f7bc8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5109d41376fed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bbbfff640 1 -- 172.21.15.16:0/3642591050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc802f070 con 0x55f074dc4680 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bbbfff640 1 -- 172.21.15.16:0/3642591050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7bc8003a00 con 0x55f074dc4680 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc800f420 con 0x55f074dc4680 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bbbfff640 1 -- 172.21.15.16:0/3642591050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7bc800e840 con 0x55f074dc4680 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7be5451740 10 client.16488 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f074e43640 con 0x55f074dc4680 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.919+0000 7f7bbbfff640 1 -- 172.21.15.16:0/3642591050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc800f420 con 0x55f074dc4680 2024-09-15T22:43:47.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7bbbfff640 1 -- 172.21.15.16:0/3642591050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7bc801e040 con 0x55f074dc4680 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be5451740 10 client.16488 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be5451740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be5451740 20 client.16488 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be5451740 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074e43c50 0x55f074e46040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be5451740 4 client.16488 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f074d26d60 con 0x55f074e43c50 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be0e14640 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074e43c50 0x55f074e46040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be0e14640 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074e43c50 0x55f074e46040 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f886c847cf768de0 server_cookie=cb1a9ec3d4f3917b in_seq=0 out_seq=0 2024-09-15T22:43:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7bbbfff640 10 client.16488 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7bbbfff640 1 -- 172.21.15.16:0/3642591050 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f074d26d60 con 0x55f074e43c50 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7bbbfff640 10 client.16488 handle_command_reply: tid=0 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.920+0000 7f7be5451740 1 client.16488 shutdown 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 msgr2=0x55f074dcb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dcb160 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800f650 tx=0x7f7bc8005970 comp rx=0 tx=0).stop 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 shutdown_connections 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f074e43c50 0x55f074e46040 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7bbbfff640 0 client.16488 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f074dc4680 0x55f074dcb160 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f074db9dc0 0x55f074dcac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 --2- 172.21.15.16:0/3642591050 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f074db94a0 0x55f074dca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 >> 172.21.15.16:0/3642591050 conn(0x55f074db5a30 msgr2=0x55f074db4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 shutdown_connections 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 1 -- 172.21.15.16:0/3642591050 wait complete. 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.921+0000 7f7be5451740 20 client.16488 trim_cache size 0 max 16384 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:43:47.930 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:43:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7bcc0a33b0 msgr2=0x7f7bcc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7bcc0a33b0 0x7f7bcc0a5870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7bd409bde0 tx=0x7f7bd40bc040 comp rx=0 tx=0).stop 2024-09-15T22:43:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc10c170 msgr2=0x7f7bdc19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc10c170 0x7f7bdc19cdc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400cad0 tx=0x7f7bd400cf90 comp rx=0 tx=0).stop 2024-09-15T22:43:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 shutdown_connections 2024-09-15T22:43:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bdc10c170 0x7f7bdc19cdc0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7bcc0a33b0 0x7f7bcc0a5870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7bdc103660 0x7f7bdc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 --2- 172.21.15.16:0/40579973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7bdc102d40 0x7f7bdc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 >> 172.21.15.16:0/40579973 conn(0x7f7bdc0fc7a0 msgr2=0x7f7bdc0fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 shutdown_connections 2024-09-15T22:43:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:47.922+0000 7f7be3078640 1 -- 172.21.15.16:0/40579973 wait complete. 2024-09-15T22:43:48.732 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:43:48.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:48 smithi016 ceph-mon[31492]: pgmap v164: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:48.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:48 smithi059 ceph-mon[34872]: pgmap v164: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:49.000 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:49.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:48 smithi130 ceph-mon[35742]: pgmap v164: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:49.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 -- 172.21.15.16:0/2888514113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0101de0 msgr2=0x7fb4c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 --2- 172.21.15.16:0/2888514113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0101de0 0x7fb4c01021c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009900 tx=0x7fb4b002a410 comp rx=0 tx=0).stop 2024-09-15T22:43:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 -- 172.21.15.16:0/2888514113 shutdown_connections 2024-09-15T22:43:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 --2- 172.21.15.16:0/2888514113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c0106f10 0x7fb4c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 --2- 172.21.15.16:0/2888514113 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4c0102790 0x7fb4c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 --2- 172.21.15.16:0/2888514113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0101de0 0x7fb4c01021c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 -- 172.21.15.16:0/2888514113 >> 172.21.15.16:0/2888514113 conn(0x7fb4c00fd8c0 msgr2=0x7fb4c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 -- 172.21.15.16:0/2888514113 shutdown_connections 2024-09-15T22:43:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.458+0000 7fb4c5835640 1 -- 172.21.15.16:0/2888514113 wait complete. 2024-09-15T22:43:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 Processor -- start 2024-09-15T22:43:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 -- start start 2024-09-15T22:43:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4c0101de0 0x7fb4c01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:49.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0102790 0x7fb4c0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c0106f10 0x7fb4c019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0112060 con 0x7fb4c0102790 2024-09-15T22:43:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0111ee0 con 0x7fb4c0106f10 2024-09-15T22:43:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4c5835640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01121e0 con 0x7fb4c0101de0 2024-09-15T22:43:49.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0102790 0x7fb4c0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0102790 0x7fb4c0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45424/0 (socket says 172.21.15.16:45424) 2024-09-15T22:43:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 -- 172.21.15.16:0/3188226566 learned_addr learned my addr 172.21.15.16:0/3188226566 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4beffd640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4c0101de0 0x7fb4c01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4bf7fe640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c0106f10 0x7fb4c019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 -- 172.21.15.16:0/3188226566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4c0101de0 msgr2=0x7fb4c01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4c0101de0 0x7fb4c01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 -- 172.21.15.16:0/3188226566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c0106f10 msgr2=0x7fb4c019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c0106f10 0x7fb4c019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4b7fff640 1 -- 172.21.15.16:0/3188226566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c019d570 con 0x7fb4c0102790 2024-09-15T22:43:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.459+0000 7fb4bf7fe640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c0106f10 0x7fb4c019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.460+0000 7fb4b7fff640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0102790 0x7fb4c0198b00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b4a0 tx=0x7fb4a800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.460+0000 7fb4c4833640 1 -- 172.21.15.16:0/3188226566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8014070 con 0x7fb4c0102790 2024-09-15T22:43:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.460+0000 7fb4c4833640 1 -- 172.21.15.16:0/3188226566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a8010af0 con 0x7fb4c0102790 2024-09-15T22:43:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.460+0000 7fb4c4833640 1 -- 172.21.15.16:0/3188226566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8017d10 con 0x7fb4c0102790 2024-09-15T22:43:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.460+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c019d860 con 0x7fb4c0102790 2024-09-15T22:43:49.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.460+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c01a5140 con 0x7fb4c0102790 2024-09-15T22:43:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.461+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c0102240 con 0x7fb4c0102790 2024-09-15T22:43:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.462+0000 7fb4c4833640 1 -- 172.21.15.16:0/3188226566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb4a8017070 con 0x7fb4c0102790 2024-09-15T22:43:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.463+0000 7fb4c4833640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4ac0a33b0 0x7fb4ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.463+0000 7fb4beffd640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4ac0a33b0 0x7fb4ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.463+0000 7fb4c4833640 1 -- 172.21.15.16:0/3188226566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb4a80cf6d0 con 0x7fb4c0102790 2024-09-15T22:43:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.463+0000 7fb4beffd640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4ac0a33b0 0x7fb4ac0a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0000c00 tx=0x7fb4b0005df0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.466+0000 7fb4c4833640 1 -- 172.21.15.16:0/3188226566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4a809be60 con 0x7fb4c0102790 2024-09-15T22:43:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.678+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4c0199920 con 0x7fb4c0102790 2024-09-15T22:43:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.678+0000 7fb4c4833640 1 -- 172.21.15.16:0/3188226566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fb4c0199920 con 0x7fb4c0102790 2024-09-15T22:43:49.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:43:49.681 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:43:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:43:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4ac0a33b0 msgr2=0x7fb4ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4ac0a33b0 0x7fb4ac0a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0000c00 tx=0x7fb4b0005df0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0102790 msgr2=0x7fb4c0198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0102790 0x7fb4c0198b00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b4a0 tx=0x7fb4a800b960 comp rx=0 tx=0).stop 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 shutdown_connections 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c0106f10 0x7fb4c019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4c0102790 0x7fb4c0198b00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb4ac0a33b0 0x7fb4ac0a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 --2- 172.21.15.16:0/3188226566 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb4c0101de0 0x7fb4c01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 >> 172.21.15.16:0/3188226566 conn(0x7fb4c00fd8c0 msgr2=0x7fb4c00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 shutdown_connections 2024-09-15T22:43:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:49.681+0000 7fb4c5835640 1 -- 172.21.15.16:0/3188226566 wait complete. 2024-09-15T22:43:50.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:43:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:50.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:50 smithi059 ceph-mon[34872]: pgmap v165: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:50.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:50 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3188226566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:50.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:50 smithi016 ceph-mon[31492]: pgmap v165: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:50.970 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:50 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3188226566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:51.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:50 smithi130 ceph-mon[35742]: pgmap v165: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:51.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:50 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3188226566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 -- 172.21.15.16:0/1040373538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc10c170 msgr2=0x7fa2dc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/1040373538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc10c170 0x7fa2dc10e610 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009ca0 tx=0x7fa2d402a950 comp rx=0 tx=0).stop 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 -- 172.21.15.16:0/1040373538 shutdown_connections 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/1040373538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc10c170 0x7fa2dc10e610 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/1040373538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2dc103660 0x7fa2dc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/1040373538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2dc102d40 0x7fa2dc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 -- 172.21.15.16:0/1040373538 >> 172.21.15.16:0/1040373538 conn(0x7fa2dc0fc7a0 msgr2=0x7fa2dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:51.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 -- 172.21.15.16:0/1040373538 shutdown_connections 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.171+0000 7fa2e21e6640 1 -- 172.21.15.16:0/1040373538 wait complete. 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 Processor -- start 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 -- start start 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102d40 0x7fa2dc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2dc103660 0x7fa2dc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2dc10c170 0x7fa2dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc111f70 con 0x7fa2dc102d40 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc111df0 con 0x7fa2dc103660 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2e21e6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1120f0 con 0x7fa2dc10c170 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102d40 0x7fa2dc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102d40 0x7fa2dc19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45432/0 (socket says 172.21.15.16:45432) 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2db7fe640 1 -- 172.21.15.16:0/2302141343 learned_addr learned my addr 172.21.15.16:0/2302141343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:51.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.172+0000 7fa2daffd640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2dc103660 0x7fa2dc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2dc10c170 0x7fa2dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2db7fe640 1 -- 172.21.15.16:0/2302141343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2dc10c170 msgr2=0x7fa2dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2dc10c170 0x7fa2dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2db7fe640 1 -- 172.21.15.16:0/2302141343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2dc103660 msgr2=0x7fa2dc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2dc103660 0x7fa2dc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2db7fe640 1 -- 172.21.15.16:0/2302141343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc19f620 con 0x7fa2dc102d40 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2dc10c170 0x7fa2dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102d40 0x7fa2dc19a670 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00d770 tx=0x7fa2cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:51.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2d8ff9640 1 -- 172.21.15.16:0/2302141343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc017070 con 0x7fa2dc102d40 2024-09-15T22:43:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc19f910 con 0x7fa2dc102d40 2024-09-15T22:43:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2d8ff9640 1 -- 172.21.15.16:0/2302141343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2cc005500 con 0x7fa2dc102d40 2024-09-15T22:43:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2d8ff9640 1 -- 172.21.15.16:0/2302141343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc004e00 con 0x7fa2dc102d40 2024-09-15T22:43:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.173+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc1a71f0 con 0x7fa2dc102d40 2024-09-15T22:43:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2d8ff9640 1 -- 172.21.15.16:0/2302141343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa2cc004fa0 con 0x7fa2dc102d40 2024-09-15T22:43:51.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 Processor -- start 2024-09-15T22:43:51.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 -- start start 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f89156090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89156a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89163760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f890bc2f0 con 0x562f89161370 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f89063cf0 con 0x562f891565d0 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562f890b82b0 con 0x562f89157b40 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2d8ff9640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2c40a33b0 0x7fa2c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89163760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89163760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45444/0 (socket says 172.21.15.16:45444) 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2daffd640 1 -- 172.21.15.16:0/2533671145 learned_addr learned my addr 172.21.15.16:0/2533671145 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2c40a33b0 0x7fa2c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.176+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2d8ff9640 1 -- 172.21.15.16:0/2302141343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa2cc0cfab0 con 0x7fa2dc102d40 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f89156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2daffd640 1 -- 172.21.15.16:0/2533671145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 msgr2=0x562f89156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2daffd640 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f89156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2daffd640 1 -- 172.21.15.16:0/2533671145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 msgr2=0x562f89156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2daffd640 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2daffd640 1 -- 172.21.15.16:0/2533671145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f890483a0 con 0x562f89161370 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2daffd640 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89163760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009c00 tx=0x7fa2d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395ee85226521d7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2c40a33b0 0x7fa2c40a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005e30 tx=0x7fa2d4005d80 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2c1ffb640 1 -- 172.21.15.16:0/2533671145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0004230 con 0x562f89161370 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2c1ffb640 1 -- 172.21.15.16:0/2533671145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d00043d0 con 0x562f89161370 2024-09-15T22:43:51.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.177+0000 7fa2c1ffb640 1 -- 172.21.15.16:0/2533671145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0004db0 con 0x562f89161370 2024-09-15T22:43:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2533671145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 msgr2=0x562f89163760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89163760 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009c00 tx=0x7fa2d0011810 comp rx=0 tx=0).stop 2024-09-15T22:43:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2533671145 shutdown_connections 2024-09-15T22:43:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89163760 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2533671145 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f89156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2533671145 >> 172.21.15.16:0/2533671145 conn(0x562f8911f310 msgr2=0x562f89152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2533671145 shutdown_connections 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.178+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2533671145 wait complete. 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 -- start start 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 Processor -- start 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f891565d0 0x562f89160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f8915df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89161370 0x562f8915e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562f89164070 con 0x562f891565d0 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f89077060 con 0x562f89161370 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562f891641f0 con 0x562f89157b40 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2e45bf740 10 client.0 _ll_register_callbacks cb 0x562f88dca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:51.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f891565d0 0x562f89160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f891565d0 0x562f89160cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45450/0 (socket says 172.21.15.16:45450) 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.179+0000 7fa2db7fe640 1 -- 172.21.15.16:0/788219171 learned_addr learned my addr 172.21.15.16:0/788219171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89161370 0x562f8915e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2daffd640 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f8915df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2db7fe640 1 -- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 msgr2=0x562f8915df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f8915df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2db7fe640 1 -- 172.21.15.16:0/788219171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89161370 msgr2=0x562f8915e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89161370 0x562f8915e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2db7fe640 1 -- 172.21.15.16:0/788219171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f88e2d3c0 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f891565d0 0x562f89160cc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc098400 tx=0x7fa2cc018cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ebf37cf3cea774 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2baffd640 1 -- 172.21.15.16:0/788219171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc020040 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2baffd640 1 -- 172.21.15.16:0/788219171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa2cc0860b0 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d00043d0 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2baffd640 1 -- 172.21.15.16:0/788219171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc09c130 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2e45bf740 10 client.16506 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.180+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562f8916fad0 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2baffd640 1 -- 172.21.15.16:0/788219171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d00043d0 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2baffd640 1 -- 172.21.15.16:0/788219171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2cc0cd120 con 0x562f891565d0 2024-09-15T22:43:51.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2e45bf740 10 client.16506 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2e45bf740 10 client.16506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2e45bf740 20 client.16506 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f89170fe0 0x562f89173470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2e45bf740 4 client.16506 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f890c9860 con 0x562f89170fe0 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.181+0000 7fa2daffd640 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f89170fe0 0x562f89173470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.182+0000 7fa2daffd640 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f89170fe0 0x562f89173470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bed1dee28dbb678b server_cookie=d04a78ab514a0a38 in_seq=0 out_seq=0 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.182+0000 7fa2baffd640 10 client.16506 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.182+0000 7fa2baffd640 1 -- 172.21.15.16:0/788219171 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f890c9860 con 0x562f89170fe0 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.182+0000 7fa2baffd640 10 client.16506 handle_command_reply: tid=0 2024-09-15T22:43:51.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.182+0000 7fa2e45bf740 1 client.16506 shutdown 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f891565d0 msgr2=0x562f89160cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f891565d0 0x562f89160cc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc098400 tx=0x7fa2cc018cf0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 shutdown_connections 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f89170fe0 0x562f89173470 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89161370 0x562f8915e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f8915df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/788219171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f891565d0 0x562f89160cc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 >> 172.21.15.16:0/788219171 conn(0x562f8911f310 msgr2=0x562f89158960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:51.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2baffd640 0 client.16506 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:51.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 shutdown_connections 2024-09-15T22:43:51.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 1 -- 172.21.15.16:0/788219171 wait complete. 2024-09-15T22:43:51.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.183+0000 7fa2e45bf740 20 client.16506 trim_cache size 0 max 16384 2024-09-15T22:43:51.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 Processor -- start 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 -- start start 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f891565d0 0x562f8916d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89157b40 0x562f8915d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89162760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1b39d0 con 0x562f89161370 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1b12a0 con 0x562f89157b40 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1af0f0 con 0x562f891565d0 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89162760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89162760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45464/0 (socket says 172.21.15.16:45464) 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2dbfff640 1 -- 172.21.15.16:0/2426925091 learned_addr learned my addr 172.21.15.16:0/2426925091 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f891565d0 0x562f8916d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.196+0000 7fa2daffd640 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89157b40 0x562f8915d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2dbfff640 1 -- 172.21.15.16:0/2426925091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f891565d0 msgr2=0x562f8916d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f891565d0 0x562f8916d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2dbfff640 1 -- 172.21.15.16:0/2426925091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89157b40 msgr2=0x562f8915d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89157b40 0x562f8915d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2dbfff640 1 -- 172.21.15.16:0/2426925091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f8915c4c0 con 0x562f89161370 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89162760 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005fc0 tx=0x7fa2d403d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358529bb9cd9bce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2c17fa640 1 -- 172.21.15.16:0/2426925091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4034040 con 0x562f89161370 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2c17fa640 1 -- 172.21.15.16:0/2426925091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d4004040 con 0x562f89161370 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.197+0000 7fa2c17fa640 1 -- 172.21.15.16:0/2426925091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40306a0 con 0x562f89161370 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2426925091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 msgr2=0x562f89162760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89162760 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005fc0 tx=0x7fa2d403d040 comp rx=0 tx=0).stop 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2426925091 shutdown_connections 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89162760 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f89157b40 0x562f8915d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/2426925091 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f891565d0 0x562f8916d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2426925091 >> 172.21.15.16:0/2426925091 conn(0x562f8911f310 msgr2=0x562f89172ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:51.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2426925091 shutdown_connections 2024-09-15T22:43:51.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 -- 172.21.15.16:0/2426925091 wait complete. 2024-09-15T22:43:51.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 -- start start 2024-09-15T22:43:51.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.198+0000 7fa2e45bf740 1 Processor -- start 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89177e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f891783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2e45bf740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89168480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1af760 con 0x562f89161370 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1b0bd0 con 0x562f891565d0 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2e45bf740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc1afe30 con 0x562f89157b40 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2e45bf740 10 client.0 _ll_register_callbacks cb 0x7fa2cc00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:51.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89168480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89168480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45466/0 (socket says 172.21.15.16:45466) 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 -- 172.21.15.16:0/35662812 learned_addr learned my addr 172.21.15.16:0/35662812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2daffd640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f891783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89177e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 -- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 msgr2=0x562f891783d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f891783d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 -- 172.21.15.16:0/35662812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 msgr2=0x562f89177e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89177e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2dbfff640 1 -- 172.21.15.16:0/35662812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f891689c0 con 0x562f89161370 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.199+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89177e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2dbfff640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89168480 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4006010 tx=0x7fa2d4004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b61f1ca0913169 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:51.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2baffd640 1 -- 172.21.15.16:0/35662812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4003a30 con 0x562f89161370 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2baffd640 1 -- 172.21.15.16:0/35662812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fa2d4003bd0 con 0x562f89161370 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d4004040 con 0x562f89161370 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2baffd640 1 -- 172.21.15.16:0/35662812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2d4068050 con 0x562f89161370 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 10 client.16512 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562f89169cc0 con 0x562f89161370 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2baffd640 1 -- 172.21.15.16:0/35662812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d4004040 con 0x562f89161370 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2baffd640 1 -- 172.21.15.16:0/35662812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2d4063030 con 0x562f89161370 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 10 client.16512 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 10 client.16512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 20 client.16512 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f891dbe20 0x562f891de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:51.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 4 client.16512 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.200+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f890c3d70 con 0x562f891dbe20 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f891dbe20 0x562f891de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2db7fe640 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f891dbe20 0x562f891de210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71390704e91552d4 server_cookie=acd86c5ab6046025 in_seq=0 out_seq=0 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2baffd640 10 client.16512 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2baffd640 1 -- 172.21.15.16:0/35662812 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f890c3d70 con 0x562f891dbe20 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2baffd640 10 client.16512 handle_command_reply: tid=0 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 client.16512 shutdown 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 msgr2=0x562f89168480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89168480 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4006010 tx=0x7fa2d4004790 comp rx=0 tx=0).stop 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 shutdown_connections 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x562f891dbe20 0x562f891de210 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562f89161370 0x562f89168480 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562f89157b40 0x562f891783d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 --2- 172.21.15.16:0/35662812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f891565d0 0x562f89177e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 >> 172.21.15.16:0/35662812 conn(0x562f8911f310 msgr2=0x562f8915c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.201+0000 7fa2baffd640 0 client.16512 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 shutdown_connections 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e45bf740 1 -- 172.21.15.16:0/35662812 wait complete. 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e45bf740 20 client.16512 trim_cache size 0 max 16384 2024-09-15T22:43:51.209 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:43:51.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2c40a33b0 msgr2=0x7fa2c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2c40a33b0 0x7fa2c40a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4005e30 tx=0x7fa2d4005d80 comp rx=0 tx=0).stop 2024-09-15T22:43:51.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102d40 msgr2=0x7fa2dc19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:51.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102d40 0x7fa2dc19a670 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00d770 tx=0x7fa2cc00dc30 comp rx=0 tx=0).stop 2024-09-15T22:43:51.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 shutdown_connections 2024-09-15T22:43:51.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2c40a33b0 0x7fa2c40a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2dc10c170 0x7fa2dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2dc103660 0x7fa2dc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 --2- 172.21.15.16:0/2302141343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2dc102d40 0x7fa2dc19a670 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:51.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.202+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 >> 172.21.15.16:0/2302141343 conn(0x7fa2dc0fc7a0 msgr2=0x7fa2dc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:51.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.203+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 shutdown_connections 2024-09-15T22:43:51.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:51.203+0000 7fa2e21e6640 1 -- 172.21.15.16:0/2302141343 wait complete. 2024-09-15T22:43:51.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:43:52.198 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 -- 172.21.15.16:0/1210650336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda98074f40 msgr2=0x7fda9810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/1210650336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda98074f40 0x7fda9810de80 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fda88009900 tx=0x7fda8802a480 comp rx=0 tx=0).stop 2024-09-15T22:43:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 -- 172.21.15.16:0/1210650336 shutdown_connections 2024-09-15T22:43:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/1210650336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda9810e3c0 0x7fda98110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/1210650336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda98074f40 0x7fda9810de80 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/1210650336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda98074620 0x7fda98074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 -- 172.21.15.16:0/1210650336 >> 172.21.15.16:0/1210650336 conn(0x7fda980fd8c0 msgr2=0x7fda980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 -- 172.21.15.16:0/1210650336 shutdown_connections 2024-09-15T22:43:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.671+0000 7fda9fbe8640 1 -- 172.21.15.16:0/1210650336 wait complete. 2024-09-15T22:43:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 Processor -- start 2024-09-15T22:43:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 -- start start 2024-09-15T22:43:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda98074620 0x7fda9819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda98074f40 0x7fda9819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9810e3c0 0x7fda9819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda98114160 con 0x7fda9810e3c0 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda98113fe0 con 0x7fda98074f40 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9fbe8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fda981142e0 con 0x7fda98074620 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9e185640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9810e3c0 0x7fda9819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9e185640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9810e3c0 0x7fda9819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35796/0 (socket says 172.21.15.16:35796) 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9e185640 1 -- 172.21.15.16:0/951764572 learned_addr learned my addr 172.21.15.16:0/951764572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9d984640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda98074620 0x7fda9819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.672+0000 7fda9d183640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda98074f40 0x7fda9819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.673+0000 7fda9e185640 1 -- 172.21.15.16:0/951764572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda98074620 msgr2=0x7fda9819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.673+0000 7fda9e185640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda98074620 0x7fda9819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.673+0000 7fda9e185640 1 -- 172.21.15.16:0/951764572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda98074f40 msgr2=0x7fda9819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.673+0000 7fda9e185640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda98074f40 0x7fda9819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.673+0000 7fda9e185640 1 -- 172.21.15.16:0/951764572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9819f650 con 0x7fda9810e3c0 2024-09-15T22:43:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.673+0000 7fda9d183640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda98074f40 0x7fda9819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:52.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.673+0000 7fda9e185640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9810e3c0 0x7fda9819ef70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda9000eaa0 tx=0x7fda9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.676+0000 7fda8effd640 1 -- 172.21.15.16:0/951764572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9000cdb0 con 0x7fda9810e3c0 2024-09-15T22:43:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.676+0000 7fda8effd640 1 -- 172.21.15.16:0/951764572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda90005020 con 0x7fda9810e3c0 2024-09-15T22:43:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.676+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9819f940 con 0x7fda9810e3c0 2024-09-15T22:43:52.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.676+0000 7fda8effd640 1 -- 172.21.15.16:0/951764572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda900116b0 con 0x7fda9810e3c0 2024-09-15T22:43:52.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.676+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda981a7220 con 0x7fda9810e3c0 2024-09-15T22:43:52.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.679+0000 7fda8effd640 1 -- 172.21.15.16:0/951764572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fda90005560 con 0x7fda9810e3c0 2024-09-15T22:43:52.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.679+0000 7fdaa1fc1740 1 Processor -- start 2024-09-15T22:43:52.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.679+0000 7fdaa1fc1740 1 -- start start 2024-09-15T22:43:52.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b7a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b7aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b87760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564326ae02f0 con 0x564326b85370 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564326a87cf0 con 0x564326b7bb40 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564326adc2b0 con 0x564326b7a5d0 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b87760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b87760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35806/0 (socket says 172.21.15.16:35806) 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d183640 1 -- 172.21.15.16:0/242803750 learned_addr learned my addr 172.21.15.16:0/242803750 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda8effd640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fda840a3430 0x7fda840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d984640 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b7aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d183640 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b7a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9e185640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fda840a3430 0x7fda840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda8effd640 1 -- 172.21.15.16:0/951764572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fda900d0240 con 0x7fda9810e3c0 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d984640 1 -- 172.21.15.16:0/242803750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 msgr2=0x564326b7a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d984640 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b7a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d984640 1 -- 172.21.15.16:0/242803750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 msgr2=0x564326b87760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.680+0000 7fda9d984640 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b87760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.681+0000 7fda9d984640 1 -- 172.21.15.16:0/242803750 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564326a6c3a0 con 0x564326b7a5d0 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.681+0000 7fda9d183640 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b7a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.681+0000 7fda9e185640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fda840a3430 0x7fda840a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda900092f0 tx=0x7fda900514d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.681+0000 7fda9d984640 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b7aa10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda94004630 tx=0x7fda9400bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7dd864a21de36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.681+0000 7fda83fff640 1 -- 172.21.15.16:0/242803750 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94003bd0 con 0x564326b7a5d0 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.681+0000 7fda83fff640 1 -- 172.21.15.16:0/242803750 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda94015070 con 0x564326b7a5d0 2024-09-15T22:43:52.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.681+0000 7fda83fff640 1 -- 172.21.15.16:0/242803750 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94004d30 con 0x564326b7a5d0 2024-09-15T22:43:52.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/242803750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 msgr2=0x564326b7aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b7aa10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fda94004630 tx=0x7fda9400bb20 comp rx=0 tx=0).stop 2024-09-15T22:43:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/242803750 shutdown_connections 2024-09-15T22:43:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b87760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b7aa10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/242803750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b7a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/242803750 >> 172.21.15.16:0/242803750 conn(0x564326b43310 msgr2=0x564326b76c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:52.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/242803750 shutdown_connections 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/242803750 wait complete. 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 -- start start 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.682+0000 7fdaa1fc1740 1 Processor -- start 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7a5d0 0x564326b84da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b82050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564326b88070 con 0x564326b85370 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564326a9b060 con 0x564326b7a5d0 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564326b881f0 con 0x564326b7bb40 2024-09-15T22:43:52.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fdaa1fc1740 10 client.0 _ll_register_callbacks cb 0x5643267ee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b827e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35820/0 (socket says 172.21.15.16:35820) 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d183640 1 -- 172.21.15.16:0/1988662446 learned_addr learned my addr 172.21.15.16:0/1988662446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d984640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b82050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d183640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7a5d0 0x564326b84da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d984640 1 -- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7a5d0 msgr2=0x564326b84da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d984640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7a5d0 0x564326b84da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d984640 1 -- 172.21.15.16:0/1988662446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 msgr2=0x564326b827e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d984640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b827e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d984640 1 -- 172.21.15.16:0/1988662446 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643268513c0 con 0x564326b7bb40 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.683+0000 7fda9d183640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:52.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fda9d984640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b82050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fda940048f0 tx=0x7fda9400bfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe546aeb37214197 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fda817fa640 10 client.0 ms_handle_connect on v2:172.21.15.130:3300/0 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fda817fa640 1 -- 172.21.15.16:0/1988662446 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94028060 con 0x564326b7bb40 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fda817fa640 1 -- 172.21.15.16:0/1988662446 <== mon.1 v2:172.21.15.130:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fda94011440 con 0x564326b7bb40 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda94015070 con 0x564326b7bb40 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fda817fa640 1 -- 172.21.15.16:0/1988662446 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda9402f050 con 0x564326b7bb40 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 10 client.26044 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564326b93b50 con 0x564326b7bb40 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fda817fa640 1 -- 172.21.15.16:0/1988662446 <== mon.1 v2:172.21.15.130:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda94015070 con 0x564326b7bb40 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fda817fa640 1 -- 172.21.15.16:0/1988662446 <== mon.1 v2:172.21.15.130:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda94022070 con 0x564326b7bb40 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 10 client.26044 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 10 client.26044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 20 client.26044 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326b94f50 0x564326b97340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 4 client.26044 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.684+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564326aed860 con 0x564326b94f50 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.685+0000 7fda9d183640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326b94f50 0x564326b97340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.685+0000 7fda9d183640 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326b94f50 0x564326b97340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d98bd2f96f641842 server_cookie=d80bd0527f07dd72 in_seq=0 out_seq=0 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.685+0000 7fda817fa640 10 client.26044 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fda817fa640 1 -- 172.21.15.16:0/1988662446 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564326aed860 con 0x564326b94f50 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fda817fa640 10 client.26044 handle_command_reply: tid=0 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 client.26044 shutdown 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 msgr2=0x564326b82050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b82050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fda940048f0 tx=0x7fda9400bfa0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 shutdown_connections 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326b94f50 0x564326b97340 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326b827e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b82050 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fda817fa640 0 client.26044 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1988662446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7a5d0 0x564326b84da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 >> 172.21.15.16:0/1988662446 conn(0x564326b43310 msgr2=0x564326b7c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.686+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 shutdown_connections 2024-09-15T22:43:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.687+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1988662446 wait complete. 2024-09-15T22:43:52.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.687+0000 7fdaa1fc1740 20 client.26044 trim_cache size 0 max 16384 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 Processor -- start 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 -- start start 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b7a5d0 0x564326b7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b7d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b85370 0x564326b9c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b3ac0 con 0x564326b7a5d0 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b1390 con 0x564326b85370 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fda981af1e0 con 0x564326b7bb40 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b7a5d0 0x564326b7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b7a5d0 0x564326b7d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35830/0 (socket says 172.21.15.16:35830) 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 -- 172.21.15.16:0/3631265243 learned_addr learned my addr 172.21.15.16:0/3631265243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d984640 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b7d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 -- 172.21.15.16:0/3631265243 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 msgr2=0x564326b7d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b7d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 -- 172.21.15.16:0/3631265243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b85370 msgr2=0x564326b9c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b85370 0x564326b9c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 -- 172.21.15.16:0/3631265243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564326b917f0 con 0x564326b7a5d0 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.700+0000 7fda9d183640 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b7a5d0 0x564326b7d120 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fda8802a9a0 tx=0x7fda88005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be82cc5a80643244 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fda837fe640 1 -- 172.21.15.16:0/3631265243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88035040 con 0x564326b7a5d0 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fda837fe640 1 -- 172.21.15.16:0/3631265243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda880318f0 con 0x564326b7a5d0 2024-09-15T22:43:52.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fda837fe640 1 -- 172.21.15.16:0/3631265243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88003970 con 0x564326b7a5d0 2024-09-15T22:43:52.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/3631265243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b7a5d0 msgr2=0x564326b7d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b7a5d0 0x564326b7d120 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fda8802a9a0 tx=0x7fda88005d50 comp rx=0 tx=0).stop 2024-09-15T22:43:52.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/3631265243 shutdown_connections 2024-09-15T22:43:52.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b85370 0x564326b9c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7bb40 0x564326b7d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/3631265243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b7a5d0 0x564326b7d120 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/3631265243 >> 172.21.15.16:0/3631265243 conn(0x564326b43310 msgr2=0x564326b81140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/3631265243 shutdown_connections 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.701+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/3631265243 wait complete. 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 -- start start 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 Processor -- start 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b88bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b89100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326bf78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fda981af850 con 0x564326b85370 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fda981b0cc0 con 0x564326b7bb40 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fda981aff20 con 0x564326b7a5d0 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fdaa1fc1740 10 client.0 _ll_register_callbacks cb 0x7fda88009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:52.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326bf78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fda9d183640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326bf78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35840/0 (socket says 172.21.15.16:35840) 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fda9d183640 1 -- 172.21.15.16:0/1206459455 learned_addr learned my addr 172.21.15.16:0/1206459455 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fda9d984640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b89100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.702+0000 7fda9d183640 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b88bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda9d183640 1 -- 172.21.15.16:0/1206459455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 msgr2=0x564326b88bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda9d183640 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b88bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda9d183640 1 -- 172.21.15.16:0/1206459455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 msgr2=0x564326b89100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda9d183640 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b89100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda9d183640 1 -- 172.21.15.16:0/1206459455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564326bf7f20 con 0x564326b85370 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda9d183640 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326bf78e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fda88000c00 tx=0x7fda880045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddde2a2cd1d0ac14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda817fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda817fa640 1 -- 172.21.15.16:0/1206459455 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88035070 con 0x564326b85370 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda880318f0 con 0x564326b85370 2024-09-15T22:43:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda817fa640 1 -- 172.21.15.16:0/1206459455 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fda88005020 con 0x564326b85370 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda817fa640 1 -- 172.21.15.16:0/1206459455 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda88005220 con 0x564326b85370 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fdaa1fc1740 10 client.16530 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564326bff2b0 con 0x564326b85370 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.703+0000 7fda817fa640 1 -- 172.21.15.16:0/1206459455 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda880318f0 con 0x564326b85370 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fda817fa640 1 -- 172.21.15.16:0/1206459455 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda88054040 con 0x564326b85370 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fdaa1fc1740 10 client.16530 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fdaa1fc1740 10 client.16530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fdaa1fc1740 20 client.16530 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326bff920 0x564326c01d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fdaa1fc1740 4 client.16530 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564326ae7d70 con 0x564326bff920 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.704+0000 7fda9d984640 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326bff920 0x564326c01d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:52.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fda9d984640 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326bff920 0x564326c01d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a5adc563d04619a server_cookie=73c9c50cb82a5bc6 in_seq=0 out_seq=0 2024-09-15T22:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fda817fa640 10 client.16530 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fda817fa640 1 -- 172.21.15.16:0/1206459455 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564326ae7d70 con 0x564326bff920 2024-09-15T22:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fda817fa640 10 client.16530 handle_command_reply: tid=0 2024-09-15T22:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 client.16530 shutdown 2024-09-15T22:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 msgr2=0x564326bf78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326bf78e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fda88000c00 tx=0x7fda880045b0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 shutdown_connections 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x564326bff920 0x564326c01d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564326b85370 0x564326bf78e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564326b7bb40 0x564326b89100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 --2- 172.21.15.16:0/1206459455 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564326b7a5d0 0x564326b88bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fda817fa640 0 client.16530 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 >> 172.21.15.16:0/1206459455 conn(0x564326b43310 msgr2=0x564326b838b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 shutdown_connections 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 1 -- 172.21.15.16:0/1206459455 wait complete. 2024-09-15T22:43:52.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.705+0000 7fdaa1fc1740 20 client.16530 trim_cache size 0 max 16384 2024-09-15T22:43:52.713 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fda840a3430 msgr2=0x7fda840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fda840a3430 0x7fda840a58f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda900092f0 tx=0x7fda900514d0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9810e3c0 msgr2=0x7fda9819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9810e3c0 0x7fda9819ef70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fda9000eaa0 tx=0x7fda9000ef60 comp rx=0 tx=0).stop 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 shutdown_connections 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda9810e3c0 0x7fda9819ef70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda98074f40 0x7fda9819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fda840a3430 0x7fda840a58f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 --2- 172.21.15.16:0/951764572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fda98074620 0x7fda9819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 >> 172.21.15.16:0/951764572 conn(0x7fda980fd8c0 msgr2=0x7fda980fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 shutdown_connections 2024-09-15T22:43:52.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:52.706+0000 7fda9fbe8640 1 -- 172.21.15.16:0/951764572 wait complete. 2024-09-15T22:43:52.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:52 smithi059 ceph-mon[34872]: pgmap v166: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:53.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:52 smithi130 ceph-mon[35742]: pgmap v166: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:53.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:52 smithi016 ceph-mon[31492]: pgmap v166: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:53.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:43:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.387+0000 7f3c1fa27640 1 -- 172.21.15.16:0/1914470929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 msgr2=0x7f3c181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.387+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/1914470929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c181029b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009ca0 tx=0x7f3c0c02a950 comp rx=0 tx=0).stop 2024-09-15T22:43:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.387+0000 7f3c1fa27640 1 -- 172.21.15.16:0/1914470929 shutdown_connections 2024-09-15T22:43:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.387+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/1914470929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c18102ef0 0x7f3c1810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.387+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/1914470929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c181029b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.387+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/1914470929 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c18100e20 0x7f3c18101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.387+0000 7f3c1fa27640 1 -- 172.21.15.16:0/1914470929 >> 172.21.15.16:0/1914470929 conn(0x7f3c180fc7d0 msgr2=0x7f3c180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 -- 172.21.15.16:0/1914470929 shutdown_connections 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 -- 172.21.15.16:0/1914470929 wait complete. 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 Processor -- start 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 -- start start 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c18100e20 0x7f3c181985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c18198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c18102ef0 0x7f3c1819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18110fa0 con 0x7f3c18102570 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18110e20 con 0x7f3c18100e20 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.388+0000 7f3c1fa27640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3c18111120 con 0x7f3c18102ef0 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c18198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c18198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35846/0 (socket says 172.21.15.16:35846) 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/3353499889 learned_addr learned my addr 172.21.15.16:0/3353499889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c18100e20 0x7f3c181985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1dfc4640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c18102ef0 0x7f3c1819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/3353499889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c18102ef0 msgr2=0x7f3c1819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c18102ef0 0x7f3c1819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/3353499889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c18100e20 msgr2=0x7f3c181985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c18100e20 0x7f3c181985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/3353499889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1819d570 con 0x7f3c18102570 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c18198b00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c005cc0 tx=0x7f3c0c02f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c167fc640 1 -- 172.21.15.16:0/3353499889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c034040 con 0x7f3c18102570 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c167fc640 1 -- 172.21.15.16:0/3353499889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c0c030920 con 0x7f3c18102570 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1819d800 con 0x7f3c18102570 2024-09-15T22:43:54.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c167fc640 1 -- 172.21.15.16:0/3353499889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0047c0 con 0x7f3c18102570 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.389+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c181a5140 con 0x7f3c18102570 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c167fc640 1 -- 172.21.15.16:0/3353499889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3c0c037020 con 0x7f3c18102570 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 Processor -- start 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 -- start start 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a3822090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3822a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a3833b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609a37882f0 con 0x5609a3831780 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5609a372fcf0 con 0x5609a38225d0 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.392+0000 7f3c21e00740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5609a37842b0 con 0x5609a3823b40 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1cfc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a3833b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1dfc4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a3822090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1cfc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a3833b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35852/0 (socket says 172.21.15.16:35852) 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/1232542240 learned_addr learned my addr 172.21.15.16:0/1232542240 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c167fc640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3c000a3430 0x7f3c000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3822a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1dfc4640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3c000a3430 0x7f3c000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1d7c3640 1 -- 172.21.15.16:0/1232542240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 msgr2=0x5609a3822090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a3822090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1d7c3640 1 -- 172.21.15.16:0/1232542240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 msgr2=0x5609a3833b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c167fc640 1 -- 172.21.15.16:0/3353499889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f3c0c0f0f80 con 0x7f3c18102570 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a3833b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1d7c3640 1 -- 172.21.15.16:0/1232542240 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a37143a0 con 0x5609a38225d0 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a3833b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:54.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3822a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c080099a0 tx=0x7f3c08009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63d8d34050c87f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.393+0000 7f3c1dfc4640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3c000a3430 0x7f3c000a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c10005850 tx=0x7f3c10004730 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3bff7fe640 1 -- 172.21.15.16:0/1232542240 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08017070 con 0x5609a38225d0 2024-09-15T22:43:54.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3bff7fe640 1 -- 172.21.15.16:0/1232542240 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013470 con 0x5609a38225d0 2024-09-15T22:43:54.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3bff7fe640 1 -- 172.21.15.16:0/1232542240 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08012630 con 0x5609a38225d0 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 -- 172.21.15.16:0/1232542240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 msgr2=0x5609a3822a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3822a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c080099a0 tx=0x7f3c08009e60 comp rx=0 tx=0).stop 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 -- 172.21.15.16:0/1232542240 shutdown_connections 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a3833b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3822a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 --2- 172.21.15.16:0/1232542240 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a3822090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 -- 172.21.15.16:0/1232542240 >> 172.21.15.16:0/1232542240 conn(0x5609a37eb310 msgr2=0x5609a381ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 -- 172.21.15.16:0/1232542240 shutdown_connections 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.394+0000 7f3c21e00740 1 -- 172.21.15.16:0/1232542240 wait complete. 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 -- start start 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 Processor -- start 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3831100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a382e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a382e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5609a3834480 con 0x5609a3831780 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5609a3743060 con 0x5609a38225d0 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5609a3834600 con 0x5609a3823b40 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c21e00740 10 client.0 _ll_register_callbacks cb 0x5609a34968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c1dfc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a382e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c1d7c3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3831100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c1dfc4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a382e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35856/0 (socket says 172.21.15.16:35856) 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.395+0000 7f3c1dfc4640 1 -- 172.21.15.16:0/2135695335 learned_addr learned my addr 172.21.15.16:0/2135695335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a382e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1dfc4640 1 -- 172.21.15.16:0/2135695335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 msgr2=0x5609a382e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1dfc4640 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a382e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1dfc4640 1 -- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 msgr2=0x5609a3831100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1dfc4640 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3831100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1dfc4640 1 -- 172.21.15.16:0/2135695335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a34f93c0 con 0x5609a3831780 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3831100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c1dfc4640 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a382e930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3c10010950 tx=0x7f3c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9069651bb08a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3bfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3bfcff9640 1 -- 172.21.15.16:0/2135695335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10019070 con 0x5609a3831780 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3bfcff9640 1 -- 172.21.15.16:0/2135695335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f3c10011d70 con 0x5609a3831780 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c08013470 con 0x5609a3831780 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3bfcff9640 1 -- 172.21.15.16:0/2135695335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c10030050 con 0x5609a3831780 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c21e00740 10 client.16542 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.396+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609a383ff60 con 0x5609a3831780 2024-09-15T22:43:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3bfcff9640 1 -- 172.21.15.16:0/2135695335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013470 con 0x5609a3831780 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3bfcff9640 1 -- 172.21.15.16:0/2135695335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c10004be0 con 0x5609a3831780 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c21e00740 10 client.16542 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c21e00740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c21e00740 20 client.16542 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c21e00740 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a3841370 0x5609a38437e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c21e00740 4 client.16542 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a3795860 con 0x5609a3841370 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a3841370 0x5609a38437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a3841370 0x5609a38437e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b773fc9d6275dc server_cookie=87660dc7dc8f6898 in_seq=0 out_seq=0 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.397+0000 7f3bfcff9640 10 client.16542 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3bfcff9640 1 -- 172.21.15.16:0/2135695335 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609a3795860 con 0x5609a3841370 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3bfcff9640 10 client.16542 handle_command_reply: tid=0 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 client.16542 shutdown 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 msgr2=0x5609a382e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a382e930 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3c10010950 tx=0x7f3c100046c0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 shutdown_connections 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a3841370 0x5609a38437e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3831780 0x5609a382e930 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3823b40 0x5609a382e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3bfcff9640 0 client.16542 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 --2- 172.21.15.16:0/2135695335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a3831100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.398+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 >> 172.21.15.16:0/2135695335 conn(0x5609a37eb310 msgr2=0x5609a3824960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.399+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 shutdown_connections 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.399+0000 7f3c21e00740 1 -- 172.21.15.16:0/2135695335 wait complete. 2024-09-15T22:43:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.399+0000 7f3c21e00740 20 client.16542 trim_cache size 0 max 16384 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 Processor -- start 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 -- start start 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a38225d0 0x5609a383d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a3823b40 0x5609a381e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 0x5609a381ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181af510 con 0x5609a38225d0 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181acde0 con 0x5609a3823b40 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c21e00740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181aac30 con 0x5609a3831780 2024-09-15T22:43:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a38225d0 0x5609a383d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a38225d0 0x5609a383d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35864/0 (socket says 172.21.15.16:35864) 2024-09-15T22:43:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 -- 172.21.15.16:0/433563603 learned_addr learned my addr 172.21.15.16:0/433563603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 -- 172.21.15.16:0/433563603 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 msgr2=0x5609a381ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/433563603 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 0x5609a381ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 -- 172.21.15.16:0/433563603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a3823b40 msgr2=0x5609a381e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/433563603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a3823b40 0x5609a381e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 -- 172.21.15.16:0/433563603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a3825120 con 0x5609a38225d0 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.412+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/433563603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a38225d0 0x5609a383d720 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009970 tx=0x7f3c08009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6594711cd2cbf82d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3bfeffd640 1 -- 172.21.15.16:0/433563603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08017040 con 0x5609a38225d0 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3bfeffd640 1 -- 172.21.15.16:0/433563603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013be0 con 0x5609a38225d0 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3bfeffd640 1 -- 172.21.15.16:0/433563603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08026b40 con 0x5609a38225d0 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 -- 172.21.15.16:0/433563603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a38225d0 msgr2=0x5609a383d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 --2- 172.21.15.16:0/433563603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a38225d0 0x5609a383d720 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009970 tx=0x7f3c08009110 comp rx=0 tx=0).stop 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 -- 172.21.15.16:0/433563603 shutdown_connections 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 --2- 172.21.15.16:0/433563603 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 0x5609a381ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 --2- 172.21.15.16:0/433563603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a3823b40 0x5609a381e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 --2- 172.21.15.16:0/433563603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a38225d0 0x5609a383d720 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 -- 172.21.15.16:0/433563603 >> 172.21.15.16:0/433563603 conn(0x5609a37eb310 msgr2=0x5609a382e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 -- 172.21.15.16:0/433563603 shutdown_connections 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.413+0000 7f3c21e00740 1 -- 172.21.15.16:0/433563603 wait complete. 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 -- start start 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 Processor -- start 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a382b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3823b40 0x5609a382b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 0x5609a382bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ab2a0 con 0x5609a3823b40 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ac710 con 0x5609a38225d0 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3c181ab970 con 0x5609a3831780 2024-09-15T22:43:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c21e00740 10 client.0 _ll_register_callbacks cb 0x7f3c1000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c1cfc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3823b40 0x5609a382b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c1cfc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3823b40 0x5609a382b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35874/0 (socket says 172.21.15.16:35874) 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.414+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/769615114 learned_addr learned my addr 172.21.15.16:0/769615114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1dfc4640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 0x5609a382bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a382b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/769615114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 msgr2=0x5609a382bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 0x5609a382bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 msgr2=0x5609a382b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a382b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1cfc2640 1 -- 172.21.15.16:0/769615114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a38ab0d0 con 0x5609a3823b40 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a382b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c1cfc2640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3823b40 0x5609a382b830 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b99e0 tx=0x7f3c0c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf9e8cf425fd9db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3bfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3bfcff9640 1 -- 172.21.15.16:0/769615114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0bd710 con 0x5609a3823b40 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3bfcff9640 1 -- 172.21.15.16:0/769615114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f3c0c009800 con 0x5609a3823b40 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c08013be0 con 0x5609a3823b40 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3bfcff9640 1 -- 172.21.15.16:0/769615114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c0c0bd010 con 0x5609a3823b40 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c21e00740 10 client.16548 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609a38ac350 con 0x5609a3823b40 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.415+0000 7f3bfcff9640 1 -- 172.21.15.16:0/769615114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c08013be0 con 0x5609a3823b40 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3bfcff9640 1 -- 172.21.15.16:0/769615114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c0c04d030 con 0x5609a3823b40 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c21e00740 10 client.16548 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c21e00740 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c21e00740 20 client.16548 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c21e00740 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a38acad0 0x5609a38aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c21e00740 4 client.16548 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609a378fd70 con 0x5609a38acad0 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a38acad0 0x5609a38aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3c1d7c3640 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a38acad0 0x5609a38aeef0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60becf811b58a9c5 server_cookie=b931ea25134168d9 in_seq=0 out_seq=0 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.416+0000 7f3bfcff9640 10 client.16548 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3bfcff9640 1 -- 172.21.15.16:0/769615114 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609a378fd70 con 0x5609a38acad0 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3bfcff9640 10 client.16548 handle_command_reply: tid=0 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 client.16548 shutdown 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3823b40 msgr2=0x5609a382b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3823b40 0x5609a382b830 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b99e0 tx=0x7f3c0c02a950 comp rx=0 tx=0).stop 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 shutdown_connections 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5609a38acad0 0x5609a38aeef0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3bfcff9640 0 client.16548 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5609a3831780 0x5609a382bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5609a3823b40 0x5609a382b830 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 --2- 172.21.15.16:0/769615114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5609a38225d0 0x5609a382b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 >> 172.21.15.16:0/769615114 conn(0x5609a37eb310 msgr2=0x5609a381e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 shutdown_connections 2024-09-15T22:43:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 1 -- 172.21.15.16:0/769615114 wait complete. 2024-09-15T22:43:54.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.417+0000 7f3c21e00740 20 client.16548 trim_cache size 0 max 16384 2024-09-15T22:43:54.428 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3c000a3430 msgr2=0x7f3c000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3c000a3430 0x7f3c000a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3c10005850 tx=0x7f3c10004730 comp rx=0 tx=0).stop 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 msgr2=0x7f3c18198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c18198b00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c005cc0 tx=0x7f3c0c02f650 comp rx=0 tx=0).stop 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 shutdown_connections 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3c000a3430 0x7f3c000a58f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3c18102ef0 0x7f3c1819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c18102570 0x7f3c18198b00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 --2- 172.21.15.16:0/3353499889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c18100e20 0x7f3c181985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 >> 172.21.15.16:0/3353499889 conn(0x7f3c180fc7d0 msgr2=0x7f3c1806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 shutdown_connections 2024-09-15T22:43:54.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:54.418+0000 7f3c1fa27640 1 -- 172.21.15.16:0/3353499889 wait complete. 2024-09-15T22:43:54.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:54 smithi059 ceph-mon[34872]: pgmap v167: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:55.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:54 smithi130 ceph-mon[35742]: pgmap v167: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:55.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:54 smithi016 ceph-mon[31492]: pgmap v167: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:56.101 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:43:56.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:43:56.373 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:56.756 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:56 smithi016 ceph-mon[31492]: pgmap v168: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:56.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3879296426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38106f10 msgr2=0x7fcc3810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3879296426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38106f10 0x7fcc3810ea90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009ca0 tx=0x7fcc3002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3879296426 shutdown_connections 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3879296426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38106f10 0x7fcc3810ea90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3879296426 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc38102790 0x7fcc381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3879296426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc38101de0 0x7fcc381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3879296426 >> 172.21.15.16:0/3879296426 conn(0x7fcc380fd8c0 msgr2=0x7fcc380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3879296426 shutdown_connections 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3879296426 wait complete. 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 Processor -- start 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.842+0000 7fcc3e7ad640 1 -- start start 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc3e7ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38101de0 0x7fcc381985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc3e7ad640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc38102790 0x7fcc38198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc3e7ad640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc38106f10 0x7fcc3819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc3e7ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38112010 con 0x7fcc38101de0 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc3e7ad640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38111e90 con 0x7fcc38106f10 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc3e7ad640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38112190 con 0x7fcc38102790 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38101de0 0x7fcc381985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38101de0 0x7fcc381985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35896/0 (socket says 172.21.15.16:35896) 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 -- 172.21.15.16:0/3320250953 learned_addr learned my addr 172.21.15.16:0/3320250953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc377fe640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc38102790 0x7fcc38198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc38106f10 0x7fcc3819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 -- 172.21.15.16:0/3320250953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc38102790 msgr2=0x7fcc38198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc38102790 0x7fcc38198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 -- 172.21.15.16:0/3320250953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc38106f10 msgr2=0x7fcc3819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc38106f10 0x7fcc3819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 -- 172.21.15.16:0/3320250953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3819d5a0 con 0x7fcc38101de0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc377fe640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc38102790 0x7fcc38198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.843+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38101de0 0x7fcc381985f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800b4a0 tx=0x7fcc2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.844+0000 7fcc357fa640 1 -- 172.21.15.16:0/3320250953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28014070 con 0x7fcc38101de0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.844+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3819d890 con 0x7fcc38101de0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.844+0000 7fcc357fa640 1 -- 172.21.15.16:0/3320250953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc28010af0 con 0x7fcc38101de0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.844+0000 7fcc357fa640 1 -- 172.21.15.16:0/3320250953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28017d10 con 0x7fcc38101de0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.844+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc381a5170 con 0x7fcc38101de0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.846+0000 7fcc357fa640 1 -- 172.21.15.16:0/3320250953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcc28017070 con 0x7fcc38101de0 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.846+0000 7fcc40b86740 1 Processor -- start 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.846+0000 7fcc40b86740 1 -- start start 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a00b60 0x555eb5a00f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a0b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a0bd40 0x555eb5a0e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc40b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555eb5966280 con 0x555eb5a01480 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc40b86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eb590db10 con 0x555eb5a0bd40 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc40b86740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eb5962240 con 0x555eb5a00b60 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a0b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a0b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35900/0 (socket says 172.21.15.16:35900) 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 -- 172.21.15.16:0/1114855462 learned_addr learned my addr 172.21.15.16:0/1114855462 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc357fa640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcc200a33b0 0x7fcc200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc377fe640 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a0bd40 0x555eb5a0e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a00b60 0x555eb5a00f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcc200a33b0 0x7fcc200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc357fa640 1 -- 172.21.15.16:0/3320250953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fcc280cf3f0 con 0x7fcc38101de0 2024-09-15T22:43:56.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 -- 172.21.15.16:0/1114855462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a00b60 msgr2=0x555eb5a00f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a00b60 0x555eb5a00f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 -- 172.21.15.16:0/1114855462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a0bd40 msgr2=0x555eb5a0e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a0bd40 0x555eb5a0e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc37fff640 1 -- 172.21.15.16:0/1114855462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb591a1f0 con 0x555eb5a01480 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.847+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a00b60 0x555eb5a00f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc37fff640 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a0b800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcc280cc890 tx=0x7fcc280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdd7916cafe97a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc1e7fc640 1 -- 172.21.15.16:0/1114855462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2801e070 con 0x555eb5a01480 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcc200a33b0 0x7fcc200a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc30005fe0 tx=0x7fcc300095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc1e7fc640 1 -- 172.21.15.16:0/1114855462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc2809bb80 con 0x555eb5a01480 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc1e7fc640 1 -- 172.21.15.16:0/1114855462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28010ca0 con 0x555eb5a01480 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 -- 172.21.15.16:0/1114855462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 msgr2=0x555eb5a0b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a0b800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcc280cc890 tx=0x7fcc280046c0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 -- 172.21.15.16:0/1114855462 shutdown_connections 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a0bd40 0x555eb5a0e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a0b800 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 --2- 172.21.15.16:0/1114855462 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a00b60 0x555eb5a00f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 -- 172.21.15.16:0/1114855462 >> 172.21.15.16:0/1114855462 conn(0x555eb59fd0f0 msgr2=0x555eb59fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.848+0000 7fcc40b86740 1 -- 172.21.15.16:0/1114855462 shutdown_connections 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 -- 172.21.15.16:0/1114855462 wait complete. 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 -- start start 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 Processor -- start 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a00b60 0x555eb5a08c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a09190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a096d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555eb5a0e8b0 con 0x555eb5a01480 2024-09-15T22:43:56.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eb5920f10 con 0x555eb5a00b60 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555eb5a0ea30 con 0x555eb5a0bd40 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc40b86740 10 client.0 _ll_register_callbacks cb 0x555eb58e9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.849+0000 7fcc377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a09190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a09190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35914/0 (socket says 172.21.15.16:35914) 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 -- 172.21.15.16:0/2167561334 learned_addr learned my addr 172.21.15.16:0/2167561334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc37fff640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a00b60 0x555eb5a08c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 -- 172.21.15.16:0/2167561334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 msgr2=0x555eb5a096d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a096d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 -- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a00b60 msgr2=0x555eb5a08c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a00b60 0x555eb5a08c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 -- 172.21.15.16:0/2167561334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb56d76a0 con 0x555eb5a01480 2024-09-15T22:43:56.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc37fff640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a00b60 0x555eb5a08c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc377fe640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a09190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c010500 tx=0x7fcc2c0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c5cc672ec996f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc177fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc177fe640 1 -- 172.21.15.16:0/2167561334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c018070 con 0x555eb5a01480 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc177fe640 1 -- 172.21.15.16:0/2167561334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fcc2c014640 con 0x555eb5a01480 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc2809bb80 con 0x555eb5a01480 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc177fe640 1 -- 172.21.15.16:0/2167561334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc2c024050 con 0x555eb5a01480 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc40b86740 10 client.16560 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.850+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555eb5a1a540 con 0x555eb5a01480 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc177fe640 1 -- 172.21.15.16:0/2167561334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc2809bb80 con 0x555eb5a01480 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc177fe640 1 -- 172.21.15.16:0/2167561334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcc2c012a30 con 0x555eb5a01480 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc40b86740 10 client.16560 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc40b86740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc40b86740 20 client.16560 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:56.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc40b86740 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a1b950 0x555eb5a1ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc40b86740 4 client.16560 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb59737f0 con 0x555eb5a1b950 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.851+0000 7fcc37fff640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a1b950 0x555eb5a1ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.852+0000 7fcc37fff640 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a1b950 0x555eb5a1ddf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7360386f16b2dbad server_cookie=53e137cffeaaeec3 in_seq=0 out_seq=0 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.852+0000 7fcc177fe640 10 client.16560 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.852+0000 7fcc177fe640 1 -- 172.21.15.16:0/2167561334 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555eb59737f0 con 0x555eb5a1b950 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.852+0000 7fcc177fe640 10 client.16560 handle_command_reply: tid=0 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.852+0000 7fcc40b86740 1 client.16560 shutdown 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.852+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 msgr2=0x555eb5a09190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.852+0000 7fcc40b86740 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a09190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c010500 tx=0x7fcc2c0109c0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 shutdown_connections 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a1b950 0x555eb5a1ddf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a01480 0x555eb5a09190 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 --2- 172.21.15.16:0/2167561334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a00b60 0x555eb5a08c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc177fe640 0 client.16560 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 >> 172.21.15.16:0/2167561334 conn(0x555eb59fd0f0 msgr2=0x555eb59fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 shutdown_connections 2024-09-15T22:43:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 1 -- 172.21.15.16:0/2167561334 wait complete. 2024-09-15T22:43:56.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.853+0000 7fcc40b86740 20 client.16560 trim_cache size 0 max 16384 2024-09-15T22:43:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 Processor -- start 2024-09-15T22:43:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 -- start start 2024-09-15T22:43:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a02cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381b1970 con 0x555eb5a00b60 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381af240 con 0x555eb5a01480 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.870+0000 7fcc40b86740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381ad090 con 0x555eb5a0bd40 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a02cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a02cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35926/0 (socket says 172.21.15.16:35926) 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 -- 172.21.15.16:0/3529078820 learned_addr learned my addr 172.21.15.16:0/3529078820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc377fe640 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 -- 172.21.15.16:0/3529078820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 msgr2=0x555eb5a23c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a23c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 -- 172.21.15.16:0/3529078820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 msgr2=0x555eb5a236c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a236c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 -- 172.21.15.16:0/3529078820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb5a039d0 con 0x555eb5a00b60 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc377fe640 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a236c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a02cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcc2809b180 tx=0x7fcc28017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6928f9b85da34cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc1dffb640 1 -- 172.21.15.16:0/3529078820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2801e040 con 0x555eb5a00b60 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc1dffb640 1 -- 172.21.15.16:0/3529078820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc280cf150 con 0x555eb5a00b60 2024-09-15T22:43:56.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.871+0000 7fcc1dffb640 1 -- 172.21.15.16:0/3529078820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc280cca60 con 0x555eb5a00b60 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 -- 172.21.15.16:0/3529078820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 msgr2=0x555eb5a02cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a02cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fcc2809b180 tx=0x7fcc28017730 comp rx=0 tx=0).stop 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 -- 172.21.15.16:0/3529078820 shutdown_connections 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a236c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3529078820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a02cc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 -- 172.21.15.16:0/3529078820 >> 172.21.15.16:0/3529078820 conn(0x555eb59fd0f0 msgr2=0x555eb5a025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:56.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 -- 172.21.15.16:0/3529078820 shutdown_connections 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 -- 172.21.15.16:0/3529078820 wait complete. 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.872+0000 7fcc40b86740 1 -- start start 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 1 Processor -- start 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a0fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a0ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381ad700 con 0x555eb5a00b60 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381aeb70 con 0x555eb5a01480 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381addd0 con 0x555eb5a0bd40 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc40b86740 10 client.0 _ll_register_callbacks cb 0x7fcc2c007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a0fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a0fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35934/0 (socket says 172.21.15.16:35934) 2024-09-15T22:43:56.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc37fff640 1 -- 172.21.15.16:0/3373593103 learned_addr learned my addr 172.21.15.16:0/3373593103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc377fe640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a0ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc37fff640 1 -- 172.21.15.16:0/3373593103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 msgr2=0x555eb5a104d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a104d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc37fff640 1 -- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 msgr2=0x555eb5a0ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.873+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a0ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc37fff640 1 -- 172.21.15.16:0/3373593103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb5a856c0 con 0x555eb5a00b60 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc3cd4a640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc37fff640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a0fa50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcc280176e0 tx=0x7fcc28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b214f05db1101762 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc177fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc177fe640 1 -- 172.21.15.16:0/3373593103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2801e070 con 0x555eb5a00b60 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc177fe640 1 -- 172.21.15.16:0/3373593103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fcc280cca60 con 0x555eb5a00b60 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc280cf150 con 0x555eb5a00b60 2024-09-15T22:43:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc177fe640 1 -- 172.21.15.16:0/3373593103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc2809bb80 con 0x555eb5a00b60 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc40b86740 10 client.16566 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555eb5a86930 con 0x555eb5a00b60 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc177fe640 1 -- 172.21.15.16:0/3373593103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc280cf150 con 0x555eb5a00b60 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.874+0000 7fcc177fe640 1 -- 172.21.15.16:0/3373593103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcc2809a8d0 con 0x555eb5a00b60 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc40b86740 10 client.16566 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc40b86740 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc40b86740 20 client.16566 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a86bb0 0x555eb5a88fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc40b86740 4 client.16566 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555eb596dd00 con 0x555eb5a86bb0 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc377fe640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a86bb0 0x555eb5a88fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc377fe640 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a86bb0 0x555eb5a88fa0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80876c6773a05a27 server_cookie=51e8b01fbf64a3de in_seq=0 out_seq=0 2024-09-15T22:43:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.875+0000 7fcc177fe640 10 client.16566 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc177fe640 1 -- 172.21.15.16:0/3373593103 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555eb596dd00 con 0x555eb5a86bb0 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc177fe640 10 client.16566 handle_command_reply: tid=0 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 client.16566 shutdown 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 msgr2=0x555eb5a0fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a0fa50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcc280176e0 tx=0x7fcc28000bc0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 shutdown_connections 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555eb5a86bb0 0x555eb5a88fa0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555eb5a0bd40 0x555eb5a104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eb5a01480 0x555eb5a0ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 --2- 172.21.15.16:0/3373593103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555eb5a00b60 0x555eb5a0fa50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.876+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 >> 172.21.15.16:0/3373593103 conn(0x555eb59fd0f0 msgr2=0x555eb5a025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc177fe640 0 client.16566 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:56.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 shutdown_connections 2024-09-15T22:43:56.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc40b86740 1 -- 172.21.15.16:0/3373593103 wait complete. 2024-09-15T22:43:56.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc40b86740 20 client.16566 trim_cache size 0 max 16384 2024-09-15T22:43:56.883 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:43:56.883 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:43:56.883 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "dab170ba-5c44-4d54-a0a7-a531687c4ab0", 2024-09-15T22:43:56.883 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:43:56.883 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcc200a33b0 msgr2=0x7fcc200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcc200a33b0 0x7fcc200a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc30005fe0 tx=0x7fcc300095c0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38101de0 msgr2=0x7fcc381985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38101de0 0x7fcc381985f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800b4a0 tx=0x7fcc2800b960 comp rx=0 tx=0).stop 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 shutdown_connections 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc38106f10 0x7fcc3819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcc200a33b0 0x7fcc200a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcc38102790 0x7fcc38198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 --2- 172.21.15.16:0/3320250953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc38101de0 0x7fcc381985f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:56.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 >> 172.21.15.16:0/3320250953 conn(0x7fcc380fd8c0 msgr2=0x7fcc380fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:56.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 shutdown_connections 2024-09-15T22:43:56.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:56.877+0000 7fcc3e7ad640 1 -- 172.21.15.16:0/3320250953 wait complete. 2024-09-15T22:43:57.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:56 smithi130 ceph-mon[35742]: pgmap v168: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:57.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:56 smithi059 ceph-mon[34872]: pgmap v168: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:43:57.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:43:57.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:57 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:43:57.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:57 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:57.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:57 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:43:57.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:57 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:57.834 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:58.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:57 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:43:58.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:57 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:58.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:57 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:43:58.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:57 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:58.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:57 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:43:58.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:57 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:58.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:57 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:43:58.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:57 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 -- 172.21.15.16:0/206087550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef8102d30 msgr2=0x7f5ef8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 --2- 172.21.15.16:0/206087550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef8102d30 0x7f5ef8103110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009cd0 tx=0x7f5ee802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 -- 172.21.15.16:0/206087550 shutdown_connections 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 --2- 172.21.15.16:0/206087550 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ef81082a0 0x7f5ef810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 --2- 172.21.15.16:0/206087550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef81036e0 0x7f5ef8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 --2- 172.21.15.16:0/206087550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef8102d30 0x7f5ef8103110 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 -- 172.21.15.16:0/206087550 >> 172.21.15.16:0/206087550 conn(0x7f5ef80fc7d0 msgr2=0x7f5ef80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.464+0000 7f5eff0db640 1 -- 172.21.15.16:0/206087550 shutdown_connections 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 -- 172.21.15.16:0/206087550 wait complete. 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 Processor -- start 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 -- start start 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ef8102d30 0x7f5ef8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef81036e0 0x7f5ef8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef81082a0 0x7f5ef8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81120a0 con 0x7f5ef81082a0 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8111f20 con 0x7f5ef81036e0 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5eff0db640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef8112220 con 0x7f5ef8102d30 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.465+0000 7f5efd678640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef81082a0 0x7f5ef8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef81082a0 0x7f5ef8101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35940/0 (socket says 172.21.15.16:35940) 2024-09-15T22:43:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 -- 172.21.15.16:0/2805636869 learned_addr learned my addr 172.21.15.16:0/2805636869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef81036e0 0x7f5ef8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efce77640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ef8102d30 0x7f5ef8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 -- 172.21.15.16:0/2805636869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ef8102d30 msgr2=0x7f5ef8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ef8102d30 0x7f5ef8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 -- 172.21.15.16:0/2805636869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef81036e0 msgr2=0x7f5ef8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef81036e0 0x7f5ef8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 -- 172.21.15.16:0/2805636869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef81a69e0 con 0x7f5ef81082a0 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efce77640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ef8102d30 0x7f5ef8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5efd678640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef81082a0 0x7f5ef8101410 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c970 tx=0x7f5ef000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5ef5ffb640 1 -- 172.21.15.16:0/2805636869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0007bb0 con 0x7f5ef81082a0 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5ef5ffb640 1 -- 172.21.15.16:0/2805636869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef0007d50 con 0x7f5ef81082a0 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef81a6cd0 con 0x7f5ef81082a0 2024-09-15T22:43:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.466+0000 7f5ef5ffb640 1 -- 172.21.15.16:0/2805636869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0018800 con 0x7f5ef81082a0 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.467+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef81a71c0 con 0x7f5ef81082a0 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5ef5ffb640 1 -- 172.21.15.16:0/2805636869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5ef000fa50 con 0x7f5ef81082a0 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 Processor -- start 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 -- start start 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5ef5ffb640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ee00a33b0 0x7f5ee00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d78f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d79a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d79a680 0x555f9d79ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f9d6f52e0 con 0x555f9d78f4a0 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f9d69cce0 con 0x555f9d79a680 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5f014b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555f9d6f12a0 con 0x555f9d78fdc0 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ee00a33b0 0x7f5ee00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.469+0000 7f5efce77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d78f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d78f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35952/0 (socket says 172.21.15.16:35952) 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5ef5ffb640 1 -- 172.21.15.16:0/2805636869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5ef00d0060 con 0x7f5ef81082a0 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 -- 172.21.15.16:0/703802167 learned_addr learned my addr 172.21.15.16:0/703802167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efd678640 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d79a680 0x555f9d79ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d79a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 -- 172.21.15.16:0/703802167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 msgr2=0x555f9d79a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d79a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 -- 172.21.15.16:0/703802167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d79a680 msgr2=0x555f9d79ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d79a680 0x555f9d79ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 -- 172.21.15.16:0/703802167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f9d681440 con 0x555f9d78f4a0 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efd678640 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d79a680 0x555f9d79ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ee00a33b0 0x7f5ee00a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0058a0 tx=0x7f5eec0128d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5efce77640 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d78f880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8035000 tx=0x7f5ee8031560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cbb4bfde644fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5edeffd640 1 -- 172.21.15.16:0/703802167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8031ac0 con 0x555f9d78f4a0 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5edeffd640 1 -- 172.21.15.16:0/703802167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8036040 con 0x555f9d78f4a0 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.470+0000 7f5edeffd640 1 -- 172.21.15.16:0/703802167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8004790 con 0x555f9d78f4a0 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 -- 172.21.15.16:0/703802167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 msgr2=0x555f9d78f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d78f880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8035000 tx=0x7f5ee8031560 comp rx=0 tx=0).stop 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 -- 172.21.15.16:0/703802167 shutdown_connections 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d79a680 0x555f9d79ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d79a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 --2- 172.21.15.16:0/703802167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d78f880 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 -- 172.21.15.16:0/703802167 >> 172.21.15.16:0/703802167 conn(0x555f9d78ba30 msgr2=0x555f9d78be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 -- 172.21.15.16:0/703802167 shutdown_connections 2024-09-15T22:43:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 -- 172.21.15.16:0/703802167 wait complete. 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.471+0000 7f5f014b4740 1 -- start start 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 1 Processor -- start 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d799fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78fdc0 0x555f9d797250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d79a680 0x555f9d797790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555f9d79d250 con 0x555f9d78f4a0 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f9d6b0050 con 0x555f9d78fdc0 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x555f9d79d3d0 con 0x555f9d79a680 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5f014b4740 10 client.0 _ll_register_callbacks cb 0x555f9d6f0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d799fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d799fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35960/0 (socket says 172.21.15.16:35960) 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 -- 172.21.15.16:0/1291806682 learned_addr learned my addr 172.21.15.16:0/1291806682 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78fdc0 0x555f9d797250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efd678640 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d79a680 0x555f9d797790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 -- 172.21.15.16:0/1291806682 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d79a680 msgr2=0x555f9d797790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d79a680 0x555f9d797790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 -- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78fdc0 msgr2=0x555f9d797250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78fdc0 0x555f9d797250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.472+0000 7f5efce77640 1 -- 172.21.15.16:0/1291806682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f9d4663c0 con 0x555f9d78f4a0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5efce77640 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d799fa0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5ee802a920 tx=0x7f5ee8031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf83ded1d72cd7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5ed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5ed7fff640 1 -- 172.21.15.16:0/1291806682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee804e070 con 0x555f9d78f4a0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee8036040 con 0x555f9d78f4a0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5ed7fff640 1 -- 172.21.15.16:0/1291806682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5ee8003b80 con 0x555f9d78f4a0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5ed7fff640 1 -- 172.21.15.16:0/1291806682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ee805a050 con 0x555f9d78f4a0 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 10 client.16578 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:58.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555f9d7a8bd0 con 0x555f9d78f4a0 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5ed7fff640 1 -- 172.21.15.16:0/1291806682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8036040 con 0x555f9d78f4a0 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5ed7fff640 1 -- 172.21.15.16:0/1291806682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ee8030070 con 0x555f9d78f4a0 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 10 client.16578 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 20 client.16578 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d7a9f20 0x555f9d7ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 4 client.16578 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.473+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f9d702850 con 0x555f9d7a9f20 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.474+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d7a9f20 0x555f9d7ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.474+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d7a9f20 0x555f9d7ac310 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a9efda84dd70aa server_cookie=6e8bef6d8ff9f7ca in_seq=0 out_seq=0 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.474+0000 7f5ed7fff640 10 client.16578 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:58.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5ed7fff640 1 -- 172.21.15.16:0/1291806682 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f9d702850 con 0x555f9d7a9f20 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5ed7fff640 10 client.16578 handle_command_reply: tid=0 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 client.16578 shutdown 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 msgr2=0x555f9d799fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d799fa0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5ee802a920 tx=0x7f5ee8031b70 comp rx=0 tx=0).stop 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 shutdown_connections 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d7a9f20 0x555f9d7ac310 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d79a680 0x555f9d797790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78fdc0 0x555f9d797250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5ed7fff640 0 client.16578 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 --2- 172.21.15.16:0/1291806682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d799fa0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 >> 172.21.15.16:0/1291806682 conn(0x555f9d78ba30 msgr2=0x555f9d791710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 shutdown_connections 2024-09-15T22:43:58.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 1 -- 172.21.15.16:0/1291806682 wait complete. 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.475+0000 7f5f014b4740 20 client.16578 trim_cache size 0 max 16384 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 Processor -- start 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 -- start start 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d7af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d7af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d7944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81b3990 con 0x555f9d78f4a0 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81b1260 con 0x555f9d78ba30 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5f014b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81af0b0 con 0x555f9d78fdc0 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.492+0000 7f5ef7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d7af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d7af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35966/0 (socket says 172.21.15.16:35966) 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 -- 172.21.15.16:0/92593407 learned_addr learned my addr 172.21.15.16:0/92593407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5efce77640 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d7af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5efd678640 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d7944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 -- 172.21.15.16:0/92593407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 msgr2=0x555f9d7944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d7944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 -- 172.21.15.16:0/92593407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 msgr2=0x555f9d7af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d7af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 -- 172.21.15.16:0/92593407 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f9d78f2d0 con 0x555f9d78f4a0 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d7af880 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e6e0 tx=0x7f5eec01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dd485055f98bf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ede7fc640 1 -- 172.21.15.16:0/92593407 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec0207d0 con 0x555f9d78f4a0 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ede7fc640 1 -- 172.21.15.16:0/92593407 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eec02e070 con 0x555f9d78f4a0 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.493+0000 7f5ede7fc640 1 -- 172.21.15.16:0/92593407 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec028b40 con 0x555f9d78f4a0 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 -- 172.21.15.16:0/92593407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 msgr2=0x555f9d7af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d7af880 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e6e0 tx=0x7f5eec01d000 comp rx=0 tx=0).stop 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 -- 172.21.15.16:0/92593407 shutdown_connections 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78fdc0 0x555f9d7944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78f4a0 0x555f9d7af880 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 --2- 172.21.15.16:0/92593407 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d7af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 -- 172.21.15.16:0/92593407 >> 172.21.15.16:0/92593407 conn(0x555f9d7978f0 msgr2=0x555f9d795b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 -- 172.21.15.16:0/92593407 shutdown_connections 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 -- 172.21.15.16:0/92593407 wait complete. 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 -- start start 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.494+0000 7f5f014b4740 1 Processor -- start 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d8087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78f4a0 0x555f9d808d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5f014b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78fdc0 0x555f9d809270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5f014b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81af720 con 0x555f9d78fdc0 2024-09-15T22:43:58.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5f014b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81b0b90 con 0x555f9d78ba30 2024-09-15T22:43:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:43:58.597 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:43:58.597 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:43:58.597 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:43:58.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5f014b4740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef81afdf0 con 0x555f9d78f4a0 2024-09-15T22:43:58.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5f014b4740 10 client.0 _ll_register_callbacks cb 0x7f5ee8031430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:43:58.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78fdc0 0x555f9d809270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78fdc0 0x555f9d809270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35980/0 (socket says 172.21.15.16:35980) 2024-09-15T22:43:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 -- 172.21.15.16:0/2117707565 learned_addr learned my addr 172.21.15.16:0/2117707565 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5ef7fff640 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78f4a0 0x555f9d808d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efce77640 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d8087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 -- 172.21.15.16:0/2117707565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78f4a0 msgr2=0x555f9d808d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78f4a0 0x555f9d808d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 -- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 msgr2=0x555f9d8087f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d8087f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 -- 172.21.15.16:0/2117707565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f9d78c6c0 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5efd678640 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78fdc0 0x555f9d809270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00989b0 tx=0x7f5ef009ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a68a169ccdc2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.495+0000 7f5ed7fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2117707565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef009bfe0 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2117707565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f5ef0098650 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec02e070 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2117707565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ef00a0390 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 10 client.16584 fetch_fsmap learned FSMap version 26 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555f9d814c10 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2117707565 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eec02e070 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2117707565 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ef0020030 con 0x555f9d78fdc0 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 10 client.16584 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:43:58.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 20 client.16584 populate_metadata read hostname 'smithi016' 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d79a680 0x555f9d817420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 4 client.16584 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f9d6fcd60 con 0x555f9d79a680 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.496+0000 7f5efce77640 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d79a680 0x555f9d817420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5efce77640 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d79a680 0x555f9d817420 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1bf7d98bea31147 server_cookie=7a8dee1e8de23645 in_seq=0 out_seq=0 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5ed7fff640 10 client.16584 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5ed7fff640 1 -- 172.21.15.16:0/2117707565 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555f9d6fcd60 con 0x555f9d79a680 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5ed7fff640 10 client.16584 handle_command_reply: tid=0 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 client.16584 shutdown 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78fdc0 msgr2=0x555f9d809270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78fdc0 0x555f9d809270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00989b0 tx=0x7f5ef009ba70 comp rx=0 tx=0).stop 2024-09-15T22:43:58.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 shutdown_connections 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x555f9d79a680 0x555f9d817420 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555f9d78fdc0 0x555f9d809270 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5ed7fff640 0 client.16584 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x555f9d78f4a0 0x555f9d808d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 --2- 172.21.15.16:0/2117707565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f9d78ba30 0x555f9d8087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 >> 172.21.15.16:0/2117707565 conn(0x555f9d7978f0 msgr2=0x555f9d78ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 shutdown_connections 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 1 -- 172.21.15.16:0/2117707565 wait complete. 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.497+0000 7f5f014b4740 20 client.16584 trim_cache size 0 max 16384 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ee00a33b0 msgr2=0x7f5ee00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ee00a33b0 0x7f5ee00a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0058a0 tx=0x7f5eec0128d0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef81082a0 msgr2=0x7f5ef8101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef81082a0 0x7f5ef8101410 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c970 tx=0x7f5ef000ce30 comp rx=0 tx=0).stop 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 shutdown_connections 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ef81082a0 0x7f5ef8101410 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef81036e0 0x7f5ef8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ee00a33b0 0x7f5ee00a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 --2- 172.21.15.16:0/2805636869 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5ef8102d30 0x7f5ef8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 >> 172.21.15.16:0/2805636869 conn(0x7f5ef80fc7d0 msgr2=0x7f5ef8105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 shutdown_connections 2024-09-15T22:43:58.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:58.498+0000 7f5eff0db640 1 -- 172.21.15.16:0/2805636869 wait complete. 2024-09-15T22:43:58.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:58 smithi016 ceph-mon[31492]: pgmap v169: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:58.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:43:58 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:43:59.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:58 smithi130 ceph-mon[35742]: pgmap v169: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:59.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:43:58 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:43:59.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:58 smithi059 ceph-mon[34872]: pgmap v169: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:43:59.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:43:58 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:43:59.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:43:59.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.971+0000 7fe45c5a2640 1 -- 172.21.15.16:0/172079738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe454102d30 msgr2=0x7fe454103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.971+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/172079738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe454102d30 0x7fe454103110 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe450009930 tx=0x7fe45002a440 comp rx=0 tx=0).stop 2024-09-15T22:43:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 -- 172.21.15.16:0/172079738 shutdown_connections 2024-09-15T22:43:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/172079738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4541082a0 0x7fe45410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/172079738 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4541036e0 0x7fe454107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/172079738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe454102d30 0x7fe454103110 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 -- 172.21.15.16:0/172079738 >> 172.21.15.16:0/172079738 conn(0x7fe4540fc7d0 msgr2=0x7fe4540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 -- 172.21.15.16:0/172079738 shutdown_connections 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 -- 172.21.15.16:0/172079738 wait complete. 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 Processor -- start 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.972+0000 7fe45c5a2640 1 -- start start 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45c5a2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe454102d30 0x7fe4541a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45c5a2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4541036e0 0x7fe4541a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45c5a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4541082a0 0x7fe45419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45c5a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4541121b0 con 0x7fe4541082a0 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45c5a2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe454112030 con 0x7fe454102d30 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45c5a2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe454112330 con 0x7fe4541036e0 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4541082a0 0x7fe45419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4541082a0 0x7fe45419a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35998/0 (socket says 172.21.15.16:35998) 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 -- 172.21.15.16:0/2879979523 learned_addr learned my addr 172.21.15.16:0/2879979523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe459b3d640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4541036e0 0x7fe4541a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45a33e640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe454102d30 0x7fe4541a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 -- 172.21.15.16:0/2879979523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4541036e0 msgr2=0x7fe4541a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4541036e0 0x7fe4541a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 -- 172.21.15.16:0/2879979523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe454102d30 msgr2=0x7fe4541a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe454102d30 0x7fe4541a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 -- 172.21.15.16:0/2879979523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45419af20 con 0x7fe4541082a0 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe459b3d640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4541036e0 0x7fe4541a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45ab3f640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4541082a0 0x7fe45419a780 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b970 tx=0x7fe44c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe44b7fe640 1 -- 172.21.15.16:0/2879979523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c019070 con 0x7fe4541082a0 2024-09-15T22:43:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe44b7fe640 1 -- 172.21.15.16:0/2879979523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44c00cb40 con 0x7fe4541082a0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.973+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45419b210 con 0x7fe4541082a0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.974+0000 7fe44b7fe640 1 -- 172.21.15.16:0/2879979523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c007650 con 0x7fe4541082a0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.974+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4541a7460 con 0x7fe4541082a0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.975+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45406a850 con 0x7fe4541082a0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.977+0000 7fe44b7fe640 1 -- 172.21.15.16:0/2879979523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe44c01c070 con 0x7fe4541082a0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.977+0000 7fe44b7fe640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4400a3430 0x7fe4400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.978+0000 7fe45a33e640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4400a3430 0x7fe4400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.978+0000 7fe44b7fe640 1 -- 172.21.15.16:0/2879979523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe44c0cfa10 con 0x7fe4541082a0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.978+0000 7fe45a33e640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4400a3430 0x7fe4400a58f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe450006010 tx=0x7fe450000bc0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:43:59.981+0000 7fe44b7fe640 1 -- 172.21.15.16:0/2879979523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe44c09c1a0 con 0x7fe4541082a0 2024-09-15T22:44:00.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.191+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe454103190 con 0x7fe4541082a0 2024-09-15T22:44:00.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.192+0000 7fe44b7fe640 1 -- 172.21.15.16:0/2879979523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fe44c09baa0 con 0x7fe4541082a0 2024-09-15T22:44:00.193 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:44:00.194 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:44:00.194 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4400a3430 msgr2=0x7fe4400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4400a3430 0x7fe4400a58f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe450006010 tx=0x7fe450000bc0 comp rx=0 tx=0).stop 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4541082a0 msgr2=0x7fe45419a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4541082a0 0x7fe45419a780 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b970 tx=0x7fe44c00be30 comp rx=0 tx=0).stop 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 shutdown_connections 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4541082a0 0x7fe45419a780 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4400a3430 0x7fe4400a58f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4541036e0 0x7fe4541a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 --2- 172.21.15.16:0/2879979523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe454102d30 0x7fe4541a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.194+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 >> 172.21.15.16:0/2879979523 conn(0x7fe4540fc7d0 msgr2=0x7fe454100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:00.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.195+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 shutdown_connections 2024-09-15T22:44:00.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:00.195+0000 7fe45c5a2640 1 -- 172.21.15.16:0/2879979523 wait complete. 2024-09-15T22:44:00.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:44:01.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:00 smithi130 ceph-mon[35742]: pgmap v170: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:01.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:00 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2879979523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:01.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:00 smithi016 ceph-mon[31492]: pgmap v170: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:01.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:00 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2879979523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:01.108 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:01.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:00 smithi059 ceph-mon[34872]: pgmap v170: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:01.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:00 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2879979523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:01.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.608+0000 7f0b9a12a640 1 -- 172.21.15.16:0/2734721498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 msgr2=0x7f0b9410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.608+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/2734721498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b9410d950 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009cd0 tx=0x7f0b8c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 -- 172.21.15.16:0/2734721498 shutdown_connections 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/2734721498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b9410d950 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/2734721498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94100a80 0x7f0b94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/2734721498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b94106a80 0x7f0b94106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 -- 172.21.15.16:0/2734721498 >> 172.21.15.16:0/2734721498 conn(0x7f0b940fc7a0 msgr2=0x7f0b940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 -- 172.21.15.16:0/2734721498 shutdown_connections 2024-09-15T22:44:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 -- 172.21.15.16:0/2734721498 wait complete. 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 Processor -- start 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.609+0000 7f0b9a12a640 1 -- start start 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b9a12a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b94100a80 0x7f0b94199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b9a12a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b941997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b9a12a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94106a80 0x7f0b941a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b9a12a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94110e80 con 0x7f0b94101420 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b9a12a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94110d00 con 0x7f0b94106a80 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b9a12a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0b94111000 con 0x7f0b94100a80 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b941997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b941997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39238/0 (socket says 172.21.15.16:39238) 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 -- 172.21.15.16:0/3409243308 learned_addr learned my addr 172.21.15.16:0/3409243308 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b937fe640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b94100a80 0x7f0b94199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b93fff640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94106a80 0x7f0b941a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 -- 172.21.15.16:0/3409243308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b94100a80 msgr2=0x7f0b94199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b94100a80 0x7f0b94199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 -- 172.21.15.16:0/3409243308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94106a80 msgr2=0x7f0b941a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94106a80 0x7f0b941a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 -- 172.21.15.16:0/3409243308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b941a4850 con 0x7f0b94101420 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b93fff640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94106a80 0x7f0b941a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.610+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b941997d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00c890 tx=0x7f0b7c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.611+0000 7f0b90ff9640 1 -- 172.21.15.16:0/3409243308 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004220 con 0x7f0b94101420 2024-09-15T22:44:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.611+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b941a4a20 con 0x7f0b94101420 2024-09-15T22:44:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.611+0000 7f0b90ff9640 1 -- 172.21.15.16:0/3409243308 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c0043c0 con 0x7f0b94101420 2024-09-15T22:44:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.611+0000 7f0b90ff9640 1 -- 172.21.15.16:0/3409243308 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004a00 con 0x7f0b94101420 2024-09-15T22:44:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.611+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b941a4f10 con 0x7f0b94101420 2024-09-15T22:44:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b90ff9640 1 -- 172.21.15.16:0/3409243308 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0b7c0079b0 con 0x7f0b94101420 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 Processor -- start 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 -- start start 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6d7780 0x5566bc6d9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566bc62e2f0 con 0x5566bc6c9b40 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5566bc5d5cf0 con 0x5566bc6c85d0 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b9c503740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5566bc62a2b0 con 0x5566bc6d7780 2024-09-15T22:44:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.613+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39242/0 (socket says 172.21.15.16:39242) 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 -- 172.21.15.16:0/368577824 learned_addr learned my addr 172.21.15.16:0/368577824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b937fe640 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6d7780 0x5566bc6d9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 -- 172.21.15.16:0/368577824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6d7780 msgr2=0x5566bc6d9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6d7780 0x5566bc6d9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 -- 172.21.15.16:0/368577824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 msgr2=0x5566bc6c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 -- 172.21.15.16:0/368577824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bc5ba3a0 con 0x5566bc6c9b40 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b937fe640 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6d7780 0x5566bc6d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b93fff640 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6c8090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c031540 tx=0x7f0b8c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d3840e4bb958cc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b90ff9640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b800a33b0 0x7f0b800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b937fe640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b800a33b0 0x7f0b800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b89ffb640 1 -- 172.21.15.16:0/368577824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c031ac0 con 0x5566bc6c9b40 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b90ff9640 1 -- 172.21.15.16:0/3409243308 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0b7c0cfd80 con 0x7f0b94101420 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b89ffb640 1 -- 172.21.15.16:0/368577824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b8c036040 con 0x5566bc6c9b40 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.614+0000 7f0b89ffb640 1 -- 172.21.15.16:0/368577824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c004730 con 0x5566bc6c9b40 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 -- 172.21.15.16:0/368577824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 msgr2=0x5566bc6c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b937fe640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b800a33b0 0x7f0b800a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5566bc6c5ac0 tx=0x7f0b84005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6c8090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c031540 tx=0x7f0b8c009a20 comp rx=0 tx=0).stop 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 -- 172.21.15.16:0/368577824 shutdown_connections 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6d7780 0x5566bc6d9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 --2- 172.21.15.16:0/368577824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6c8090 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 -- 172.21.15.16:0/368577824 >> 172.21.15.16:0/368577824 conn(0x5566bc691310 msgr2=0x5566bc6c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 -- 172.21.15.16:0/368577824 shutdown_connections 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 -- 172.21.15.16:0/368577824 wait complete. 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.615+0000 7f0b9c503740 1 -- start start 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 1 Processor -- start 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5566bc6da420 con 0x5566bc6d7780 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5566bc5e9060 con 0x5566bc6c85d0 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5566bc6da5a0 con 0x5566bc6c9b40 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b9c503740 10 client.0 _ll_register_callbacks cb 0x5566bc33c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6d4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39258/0 (socket says 172.21.15.16:39258) 2024-09-15T22:44:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 -- 172.21.15.16:0/1983914808 learned_addr learned my addr 172.21.15.16:0/1983914808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b937fe640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 -- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 msgr2=0x5566bc6d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 -- 172.21.15.16:0/1983914808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 msgr2=0x5566bc6d71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6d71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b93fff640 1 -- 172.21.15.16:0/1983914808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bc39f3c0 con 0x5566bc6d7780 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.616+0000 7f0b937fe640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6d71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b93fff640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6d4c10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005fb0 tx=0x7f0b8c031bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e83b771390dab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b72ffd640 1 -- 172.21.15.16:0/1983914808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c003d80 con 0x5566bc6d7780 2024-09-15T22:44:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b72ffd640 1 -- 172.21.15.16:0/1983914808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0b8c003f20 con 0x5566bc6d7780 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b8c036040 con 0x5566bc6d7780 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b72ffd640 1 -- 172.21.15.16:0/1983914808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b8c0305a0 con 0x5566bc6d7780 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 10 client.16602 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566bc6e5ea0 con 0x5566bc6d7780 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b72ffd640 1 -- 172.21.15.16:0/1983914808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b8c036040 con 0x5566bc6d7780 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b72ffd640 1 -- 172.21.15.16:0/1983914808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b8c030070 con 0x5566bc6d7780 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 10 client.16602 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 10 client.16602 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 20 client.16602 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc6e71f0 0x5566bc6e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 4 client.16602 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.617+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566bc63b860 con 0x5566bc6e71f0 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.618+0000 7f0b937fe640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc6e71f0 0x5566bc6e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.618+0000 7f0b72ffd640 10 client.16602 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.618+0000 7f0b937fe640 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc6e71f0 0x5566bc6e95e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4f5efdb88d864f1 server_cookie=566b323255adb957 in_seq=0 out_seq=0 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.619+0000 7f0b72ffd640 1 -- 172.21.15.16:0/1983914808 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566bc63b860 con 0x5566bc6e71f0 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.619+0000 7f0b72ffd640 10 client.16602 handle_command_reply: tid=0 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.619+0000 7f0b9c503740 1 client.16602 shutdown 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.619+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 msgr2=0x5566bc6d4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.619+0000 7f0b9c503740 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6d4c10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c005fb0 tx=0x7f0b8c031bf0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.619+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 shutdown_connections 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.619+0000 7f0b9c503740 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc6e71f0 0x5566bc6e95e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b9c503740 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6d4c10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b9c503740 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b9c503740 1 --2- 172.21.15.16:0/1983914808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6d71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 >> 172.21.15.16:0/1983914808 conn(0x5566bc691310 msgr2=0x5566bc6ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b72ffd640 0 client.16602 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:01.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 shutdown_connections 2024-09-15T22:44:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b9c503740 1 -- 172.21.15.16:0/1983914808 wait complete. 2024-09-15T22:44:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.620+0000 7f0b9c503740 20 client.16602 trim_cache size 0 max 16384 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 Processor -- start 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 -- start start 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c85d0 0x5566bc6cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6d7780 0x5566bc6eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941af690 con 0x5566bc6c9b40 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941acf60 con 0x5566bc6d7780 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b9c503740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941aadb0 con 0x5566bc6c85d0 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b92ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.633+0000 7f0b92ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6d3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39262/0 (socket says 172.21.15.16:39262) 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b92ffd640 1 -- 172.21.15.16:0/2422067296 learned_addr learned my addr 172.21.15.16:0/2422067296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b93fff640 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6d7780 0x5566bc6eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b937fe640 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c85d0 0x5566bc6cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b92ffd640 1 -- 172.21.15.16:0/2422067296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c85d0 msgr2=0x5566bc6cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c85d0 0x5566bc6cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b92ffd640 1 -- 172.21.15.16:0/2422067296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6d7780 msgr2=0x5566bc6eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6d7780 0x5566bc6eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b92ffd640 1 -- 172.21.15.16:0/2422067296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bc6d29d0 con 0x5566bc6c9b40 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b937fe640 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c85d0 0x5566bc6cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6d3460 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c000c00 tx=0x7f0b7c09b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0aede450e4d57f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b897fa640 1 -- 172.21.15.16:0/2422067296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c01d040 con 0x5566bc6c9b40 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b897fa640 1 -- 172.21.15.16:0/2422067296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c09bd00 con 0x5566bc6c9b40 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.634+0000 7f0b897fa640 1 -- 172.21.15.16:0/2422067296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c007ca0 con 0x5566bc6c9b40 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 -- 172.21.15.16:0/2422067296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 msgr2=0x5566bc6d3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6d3460 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c000c00 tx=0x7f0b7c09b840 comp rx=0 tx=0).stop 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 -- 172.21.15.16:0/2422067296 shutdown_connections 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6d7780 0x5566bc6eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6c9b40 0x5566bc6d3460 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2422067296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c85d0 0x5566bc6cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 -- 172.21.15.16:0/2422067296 >> 172.21.15.16:0/2422067296 conn(0x5566bc691310 msgr2=0x5566bc6d2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 -- 172.21.15.16:0/2422067296 shutdown_connections 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 -- 172.21.15.16:0/2422067296 wait complete. 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 -- start start 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.635+0000 7f0b9c503740 1 Processor -- start 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b9c503740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b9c503740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941ab420 con 0x5566bc6d7780 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b9c503740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941ac890 con 0x5566bc6c85d0 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b9c503740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0b941abaf0 con 0x5566bc6c9b40 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b9c503740 10 client.0 _ll_register_callbacks cb 0x7f0b8c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6dfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39278/0 (socket says 172.21.15.16:39278) 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 -- 172.21.15.16:0/2230163887 learned_addr learned my addr 172.21.15.16:0/2230163887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b92ffd640 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b937fe640 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 -- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 msgr2=0x5566bc6df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6df830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 -- 172.21.15.16:0/2230163887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 msgr2=0x5566bc6cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 -- 172.21.15.16:0/2230163887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bc6e02b0 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b93fff640 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6dfd70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009430 tx=0x7f0b8c030ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e4e024cadd144c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.636+0000 7f0b72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b72ffd640 1 -- 172.21.15.16:0/2230163887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c009540 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b7c09bd00 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b72ffd640 1 -- 172.21.15.16:0/2230163887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f0b8c003b80 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b72ffd640 1 -- 172.21.15.16:0/2230163887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b8c003d80 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 10 client.16608 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566bc756340 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b72ffd640 1 -- 172.21.15.16:0/2230163887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c09bd00 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b72ffd640 1 -- 172.21.15.16:0/2230163887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b8c046c10 con 0x5566bc6d7780 2024-09-15T22:44:01.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 10 client.16608 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 10 client.16608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 20 client.16608 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc7567f0 0x5566bc758c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 4 client.16608 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.637+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566bc635d70 con 0x5566bc7567f0 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b937fe640 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc7567f0 0x5566bc758c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b937fe640 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc7567f0 0x5566bc758c10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9726d944926b41e3 server_cookie=db6cbc1611b111ec in_seq=0 out_seq=0 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b72ffd640 10 client.16608 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b72ffd640 1 -- 172.21.15.16:0/2230163887 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566bc635d70 con 0x5566bc7567f0 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b72ffd640 10 client.16608 handle_command_reply: tid=0 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 client.16608 shutdown 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 msgr2=0x5566bc6dfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6dfd70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009430 tx=0x7f0b8c030ed0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 shutdown_connections 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5566bc7567f0 0x5566bc758c10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5566bc6d7780 0x5566bc6dfd70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5566bc6c9b40 0x5566bc6df830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 --2- 172.21.15.16:0/2230163887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5566bc6c85d0 0x5566bc6cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 >> 172.21.15.16:0/2230163887 conn(0x5566bc691310 msgr2=0x5566bc6d5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.638+0000 7f0b72ffd640 0 client.16608 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 shutdown_connections 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9c503740 1 -- 172.21.15.16:0/2230163887 wait complete. 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9c503740 20 client.16608 trim_cache size 0 max 16384 2024-09-15T22:44:01.648 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b800a33b0 msgr2=0x7f0b800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b800a33b0 0x7f0b800a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5566bc6c5ac0 tx=0x7f0b84005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 msgr2=0x7f0b941997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b941997d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00c890 tx=0x7f0b7c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 shutdown_connections 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94106a80 0x7f0b941a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b94101420 0x7f0b941997d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b800a33b0 0x7f0b800a5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 --2- 172.21.15.16:0/3409243308 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b94100a80 0x7f0b94199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 >> 172.21.15.16:0/3409243308 conn(0x7f0b940fc7a0 msgr2=0x7f0b940fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.639+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 shutdown_connections 2024-09-15T22:44:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:01.640+0000 7f0b9a12a640 1 -- 172.21.15.16:0/3409243308 wait complete. 2024-09-15T22:44:02.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:44:02.580 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:02.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:02 smithi016 ceph-mon[31492]: pgmap v171: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:03.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:02 smithi130 ceph-mon[35742]: pgmap v171: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 -- 172.21.15.16:0/1922855546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 msgr2=0x7f6aec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 --2- 172.21.15.16:0/1922855546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec100ee0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009930 tx=0x7f6ae002a420 comp rx=0 tx=0).stop 2024-09-15T22:44:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 -- 172.21.15.16:0/1922855546 shutdown_connections 2024-09-15T22:44:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 --2- 172.21.15.16:0/1922855546 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6aec101420 0x7f6aec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 --2- 172.21.15.16:0/1922855546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec100ee0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 --2- 172.21.15.16:0/1922855546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aec106a80 0x7f6aec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 -- 172.21.15.16:0/1922855546 >> 172.21.15.16:0/1922855546 conn(0x7f6aec0fc7a0 msgr2=0x7f6aec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 -- 172.21.15.16:0/1922855546 shutdown_connections 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.055+0000 7f6af18b0640 1 -- 172.21.15.16:0/1922855546 wait complete. 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 Processor -- start 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 -- start start 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aec101420 0x7f6aec198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6aec106a80 0x7f6aec19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec111fa0 con 0x7f6aec100a80 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec111e20 con 0x7f6aec101420 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6af18b0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec112120 con 0x7f6aec106a80 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39296/0 (socket says 172.21.15.16:39296) 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6aeaffd640 1 -- 172.21.15.16:0/3846920658 learned_addr learned my addr 172.21.15.16:0/3846920658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.056+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aec101420 0x7f6aec198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6aec106a80 0x7f6aec19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aeaffd640 1 -- 172.21.15.16:0/3846920658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6aec106a80 msgr2=0x7f6aec19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6aec106a80 0x7f6aec19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aeaffd640 1 -- 172.21.15.16:0/3846920658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aec101420 msgr2=0x7f6aec198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aec101420 0x7f6aec198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aeaffd640 1 -- 172.21.15.16:0/3846920658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec19d3f0 con 0x7f6aec100a80 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aec101420 0x7f6aec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec198440 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00d950 tx=0x7f6adc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6af08ae640 1 -- 172.21.15.16:0/3846920658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc017070 con 0x7f6aec100a80 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6af08ae640 1 -- 172.21.15.16:0/3846920658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc009c30 con 0x7f6aec100a80 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec19d6e0 con 0x7f6aec100a80 2024-09-15T22:44:03.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6af08ae640 1 -- 172.21.15.16:0/3846920658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc00f6b0 con 0x7f6aec100a80 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.057+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec1a4fc0 con 0x7f6aec100a80 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af08ae640 1 -- 172.21.15.16:0/3846920658 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6adc0109d0 con 0x7f6aec100a80 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 Processor -- start 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 -- start start 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd717090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd7175d0 0x5650fd717a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd722370 0x5650fd724760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650fd67d2f0 con 0x5650fd7175d0 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650fd624cf0 con 0x5650fd722370 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af3c89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5650fd6792b0 con 0x5650fd718b40 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd7175d0 0x5650fd717a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.060+0000 7f6af08ae640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6ad40a33b0 0x7f6ad40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd7175d0 0x5650fd717a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39312/0 (socket says 172.21.15.16:39312) 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 -- 172.21.15.16:0/1455744649 learned_addr learned my addr 172.21.15.16:0/1455744649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd717090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd722370 0x5650fd724760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6ad40a33b0 0x7f6ad40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6af08ae640 1 -- 172.21.15.16:0/3846920658 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f6adc0cfd20 con 0x7f6aec100a80 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 -- 172.21.15.16:0/1455744649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 msgr2=0x5650fd717090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd717090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 -- 172.21.15.16:0/1455744649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd722370 msgr2=0x5650fd724760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd722370 0x5650fd724760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 -- 172.21.15.16:0/1455744649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650fd6093a0 con 0x5650fd7175d0 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd717090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd7175d0 0x5650fd717a10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6adc098610 tx=0x7f6adc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d377813d47b13dae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6ad1ffb640 1 -- 172.21.15.16:0/1455744649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc020070 con 0x5650fd7175d0 2024-09-15T22:44:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6ad40a33b0 0x7f6ad40a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005c40 tx=0x7f6ae4014040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6ad1ffb640 1 -- 172.21.15.16:0/1455744649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc09bc40 con 0x5650fd7175d0 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.061+0000 7f6ad1ffb640 1 -- 172.21.15.16:0/1455744649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0cd330 con 0x5650fd7175d0 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 -- 172.21.15.16:0/1455744649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd7175d0 msgr2=0x5650fd717a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd7175d0 0x5650fd717a10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6adc098610 tx=0x7f6adc00bf60 comp rx=0 tx=0).stop 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 -- 172.21.15.16:0/1455744649 shutdown_connections 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd722370 0x5650fd724760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd7175d0 0x5650fd717a10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1455744649 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd717090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 -- 172.21.15.16:0/1455744649 >> 172.21.15.16:0/1455744649 conn(0x5650fd6e0310 msgr2=0x5650fd713c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 -- 172.21.15.16:0/1455744649 shutdown_connections 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 -- 172.21.15.16:0/1455744649 wait complete. 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.062+0000 7f6af3c89740 1 -- start start 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 1 Processor -- start 2024-09-15T22:44:03.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd721d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd71f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5650fd725010 con 0x5650fd718b40 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5650fd638060 con 0x5650fd7175d0 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5650fd725190 con 0x5650fd722370 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6af3c89740 10 client.0 _ll_register_callbacks cb 0x5650fd38b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39314/0 (socket says 172.21.15.16:39314) 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 -- 172.21.15.16:0/2164037792 learned_addr learned my addr 172.21.15.16:0/2164037792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd71f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd721d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 -- 172.21.15.16:0/2164037792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 msgr2=0x5650fd71f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd71f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 -- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 msgr2=0x5650fd721d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd721d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aea7fc640 1 -- 172.21.15.16:0/2164037792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650fd3ee3c0 con 0x5650fd718b40 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.063+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd71f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71efc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009930 tx=0x7f6ae00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e7ae5aeb3456e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6acaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6acaffd640 1 -- 172.21.15.16:0/2164037792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0035050 con 0x5650fd718b40 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6acaffd640 1 -- 172.21.15.16:0/2164037792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f6ae0004250 con 0x5650fd718b40 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650fd68a860 con 0x5650fd718b40 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6acaffd640 1 -- 172.21.15.16:0/2164037792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ae0052050 con 0x5650fd718b40 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6af3c89740 10 client.16620 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650fd730a60 con 0x5650fd718b40 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.064+0000 7f6acaffd640 1 -- 172.21.15.16:0/2164037792 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5650fd68a860 con 0x5650fd718b40 2024-09-15T22:44:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6acaffd640 1 -- 172.21.15.16:0/2164037792 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6ae004d030 con 0x5650fd718b40 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6af3c89740 10 client.16620 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6af3c89740 10 client.16620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6af3c89740 20 client.16620 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6af3c89740 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd731fc0 0x5650fd734400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6af3c89740 4 client.16620 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650fd684d70 con 0x5650fd731fc0 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.065+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd731fc0 0x5650fd734400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.066+0000 7f6acaffd640 10 client.16620 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.066+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd731fc0 0x5650fd734400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a87a4cd748aef527 server_cookie=1bfbd04c7261b9f8 in_seq=0 out_seq=0 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.066+0000 7f6acaffd640 1 -- 172.21.15.16:0/2164037792 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650fd684d70 con 0x5650fd731fc0 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.066+0000 7f6acaffd640 10 client.16620 handle_command_reply: tid=0 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.066+0000 7f6af3c89740 1 client.16620 shutdown 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 msgr2=0x5650fd71efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71efc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009930 tx=0x7f6ae00317e0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 shutdown_connections 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd731fc0 0x5650fd734400 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd71f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71efc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 --2- 172.21.15.16:0/2164037792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd721d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 >> 172.21.15.16:0/2164037792 conn(0x5650fd6e0310 msgr2=0x5650fd719990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6acaffd640 0 client.16620 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 shutdown_connections 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 1 -- 172.21.15.16:0/2164037792 wait complete. 2024-09-15T22:44:03.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.067+0000 7f6af3c89740 20 client.16620 trim_cache size 0 max 16384 2024-09-15T22:44:03.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 Processor -- start 2024-09-15T22:44:03.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 -- start start 2024-09-15T22:44:03.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd739cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1af680 con 0x5650fd718b40 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1acf50 con 0x5650fd7175d0 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6af3c89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1aada0 con 0x5650fd722370 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39322/0 (socket says 172.21.15.16:39322) 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 -- 172.21.15.16:0/1144085595 learned_addr learned my addr 172.21.15.16:0/1144085595 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd739cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 -- 172.21.15.16:0/1144085595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 msgr2=0x5650fd739cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd739cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 -- 172.21.15.16:0/1144085595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 msgr2=0x5650fd71a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aea7fc640 1 -- 172.21.15.16:0/1144085595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650fd722080 con 0x5650fd718b40 2024-09-15T22:44:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.080+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd739cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.081+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71a6c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001eb0 tx=0x7f6ae0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f37914f75a1eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.081+0000 7f6ad17fa640 1 -- 172.21.15.16:0/1144085595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae005a070 con 0x5650fd718b40 2024-09-15T22:44:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.081+0000 7f6ad17fa640 1 -- 172.21.15.16:0/1144085595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0004250 con 0x5650fd718b40 2024-09-15T22:44:03.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.081+0000 7f6ad17fa640 1 -- 172.21.15.16:0/1144085595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0000e60 con 0x5650fd718b40 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.081+0000 7f6af3c89740 1 -- 172.21.15.16:0/1144085595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 msgr2=0x5650fd71a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.081+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71a6c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001eb0 tx=0x7f6ae0009670 comp rx=0 tx=0).stop 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- 172.21.15.16:0/1144085595 shutdown_connections 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd722370 0x5650fd739cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd718b40 0x5650fd71a6c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 --2- 172.21.15.16:0/1144085595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- 172.21.15.16:0/1144085595 >> 172.21.15.16:0/1144085595 conn(0x5650fd6e0310 msgr2=0x5650fd721040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- 172.21.15.16:0/1144085595 shutdown_connections 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- 172.21.15.16:0/1144085595 wait complete. 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- start start 2024-09-15T22:44:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 Processor -- start 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd71c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd722370 0x5650fd71cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ab410 con 0x5650fd722370 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1ac880 con 0x5650fd7175d0 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.082+0000 7f6af3c89740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec1abae0 con 0x5650fd718b40 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6af3c89740 10 client.0 _ll_register_callbacks cb 0x7f6ae0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd722370 0x5650fd71cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd722370 0x5650fd71cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39336/0 (socket says 172.21.15.16:39336) 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 -- 172.21.15.16:0/3877930502 learned_addr learned my addr 172.21.15.16:0/3877930502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd71c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 -- 172.21.15.16:0/3877930502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 msgr2=0x5650fd71c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd71c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 -- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 msgr2=0x5650fd71c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 -- 172.21.15.16:0/3877930502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650fd79bda0 con 0x5650fd722370 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aea7fc640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd71c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6aeb7fe640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd722370 0x5650fd71cba0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400db60 tx=0x7f6ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681dc01651e67b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6acaffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6acaffd640 1 -- 172.21.15.16:0/3877930502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400de60 con 0x5650fd722370 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6acaffd640 1 -- 172.21.15.16:0/3877930502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f6ae4011bc0 con 0x5650fd722370 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.083+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae0004250 con 0x5650fd722370 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6acaffd640 1 -- 172.21.15.16:0/3877930502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ae4034050 con 0x5650fd722370 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 10 client.16626 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650fd79d070 con 0x5650fd722370 2024-09-15T22:44:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6acaffd640 1 -- 172.21.15.16:0/3877930502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae0004250 con 0x5650fd722370 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6acaffd640 1 -- 172.21.15.16:0/3877930502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6ae400f420 con 0x5650fd722370 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 10 client.16626 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 10 client.16626 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 20 client.16626 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd79d870 0x5650fd79fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 4 client.16626 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.084+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650fd601170 con 0x5650fd79d870 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd79d870 0x5650fd79fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6aeaffd640 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd79d870 0x5650fd79fc90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa5297da9f98c936 server_cookie=4497514a4257aa58 in_seq=0 out_seq=0 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6acaffd640 10 client.16626 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6acaffd640 1 -- 172.21.15.16:0/3877930502 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650fd601170 con 0x5650fd79d870 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6acaffd640 10 client.16626 handle_command_reply: tid=0 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 client.16626 shutdown 2024-09-15T22:44:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd722370 msgr2=0x5650fd71cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd722370 0x5650fd71cba0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6ae400db60 tx=0x7f6ae4004a40 comp rx=0 tx=0).stop 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 shutdown_connections 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5650fd79d870 0x5650fd79fc90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5650fd722370 0x5650fd71cba0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5650fd718b40 0x5650fd71c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 --2- 172.21.15.16:0/3877930502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5650fd7175d0 0x5650fd71c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6acaffd640 0 client.16626 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 >> 172.21.15.16:0/3877930502 conn(0x5650fd6e0310 msgr2=0x5650fd7124e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 shutdown_connections 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.085+0000 7f6af3c89740 1 -- 172.21.15.16:0/3877930502 wait complete. 2024-09-15T22:44:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af3c89740 20 client.16626 trim_cache size 0 max 16384 2024-09-15T22:44:03.093 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6ad40a33b0 msgr2=0x7f6ad40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6ad40a33b0 0x7f6ad40a5870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4005c40 tx=0x7f6ae4014040 comp rx=0 tx=0).stop 2024-09-15T22:44:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 msgr2=0x7f6aec198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec198440 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00d950 tx=0x7f6adc00de10 comp rx=0 tx=0).stop 2024-09-15T22:44:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 shutdown_connections 2024-09-15T22:44:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6ad40a33b0 0x7f6ad40a5870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6aec106a80 0x7f6aec19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6aec101420 0x7f6aec198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 --2- 172.21.15.16:0/3846920658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6aec100a80 0x7f6aec198440 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 >> 172.21.15.16:0/3846920658 conn(0x7f6aec0fc7a0 msgr2=0x7f6aec0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 shutdown_connections 2024-09-15T22:44:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:03.086+0000 7f6af18b0640 1 -- 172.21.15.16:0/3846920658 wait complete. 2024-09-15T22:44:03.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:02 smithi059 ceph-mon[34872]: pgmap v171: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:03.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:44:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 -- 172.21.15.16:0/361160367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 msgr2=0x7f1304101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 --2- 172.21.15.16:0/361160367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f1304101200 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009900 tx=0x7f12f402a480 comp rx=0 tx=0).stop 2024-09-15T22:44:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 -- 172.21.15.16:0/361160367 shutdown_connections 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 --2- 172.21.15.16:0/361160367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1304102ef0 0x7f130410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 --2- 172.21.15.16:0/361160367 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1304102570 0x7f13041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 --2- 172.21.15.16:0/361160367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f1304101200 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 -- 172.21.15.16:0/361160367 >> 172.21.15.16:0/361160367 conn(0x7f13040fc7d0 msgr2=0x7f13040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 -- 172.21.15.16:0/361160367 shutdown_connections 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.567+0000 7f130b02d640 1 -- 172.21.15.16:0/361160367 wait complete. 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 Processor -- start 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 -- start start 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f13040fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1304102570 0x7f13040fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1304102ef0 0x7f13040fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1304110f80 con 0x7f1304100e20 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1304110e00 con 0x7f1304102570 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f130b02d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1304111100 con 0x7f1304102ef0 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f13040fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f13040fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39356/0 (socket says 172.21.15.16:39356) 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 -- 172.21.15.16:0/1536537710 learned_addr learned my addr 172.21.15.16:0/1536537710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1303fff640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1304102570 0x7f13040fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f13095ca640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1304102ef0 0x7f13040fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 -- 172.21.15.16:0/1536537710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1304102ef0 msgr2=0x7f13040fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1304102ef0 0x7f13040fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 -- 172.21.15.16:0/1536537710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1304102570 msgr2=0x7f13040fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1304102570 0x7f13040fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f1308dc9640 1 -- 172.21.15.16:0/1536537710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13041a69f0 con 0x7f1304100e20 2024-09-15T22:44:04.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.568+0000 7f13095ca640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1304102ef0 0x7f13040fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.569+0000 7f1308dc9640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f13040fcc80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f12f4006010 tx=0x7f12f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.569+0000 7f1301ffb640 1 -- 172.21.15.16:0/1536537710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f403f070 con 0x7f1304100e20 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.569+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13041a6c80 con 0x7f1304100e20 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.569+0000 7f1301ffb640 1 -- 172.21.15.16:0/1536537710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f4031c30 con 0x7f1304100e20 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.569+0000 7f1301ffb640 1 -- 172.21.15.16:0/1536537710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4035030 con 0x7f1304100e20 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.569+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13041a71b0 con 0x7f1304100e20 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.571+0000 7f1301ffb640 1 -- 172.21.15.16:0/1536537710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f12f4042070 con 0x7f1304100e20 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 Processor -- start 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 -- start start 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b77a72f2f0 con 0x55b77a7d4370 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b77a6d6cf0 con 0x55b77a7cab40 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f130d406740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b77a72b2b0 con 0x55b77a7c95d0 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f1301ffb640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f12ec0a33b0 0x7f12ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f1303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f1303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39364/0 (socket says 172.21.15.16:39364) 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f1303fff640 1 -- 172.21.15.16:0/2608744688 learned_addr learned my addr 172.21.15.16:0/2608744688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f13095ca640 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.572+0000 7f13095ca640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f12ec0a33b0 0x7f12ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1301ffb640 1 -- 172.21.15.16:0/1536537710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f12f40f0990 con 0x7f1304100e20 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1308dc9640 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1303fff640 1 -- 172.21.15.16:0/2608744688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 msgr2=0x55b77a7c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1303fff640 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1303fff640 1 -- 172.21.15.16:0/2608744688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 msgr2=0x55b77a7c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1303fff640 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1303fff640 1 -- 172.21.15.16:0/2608744688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77a6bb3a0 con 0x55b77a7d4370 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f1303fff640 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d6760 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f12f800bc90 tx=0x7f12f800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a362d2164c14b124 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f13095ca640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f12ec0a33b0 0x7f12ec0a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005c40 tx=0x7f12fc009200 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f12eaffd640 1 -- 172.21.15.16:0/2608744688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8015070 con 0x55b77a7d4370 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f12eaffd640 1 -- 172.21.15.16:0/2608744688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f801a940 con 0x55b77a7d4370 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.573+0000 7f12eaffd640 1 -- 172.21.15.16:0/2608744688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8011910 con 0x55b77a7d4370 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 -- 172.21.15.16:0/2608744688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 msgr2=0x55b77a7d6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d6760 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f12f800bc90 tx=0x7f12f800f730 comp rx=0 tx=0).stop 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 -- 172.21.15.16:0/2608744688 shutdown_connections 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d6760 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 --2- 172.21.15.16:0/2608744688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 -- 172.21.15.16:0/2608744688 >> 172.21.15.16:0/2608744688 conn(0x55b77a792310 msgr2=0x55b77a7c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 -- 172.21.15.16:0/2608744688 shutdown_connections 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 -- 172.21.15.16:0/2608744688 wait complete. 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 -- start start 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.574+0000 7f130d406740 1 Processor -- start 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f130d406740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7c95d0 0x55b77a7d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f130d406740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7cab40 0x55b77a7d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f130d406740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f130d406740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b77a7d7070 con 0x55b77a7cab40 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f130d406740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b77a6ea060 con 0x55b77a7c95d0 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f130d406740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b77a7d71f0 con 0x55b77a7d4370 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f130d406740 10 client.0 _ll_register_callbacks cb 0x55b77a43d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7cab40 0x55b77a7d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7cab40 0x55b77a7d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39380/0 (socket says 172.21.15.16:39380) 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 -- 172.21.15.16:0/4128208584 learned_addr learned my addr 172.21.15.16:0/4128208584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1308dc9640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7c95d0 0x55b77a7d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f13095ca640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 -- 172.21.15.16:0/4128208584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 msgr2=0x55b77a7d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 -- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7c95d0 msgr2=0x55b77a7d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7c95d0 0x55b77a7d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 -- 172.21.15.16:0/4128208584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77a4a03c0 con 0x55b77a7cab40 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f13095ca640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f1303fff640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7cab40 0x55b77a7d0fe0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006010 tx=0x7f12f800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7480a62802383f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.575+0000 7f12e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f12e3fff640 1 -- 172.21.15.16:0/4128208584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8028060 con 0x55b77a7cab40 2024-09-15T22:44:04.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f12e3fff640 1 -- 172.21.15.16:0/4128208584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f12f801a460 con 0x55b77a7cab40 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f801a940 con 0x55b77a7cab40 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f12e3fff640 1 -- 172.21.15.16:0/4128208584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12f802f050 con 0x55b77a7cab40 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 10 client.16638 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b77a7e2a40 con 0x55b77a7cab40 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f12e3fff640 1 -- 172.21.15.16:0/4128208584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f801a940 con 0x55b77a7cab40 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f12e3fff640 1 -- 172.21.15.16:0/4128208584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f12f8011830 con 0x55b77a7cab40 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 10 client.16638 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 20 client.16638 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a7e3e40 0x55b77a7e6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 4 client.16638 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.576+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b77a73c860 con 0x55b77a7e3e40 2024-09-15T22:44:04.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.577+0000 7f1308dc9640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a7e3e40 0x55b77a7e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.577+0000 7f1308dc9640 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a7e3e40 0x55b77a7e6230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88a29cae3530abfe server_cookie=4ae8844a22e96ed3 in_seq=0 out_seq=0 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.577+0000 7f12e3fff640 10 client.16638 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f12e3fff640 1 -- 172.21.15.16:0/4128208584 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b77a73c860 con 0x55b77a7e3e40 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f12e3fff640 10 client.16638 handle_command_reply: tid=0 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 client.16638 shutdown 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7cab40 msgr2=0x55b77a7d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7cab40 0x55b77a7d0fe0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006010 tx=0x7f12f800b9d0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 shutdown_connections 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a7e3e40 0x55b77a7e6230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7cab40 0x55b77a7d0fe0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 --2- 172.21.15.16:0/4128208584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7c95d0 0x55b77a7d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f12e3fff640 0 client.16638 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 >> 172.21.15.16:0/4128208584 conn(0x55b77a792310 msgr2=0x55b77a7cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 shutdown_connections 2024-09-15T22:44:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 1 -- 172.21.15.16:0/4128208584 wait complete. 2024-09-15T22:44:04.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.578+0000 7f130d406740 20 client.16638 trim_cache size 0 max 16384 2024-09-15T22:44:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 Processor -- start 2024-09-15T22:44:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 -- start start 2024-09-15T22:44:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7c95d0 0x55b77a7d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7d5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b39e0 con 0x55b77a7c95d0 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b12b0 con 0x55b77a7cab40 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f130d406740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13041af100 con 0x55b77a7d4370 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.591+0000 7f1308dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7c95d0 0x55b77a7d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7c95d0 0x55b77a7d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39392/0 (socket says 172.21.15.16:39392) 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 -- 172.21.15.16:0/2484164668 learned_addr learned my addr 172.21.15.16:0/2484164668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f13095ca640 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 -- 172.21.15.16:0/2484164668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 msgr2=0x55b77a7d5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1303fff640 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 -- 172.21.15.16:0/2484164668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 msgr2=0x55b77a7d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7d5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 -- 172.21.15.16:0/2484164668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77a7ca400 con 0x55b77a7c95d0 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f1308dc9640 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7c95d0 0x55b77a7d2f50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f12f40b8d70 tx=0x7f12f40a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8d7519c3d19d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f12ea7fc640 1 -- 172.21.15.16:0/2484164668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4049040 con 0x55b77a7c95d0 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f12ea7fc640 1 -- 172.21.15.16:0/2484164668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f40bc910 con 0x55b77a7c95d0 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.592+0000 7f12ea7fc640 1 -- 172.21.15.16:0/2484164668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f40bcb70 con 0x55b77a7c95d0 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 -- 172.21.15.16:0/2484164668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7c95d0 msgr2=0x55b77a7d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7c95d0 0x55b77a7d2f50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f12f40b8d70 tx=0x7f12f40a7040 comp rx=0 tx=0).stop 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 -- 172.21.15.16:0/2484164668 shutdown_connections 2024-09-15T22:44:04.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7d4370 0x55b77a7d5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 --2- 172.21.15.16:0/2484164668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7c95d0 0x55b77a7d2f50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 -- 172.21.15.16:0/2484164668 >> 172.21.15.16:0/2484164668 conn(0x55b77a792310 msgr2=0x55b77a7d2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 -- 172.21.15.16:0/2484164668 shutdown_connections 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 -- 172.21.15.16:0/2484164668 wait complete. 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 -- start start 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.593+0000 7f130d406740 1 Processor -- start 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f130d406740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f130d406740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f130d406740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f130d406740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f13041af770 con 0x55b77a7d4370 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f130d406740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13041b0be0 con 0x55b77a7cab40 2024-09-15T22:44:04.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f130d406740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f13041afe40 con 0x55b77a7c95d0 2024-09-15T22:44:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f130d406740 10 client.0 _ll_register_callbacks cb 0x7f12fc009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39398/0 (socket says 172.21.15.16:39398) 2024-09-15T22:44:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 -- 172.21.15.16:0/3851736396 learned_addr learned my addr 172.21.15.16:0/3851736396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f1308dc9640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f1303fff640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 -- 172.21.15.16:0/3851736396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 msgr2=0x55b77a7dc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7dc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 -- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 msgr2=0x55b77a7dc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7dc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f13095ca640 1 -- 172.21.15.16:0/3851736396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77a7dd0c0 con 0x55b77a7d4370 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.594+0000 7f1303fff640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7dc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f13095ca640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d7440 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0142e0 tx=0x7f12fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe9433baed7f149 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f12e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f12e3fff640 1 -- 172.21.15.16:0/3851736396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc027070 con 0x55b77a7d4370 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f12e3fff640 1 -- 172.21.15.16:0/3851736396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f12fc004d20 con 0x55b77a7d4370 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f40bc910 con 0x55b77a7d4370 2024-09-15T22:44:04.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f12e3fff640 1 -- 172.21.15.16:0/3851736396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12fc034050 con 0x55b77a7d4370 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 10 client.16644 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b77a84ee20 con 0x55b77a7d4370 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f12e3fff640 1 -- 172.21.15.16:0/3851736396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f40bc910 con 0x55b77a7d4370 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f12e3fff640 1 -- 172.21.15.16:0/3851736396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f12fc005630 con 0x55b77a7d4370 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 10 client.16644 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 20 client.16644 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a84f5d0 0x55b77a8519f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 4 client.16644 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.595+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b77a736d70 con 0x55b77a84f5d0 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f1308dc9640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a84f5d0 0x55b77a8519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f1308dc9640 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a84f5d0 0x55b77a8519f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a0f15e300c8e7d server_cookie=42e915379d3cd3c6 in_seq=0 out_seq=0 2024-09-15T22:44:04.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f12e3fff640 10 client.16644 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f12e3fff640 1 -- 172.21.15.16:0/3851736396 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b77a736d70 con 0x55b77a84f5d0 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f12e3fff640 10 client.16644 handle_command_reply: tid=0 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 client.16644 shutdown 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 msgr2=0x55b77a7d7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d7440 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0142e0 tx=0x7f12fc004a40 comp rx=0 tx=0).stop 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 shutdown_connections 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b77a84f5d0 0x55b77a8519f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b77a7d4370 0x55b77a7d7440 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b77a7cab40 0x55b77a7dc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.596+0000 7f130d406740 1 --2- 172.21.15.16:0/3851736396 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b77a7c95d0 0x55b77a7dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f12e3fff640 0 client.16644 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 >> 172.21.15.16:0/3851736396 conn(0x55b77a792310 msgr2=0x55b77a7d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 shutdown_connections 2024-09-15T22:44:04.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130d406740 1 -- 172.21.15.16:0/3851736396 wait complete. 2024-09-15T22:44:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130d406740 20 client.16644 trim_cache size 0 max 16384 2024-09-15T22:44:04.607 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f12ec0a33b0 msgr2=0x7f12ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f12ec0a33b0 0x7f12ec0a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f12fc005c40 tx=0x7f12fc009200 comp rx=0 tx=0).stop 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 msgr2=0x7f13040fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f13040fcc80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f12f4006010 tx=0x7f12f4003c10 comp rx=0 tx=0).stop 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 shutdown_connections 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f12ec0a33b0 0x7f12ec0a5870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1304102ef0 0x7f13040fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1304102570 0x7f13040fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 --2- 172.21.15.16:0/1536537710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1304100e20 0x7f13040fcc80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.597+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 >> 172.21.15.16:0/1536537710 conn(0x7f13040fc7d0 msgr2=0x7f130406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.598+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 shutdown_connections 2024-09-15T22:44:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:04.598+0000 7f130b02d640 1 -- 172.21.15.16:0/1536537710 wait complete. 2024-09-15T22:44:05.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:04 smithi130 ceph-mon[35742]: pgmap v172: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:05.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:04 smithi016 ceph-mon[31492]: pgmap v172: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:05.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:04 smithi059 ceph-mon[34872]: pgmap v172: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:06.314 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:44:06.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:44:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:06.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:06 smithi016 ceph-mon[31492]: pgmap v173: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:07.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:06 smithi130 ceph-mon[35742]: pgmap v173: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.069+0000 7fcbb0113640 1 -- 172.21.15.16:0/2004465431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 msgr2=0x7fcba810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.069+0000 7fcbb0113640 1 --2- 172.21.15.16:0/2004465431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba810da40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009900 tx=0x7fcba002a480 comp rx=0 tx=0).stop 2024-09-15T22:44:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 -- 172.21.15.16:0/2004465431 shutdown_connections 2024-09-15T22:44:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 --2- 172.21.15.16:0/2004465431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba810da40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 --2- 172.21.15.16:0/2004465431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba8102570 0x7fcba81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 --2- 172.21.15.16:0/2004465431 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcba8100e20 0x7fcba8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 -- 172.21.15.16:0/2004465431 >> 172.21.15.16:0/2004465431 conn(0x7fcba80fc7d0 msgr2=0x7fcba80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 -- 172.21.15.16:0/2004465431 shutdown_connections 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 -- 172.21.15.16:0/2004465431 wait complete. 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 Processor -- start 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.070+0000 7fcbb0113640 1 -- start start 2024-09-15T22:44:07.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbb0113640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba8100e20 0x7fcba80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbb0113640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcba8102570 0x7fcba80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbb0113640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbb0113640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8110fa0 con 0x7fcba8102ef0 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbb0113640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8110e20 con 0x7fcba8100e20 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbb0113640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8111120 con 0x7fcba8102570 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39416/0 (socket says 172.21.15.16:39416) 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbad6ae640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcba8102570 0x7fcba80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 -- 172.21.15.16:0/662177668 learned_addr learned my addr 172.21.15.16:0/662177668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba8100e20 0x7fcba80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 -- 172.21.15.16:0/662177668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcba8102570 msgr2=0x7fcba80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcba8102570 0x7fcba80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 -- 172.21.15.16:0/662177668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba8100e20 msgr2=0x7fcba80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba8100e20 0x7fcba80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 -- 172.21.15.16:0/662177668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba81a6970 con 0x7fcba8102ef0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcbae6b0640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba80fd700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcba0004770 tx=0x7fcba0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.071+0000 7fcb9effd640 1 -- 172.21.15.16:0/662177668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba003f070 con 0x7fcba8102ef0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.072+0000 7fcb9effd640 1 -- 172.21.15.16:0/662177668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba0031c30 con 0x7fcba8102ef0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.072+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba81a6c00 con 0x7fcba8102ef0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.072+0000 7fcb9effd640 1 -- 172.21.15.16:0/662177668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0035030 con 0x7fcba8102ef0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.072+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a7130 con 0x7fcba8102ef0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.074+0000 7fcb9effd640 1 -- 172.21.15.16:0/662177668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcba0042070 con 0x7fcba8102ef0 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.074+0000 7fcbb24ec740 1 Processor -- start 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.074+0000 7fcbb24ec740 1 -- start start 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf61f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcb9effd640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcb940a3430 0x7fcb940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576ccec7280 con 0x5576ccf6cd40 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5576cce6eb10 con 0x5576ccf61b60 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5576ccec3240 con 0x5576ccf62480 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbae6b0640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcb940a3430 0x7fcb940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39428/0 (socket says 172.21.15.16:39428) 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2751618454 learned_addr learned my addr 172.21.15.16:0/2751618454 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcb9effd640 1 -- 172.21.15.16:0/662177668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fcba00f07f0 con 0x7fcba8102ef0 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf61f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2751618454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 msgr2=0x5576ccf6c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2751618454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 msgr2=0x5576ccf61f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf61f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2751618454 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576cce7b1f0 con 0x5576ccf6cd40 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbae6b0640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcb940a3430 0x7fcb940a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcba00edc90 tx=0x7fcba003b870 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.075+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6f130 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcb980125c0 tx=0x7fcb9800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f172fcc122979c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcb93fff640 1 -- 172.21.15.16:0/2751618454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb980106b0 con 0x5576ccf6cd40 2024-09-15T22:44:07.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcb93fff640 1 -- 172.21.15.16:0/2751618454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb98010d30 con 0x5576ccf6cd40 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcb93fff640 1 -- 172.21.15.16:0/2751618454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9801dd20 con 0x5576ccf6cd40 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2751618454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 msgr2=0x5576ccf6f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6f130 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcb980125c0 tx=0x7fcb9800b870 comp rx=0 tx=0).stop 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2751618454 shutdown_connections 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6f130 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2751618454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf61f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.076+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2751618454 >> 172.21.15.16:0/2751618454 conn(0x5576ccf5e0f0 msgr2=0x5576ccf5e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.077+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2751618454 shutdown_connections 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.077+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2751618454 wait complete. 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.077+0000 7fcbb24ec740 1 -- start start 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.077+0000 7fcbb24ec740 1 Processor -- start 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf69c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5576ccf6f910 con 0x5576ccf6cd40 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5576cce81f10 con 0x5576ccf61b60 2024-09-15T22:44:07.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5576ccf6fa90 con 0x5576ccf62480 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 10 client.0 _ll_register_callbacks cb 0x5576cce4aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39442/0 (socket says 172.21.15.16:39442) 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 -- 172.21.15.16:0/484586674 learned_addr learned my addr 172.21.15.16:0/484586674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf69c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 -- 172.21.15.16:0/484586674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 msgr2=0x5576ccf6a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 -- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 msgr2=0x5576ccf69c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf69c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 -- 172.21.15.16:0/484586674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ccc386a0 con 0x5576ccf6cd40 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:07.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6a690 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800be90 tx=0x7fcb9801d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49b996db4811ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcb917fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcb917fa640 1 -- 172.21.15.16:0/484586674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98023040 con 0x5576ccf6cd40 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcb917fa640 1 -- 172.21.15.16:0/484586674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fcb98005050 con 0x5576ccf6cd40 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.078+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb98010d30 con 0x5576ccf6cd40 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcb917fa640 1 -- 172.21.15.16:0/484586674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb98005250 con 0x5576ccf6cd40 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 10 client.16656 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576ccf7b540 con 0x5576ccf6cd40 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcb917fa640 1 -- 172.21.15.16:0/484586674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb98010d30 con 0x5576ccf6cd40 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcb917fa640 1 -- 172.21.15.16:0/484586674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcb9801c070 con 0x5576ccf6cd40 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 10 client.16656 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:07.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 20 client.16656 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccf7c8f0 0x5576ccf7ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 4 client.16656 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576cced47f0 con 0x5576ccf7c8f0 2024-09-15T22:44:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.079+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccf7c8f0 0x5576ccf7ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.080+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccf7c8f0 0x5576ccf7ece0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64bf6e9314ecdf14 server_cookie=c07ef3399b2c1110 in_seq=0 out_seq=0 2024-09-15T22:44:07.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.080+0000 7fcb917fa640 10 client.16656 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcb917fa640 1 -- 172.21.15.16:0/484586674 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576cced47f0 con 0x5576ccf7c8f0 2024-09-15T22:44:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcb917fa640 10 client.16656 handle_command_reply: tid=0 2024-09-15T22:44:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 client.16656 shutdown 2024-09-15T22:44:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 msgr2=0x5576ccf6a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6a690 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800be90 tx=0x7fcb9801d8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 shutdown_connections 2024-09-15T22:44:07.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccf7c8f0 0x5576ccf7ece0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf6a690 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf62480 0x5576ccf6a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/484586674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcb917fa640 0 client.16656 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 >> 172.21.15.16:0/484586674 conn(0x5576ccf5e0f0 msgr2=0x5576ccf63dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 shutdown_connections 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 1 -- 172.21.15.16:0/484586674 wait complete. 2024-09-15T22:44:07.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.081+0000 7fcbb24ec740 20 client.16656 trim_cache size 0 max 16384 2024-09-15T22:44:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 Processor -- start 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 -- start start 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf65380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf62480 0x5576ccf5d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf6cd40 0x5576ccf84420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b3a00 con 0x5576ccf62480 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b12d0 con 0x5576ccf61b60 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.098+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81af120 con 0x5576ccf6cd40 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf62480 0x5576ccf5d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf62480 0x5576ccf5d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39448/0 (socket says 172.21.15.16:39448) 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 -- 172.21.15.16:0/2397279691 learned_addr learned my addr 172.21.15.16:0/2397279691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf65380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 -- 172.21.15.16:0/2397279691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf6cd40 msgr2=0x5576ccf84420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf6cd40 0x5576ccf84420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 -- 172.21.15.16:0/2397279691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 msgr2=0x5576ccf65380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf65380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 -- 172.21.15.16:0/2397279691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ccf6f470 con 0x5576ccf62480 2024-09-15T22:44:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf62480 0x5576ccf5d160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcba400a550 tx=0x7fcba4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4149f92739e243d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcb937fe640 1 -- 172.21.15.16:0/2397279691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4007aa0 con 0x5576ccf62480 2024-09-15T22:44:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcb937fe640 1 -- 172.21.15.16:0/2397279691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba4007c40 con 0x5576ccf62480 2024-09-15T22:44:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.099+0000 7fcb937fe640 1 -- 172.21.15.16:0/2397279691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4018620 con 0x5576ccf62480 2024-09-15T22:44:07.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2397279691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf62480 msgr2=0x5576ccf5d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf62480 0x5576ccf5d160 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fcba400a550 tx=0x7fcba4009830 comp rx=0 tx=0).stop 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2397279691 shutdown_connections 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf6cd40 0x5576ccf84420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf62480 0x5576ccf5d160 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2397279691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf61b60 0x5576ccf65380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2397279691 >> 172.21.15.16:0/2397279691 conn(0x5576ccf5e0f0 msgr2=0x5576ccf5ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2397279691 shutdown_connections 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2397279691 wait complete. 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 -- start start 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.100+0000 7fcbb24ec740 1 Processor -- start 2024-09-15T22:44:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf61b60 0x5576ccf70a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf62480 0x5576ccf70f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbb24ec740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81af790 con 0x5576ccf6cd40 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b0c00 con 0x5576ccf62480 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbb24ec740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81afe60 con 0x5576ccf61b60 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbb24ec740 10 client.0 _ll_register_callbacks cb 0x7fcba40096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf714b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39462/0 (socket says 172.21.15.16:39462) 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2143201903 learned_addr learned my addr 172.21.15.16:0/2143201903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf62480 0x5576ccf70f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf61b60 0x5576ccf70a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2143201903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf61b60 msgr2=0x5576ccf70a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf61b60 0x5576ccf70a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf62480 msgr2=0x5576ccf70f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf62480 0x5576ccf70f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 -- 172.21.15.16:0/2143201903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ccfe63f0 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcbad6ae640 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf714b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800c010 tx=0x7fcb9800b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd7e666e752b05b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.101+0000 7fcb917fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcb917fa640 1 -- 172.21.15.16:0/2143201903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98038040 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcb917fa640 1 -- 172.21.15.16:0/2143201903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fcb9801d5f0 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba4007c40 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcb917fa640 1 -- 172.21.15.16:0/2143201903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb98043050 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 10 client.16662 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576ccfe76e0 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcb917fa640 1 -- 172.21.15.16:0/2143201903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba4007c40 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcb917fa640 1 -- 172.21.15.16:0/2143201903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcb9801c040 con 0x5576ccf6cd40 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 10 client.16662 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 20 client.16662 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccfe7da0 0x5576ccfea1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 4 client.16662 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.102+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5576cceced00 con 0x5576ccfe7da0 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.103+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccfe7da0 0x5576ccfea1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.103+0000 7fcb917fa640 10 client.16662 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.103+0000 7fcbadeaf640 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccfe7da0 0x5576ccfea1c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21393f97e6178dab server_cookie=5a2b1dd820bdcd2c in_seq=0 out_seq=0 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcb917fa640 1 -- 172.21.15.16:0/2143201903 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5576cceced00 con 0x5576ccfe7da0 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcb917fa640 10 client.16662 handle_command_reply: tid=0 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 client.16662 shutdown 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 msgr2=0x5576ccf714b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf714b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800c010 tx=0x7fcb9800b570 comp rx=0 tx=0).stop 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 shutdown_connections 2024-09-15T22:44:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5576ccfe7da0 0x5576ccfea1c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576ccf6cd40 0x5576ccf714b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5576ccf62480 0x5576ccf70f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 --2- 172.21.15.16:0/2143201903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5576ccf61b60 0x5576ccf70a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 >> 172.21.15.16:0/2143201903 conn(0x5576ccf5e0f0 msgr2=0x5576ccf6a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcb917fa640 0 client.16662 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 shutdown_connections 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.104+0000 7fcbb24ec740 1 -- 172.21.15.16:0/2143201903 wait complete. 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb24ec740 20 client.16662 trim_cache size 0 max 16384 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:44:07.112 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "32020d25-644c-4659-b34e-67e6d927bc92", 2024-09-15T22:44:07.113 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:44:07.113 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:44:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcb940a3430 msgr2=0x7fcb940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcb940a3430 0x7fcb940a58f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcba00edc90 tx=0x7fcba003b870 comp rx=0 tx=0).stop 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 msgr2=0x7fcba80fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba80fd700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcba0004770 tx=0x7fcba0003c10 comp rx=0 tx=0).stop 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 shutdown_connections 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba8102ef0 0x7fcba80fd700 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcb940a3430 0x7fcb940a58f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcba8102570 0x7fcba80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 --2- 172.21.15.16:0/662177668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcba8100e20 0x7fcba80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 >> 172.21.15.16:0/662177668 conn(0x7fcba80fc7d0 msgr2=0x7fcba806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 shutdown_connections 2024-09-15T22:44:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:07.105+0000 7fcbb0113640 1 -- 172.21.15.16:0/662177668 wait complete. 2024-09-15T22:44:07.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:06 smithi059 ceph-mon[34872]: pgmap v173: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:07.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:44:08.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:07 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:44:08.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:07 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:08.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:07 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:44:08.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:07 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:08.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:07 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:44:08.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:07 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:08.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:07 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:44:08.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:07 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:08.118 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:08.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:07 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:44:08.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:07 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:08.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:07 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:44:08.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:07 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 -- 172.21.15.16:0/485077572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106fc0 msgr2=0x7f67dc10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 --2- 172.21.15.16:0/485077572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106fc0 0x7f67dc10eaf0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009cd0 tx=0x7f67d402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 -- 172.21.15.16:0/485077572 shutdown_connections 2024-09-15T22:44:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 --2- 172.21.15.16:0/485077572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc106fc0 0x7f67dc10eaf0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 --2- 172.21.15.16:0/485077572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc100d20 0x7f67dc106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 --2- 172.21.15.16:0/485077572 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc100370 0x7f67dc100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 -- 172.21.15.16:0/485077572 >> 172.21.15.16:0/485077572 conn(0x7f67dc077210 msgr2=0x7f67dc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 -- 172.21.15.16:0/485077572 shutdown_connections 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.674+0000 7f67e3299640 1 -- 172.21.15.16:0/485077572 wait complete. 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 Processor -- start 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 -- start start 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc100370 0x7f67dc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc100d20 0x7f67dc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc106fc0 0x7f67dc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc111fa0 con 0x7f67dc100370 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc111e20 con 0x7f67dc100d20 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e3299640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc112120 con 0x7f67dc106fc0 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc100370 0x7f67dc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc100370 0x7f67dc1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39494/0 (socket says 172.21.15.16:39494) 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 -- 172.21.15.16:0/3225885343 learned_addr learned my addr 172.21.15.16:0/3225885343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1836640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc106fc0 0x7f67dc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e0834640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc100d20 0x7f67dc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 -- 172.21.15.16:0/3225885343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc106fc0 msgr2=0x7f67dc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc106fc0 0x7f67dc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 -- 172.21.15.16:0/3225885343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc100d20 msgr2=0x7f67dc198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc100d20 0x7f67dc198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1035640 1 -- 172.21.15.16:0/3225885343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc19d370 con 0x7f67dc100370 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.675+0000 7f67e1836640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc106fc0 0x7f67dc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.676+0000 7f67e1035640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc100370 0x7f67dc1983c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c890 tx=0x7f67d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.676+0000 7f67d27fc640 1 -- 172.21.15.16:0/3225885343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004220 con 0x7f67dc100370 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.676+0000 7f67d27fc640 1 -- 172.21.15.16:0/3225885343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d80043c0 con 0x7f67dc100370 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.676+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc19d660 con 0x7f67dc100370 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.676+0000 7f67d27fc640 1 -- 172.21.15.16:0/3225885343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d80049e0 con 0x7f67dc100370 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.676+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1a4f40 con 0x7f67dc100370 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.678+0000 7f67d27fc640 1 -- 172.21.15.16:0/3225885343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f67d80079b0 con 0x7f67dc100370 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.678+0000 7f67e5672740 1 Processor -- start 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e5672740 1 -- start start 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e5672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8cc380 0x55df4e8cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e5672740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8ccca0 0x55df4e8db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e5672740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8dde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e5672740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df4e8322e0 con 0x55df4e8cc380 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e5672740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df4e7d9ce0 con 0x55df4e8ccca0 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e5672740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55df4e82e2a0 con 0x55df4e8dba90 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8cc380 0x55df4e8cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8cc380 0x55df4e8cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39496/0 (socket says 172.21.15.16:39496) 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 -- 172.21.15.16:0/2011276945 learned_addr learned my addr 172.21.15.16:0/2011276945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e0834640 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1035640 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8ccca0 0x55df4e8db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67d27fc640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a33b0 0x7f67c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 -- 172.21.15.16:0/2011276945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 msgr2=0x55df4e8dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8dde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 -- 172.21.15.16:0/2011276945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8ccca0 msgr2=0x55df4e8db550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8ccca0 0x55df4e8db550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 -- 172.21.15.16:0/2011276945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4e7be440 con 0x55df4e8cc380 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1035640 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8ccca0 0x55df4e8db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.679+0000 7f67e1836640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a33b0 0x7f67c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.680+0000 7f67d27fc640 1 -- 172.21.15.16:0/3225885343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f67d80d0190 con 0x7f67dc100370 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.680+0000 7f67e1836640 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8cc380 0x55df4e8cc760 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f67d4031650 tx=0x7f67d4031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5379e6a60244fa35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.680+0000 7f67c77fe640 1 -- 172.21.15.16:0/2011276945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4003e40 con 0x55df4e8cc380 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.680+0000 7f67c77fe640 1 -- 172.21.15.16:0/2011276945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4036040 con 0x55df4e8cc380 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.680+0000 7f67c77fe640 1 -- 172.21.15.16:0/2011276945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4030920 con 0x55df4e8cc380 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 -- 172.21.15.16:0/2011276945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8cc380 msgr2=0x55df4e8cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e1836640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a33b0 0x7f67c80a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67d404b000 tx=0x7f67d4048000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8cc380 0x55df4e8cc760 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f67d4031650 tx=0x7f67d4031680 comp rx=0 tx=0).stop 2024-09-15T22:44:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 -- 172.21.15.16:0/2011276945 shutdown_connections 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8ccca0 0x55df4e8db550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 --2- 172.21.15.16:0/2011276945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8cc380 0x55df4e8cc760 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 -- 172.21.15.16:0/2011276945 >> 172.21.15.16:0/2011276945 conn(0x55df4e8c8a30 msgr2=0x55df4e8c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 -- 172.21.15.16:0/2011276945 shutdown_connections 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 -- 172.21.15.16:0/2011276945 wait complete. 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 -- start start 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.681+0000 7f67e5672740 1 Processor -- start 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55df4e8de660 con 0x55df4e8ccca0 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55df4e7ed050 con 0x55df4e8cc380 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55df4e8de7e0 con 0x55df4e8dba90 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 10 client.0 _ll_register_callbacks cb 0x55df4e82d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39498/0 (socket says 172.21.15.16:39498) 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 -- 172.21.15.16:0/1109250885 learned_addr learned my addr 172.21.15.16:0/1109250885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e1836640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e1035640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 -- 172.21.15.16:0/1109250885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 msgr2=0x55df4e8d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 -- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 msgr2=0x55df4e8db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8db470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 -- 172.21.15.16:0/1109250885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4e5a33c0 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e1035640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e0834640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d8720 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00bec0 tx=0x7f67cc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b766d1b8a3b08a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1109250885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc004230 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d4036040 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.682+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1109250885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f67cc0043d0 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1109250885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67cc0106a0 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 10 client.16674 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df4e8ea190 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1109250885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4036040 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1109250885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f67cc003bb0 con 0x55df4e8ccca0 2024-09-15T22:44:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 10 client.16674 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 10 client.16674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 20 client.16674 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e8eb5f0 0x55df4e8eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 4 client.16674 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df4e83f850 con 0x55df4e8eb5f0 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.683+0000 7f67e1035640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e8eb5f0 0x55df4e8eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.684+0000 7f67c4ff9640 10 client.16674 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.684+0000 7f67e1035640 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e8eb5f0 0x55df4e8eda30 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f17b0ca9c51070fa server_cookie=71ee55fee4e815d2 in_seq=0 out_seq=0 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.684+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1109250885 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df4e83f850 con 0x55df4e8eb5f0 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.684+0000 7f67c4ff9640 10 client.16674 handle_command_reply: tid=0 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 client.16674 shutdown 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 msgr2=0x55df4e8d8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d8720 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00bec0 tx=0x7f67cc012820 comp rx=0 tx=0).stop 2024-09-15T22:44:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 shutdown_connections 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e8eb5f0 0x55df4e8eda30 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d8720 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 --2- 172.21.15.16:0/1109250885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67c4ff9640 0 client.16674 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 >> 172.21.15.16:0/1109250885 conn(0x55df4e8c8a30 msgr2=0x55df4e8ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 shutdown_connections 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 1 -- 172.21.15.16:0/1109250885 wait complete. 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.685+0000 7f67e5672740 20 client.16674 trim_cache size 0 max 16384 2024-09-15T22:44:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 Processor -- start 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 -- start start 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1af6b0 con 0x55df4e8ccca0 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1acf80 con 0x55df4e8cc380 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e5672740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1aadd0 con 0x55df4e8dba90 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.702+0000 7f67e0834640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39508/0 (socket says 172.21.15.16:39508) 2024-09-15T22:44:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 -- 172.21.15.16:0/94257799 learned_addr learned my addr 172.21.15.16:0/94257799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e1035640 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e1836640 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 -- 172.21.15.16:0/94257799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 msgr2=0x55df4e8f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 -- 172.21.15.16:0/94257799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 msgr2=0x55df4e8e7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 -- 172.21.15.16:0/94257799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4e8db7e0 con 0x55df4e8ccca0 2024-09-15T22:44:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e1836640 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67e0834640 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8f3300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f67cc005d00 tx=0x7f67cc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec4c8a0b78ee05a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67c6ffd640 1 -- 172.21.15.16:0/94257799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0041d0 con 0x55df4e8ccca0 2024-09-15T22:44:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67c6ffd640 1 -- 172.21.15.16:0/94257799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67cc012d20 con 0x55df4e8ccca0 2024-09-15T22:44:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.703+0000 7f67c6ffd640 1 -- 172.21.15.16:0/94257799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc01bc00 con 0x55df4e8ccca0 2024-09-15T22:44:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 -- 172.21.15.16:0/94257799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 msgr2=0x55df4e8f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8f3300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f67cc005d00 tx=0x7f67cc005850 comp rx=0 tx=0).stop 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 -- 172.21.15.16:0/94257799 shutdown_connections 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8f3300 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 --2- 172.21.15.16:0/94257799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 -- 172.21.15.16:0/94257799 >> 172.21.15.16:0/94257799 conn(0x55df4e8c8a30 msgr2=0x55df4e8d3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 -- 172.21.15.16:0/94257799 shutdown_connections 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 -- 172.21.15.16:0/94257799 wait complete. 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 -- start start 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.704+0000 7f67e5672740 1 Processor -- start 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e5672740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e5672740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e5672740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e5672740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1ab440 con 0x55df4e8ccca0 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e5672740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1ac8b0 con 0x55df4e8cc380 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e5672740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1abb10 con 0x55df4e8dba90 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e5672740 10 client.0 _ll_register_callbacks cb 0x7f67cc00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39512/0 (socket says 172.21.15.16:39512) 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 -- 172.21.15.16:0/1103689215 learned_addr learned my addr 172.21.15.16:0/1103689215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e1836640 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e1035640 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 -- 172.21.15.16:0/1103689215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 msgr2=0x55df4e8d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 -- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 msgr2=0x55df4e8e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 -- 172.21.15.16:0/1103689215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4e8d62f0 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67e0834640 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d5520 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00bec0 tx=0x7f67cc00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7184c337c07be19 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.705+0000 7f67c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1103689215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc010a40 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1103689215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f67cc010be0 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67cc012d20 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1103689215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67cc041050 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 10 client.16680 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df4e956320 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1103689215 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67cc012d20 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1103689215 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f67cc003bf0 con 0x55df4e8ccca0 2024-09-15T22:44:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 10 client.16680 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 20 client.16680 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e9569d0 0x55df4e958df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 4 client.16680 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df4e839d60 con 0x55df4e9569d0 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.706+0000 7f67e1035640 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e9569d0 0x55df4e958df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67c4ff9640 10 client.16680 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e1035640 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e9569d0 0x55df4e958df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae287fb41110c5dd server_cookie=55f8a65d17030caa in_seq=0 out_seq=0 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67c4ff9640 1 -- 172.21.15.16:0/1103689215 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df4e839d60 con 0x55df4e9569d0 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67c4ff9640 10 client.16680 handle_command_reply: tid=0 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 client.16680 shutdown 2024-09-15T22:44:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 msgr2=0x55df4e8d5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d5520 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00bec0 tx=0x7f67cc00ba40 comp rx=0 tx=0).stop 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 shutdown_connections 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55df4e9569d0 0x55df4e958df0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55df4e8dba90 0x55df4e8d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55df4e8ccca0 0x55df4e8d5520 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67c4ff9640 0 client.16680 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 --2- 172.21.15.16:0/1103689215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55df4e8cc380 0x55df4e8e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.758 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 >> 172.21.15.16:0/1103689215 conn(0x55df4e8c8a30 msgr2=0x55df4e8d9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 shutdown_connections 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 1 -- 172.21.15.16:0/1103689215 wait complete. 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.707+0000 7f67e5672740 20 client.16680 trim_cache size 0 max 16384 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a33b0 msgr2=0x7f67c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a33b0 0x7f67c80a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f67d404b000 tx=0x7f67d4048000 comp rx=0 tx=0).stop 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc100370 msgr2=0x7f67dc1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc100370 0x7f67dc1983c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c890 tx=0x7f67d800cd50 comp rx=0 tx=0).stop 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 shutdown_connections 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f67c80a33b0 0x7f67c80a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f67dc106fc0 0x7f67dc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67dc100d20 0x7f67dc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 --2- 172.21.15.16:0/3225885343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67dc100370 0x7f67dc1983c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 >> 172.21.15.16:0/3225885343 conn(0x7f67dc077210 msgr2=0x7f67dc105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 shutdown_connections 2024-09-15T22:44:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:08.708+0000 7f67e3299640 1 -- 172.21.15.16:0/3225885343 wait complete. 2024-09-15T22:44:08.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:08 smithi016 ceph-mon[31492]: pgmap v174: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:08.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:08 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:44:09.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:08 smithi130 ceph-mon[35742]: pgmap v174: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:09.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:08 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:44:09.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:08 smithi059 ceph-mon[34872]: pgmap v174: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:09.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:08 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:44:09.591 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:44:09.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:44:09.853 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:10.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 -- 172.21.15.16:0/2008275002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1036e0 msgr2=0x7f7c2c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:10.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 --2- 172.21.15.16:0/2008275002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1036e0 0x7f7c2c107b90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009930 tx=0x7f7c1c02a420 comp rx=0 tx=0).stop 2024-09-15T22:44:10.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 -- 172.21.15.16:0/2008275002 shutdown_connections 2024-09-15T22:44:10.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 --2- 172.21.15.16:0/2008275002 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c2c1082a0 0x7f7c2c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 --2- 172.21.15.16:0/2008275002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1036e0 0x7f7c2c107b90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 --2- 172.21.15.16:0/2008275002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c102d30 0x7f7c2c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 -- 172.21.15.16:0/2008275002 >> 172.21.15.16:0/2008275002 conn(0x7f7c2c0fc7d0 msgr2=0x7f7c2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 -- 172.21.15.16:0/2008275002 shutdown_connections 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.331+0000 7f7c3449f640 1 -- 172.21.15.16:0/2008275002 wait complete. 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 Processor -- start 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 -- start start 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c2c102d30 0x7f7c2c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c1036e0 0x7f7c2c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1082a0 0x7f7c2c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c112060 con 0x7f7c2c1082a0 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c111ee0 con 0x7f7c2c1036e0 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.332+0000 7f7c3449f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c1121e0 con 0x7f7c2c102d30 2024-09-15T22:44:10.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1082a0 0x7f7c2c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c31a3a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c1036e0 0x7f7c2c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1082a0 0x7f7c2c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39526/0 (socket says 172.21.15.16:39526) 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c3223b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c2c102d30 0x7f7c2c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 -- 172.21.15.16:0/3025394437 learned_addr learned my addr 172.21.15.16:0/3025394437 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 -- 172.21.15.16:0/3025394437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c2c102d30 msgr2=0x7f7c2c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c2c102d30 0x7f7c2c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 -- 172.21.15.16:0/3025394437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c1036e0 msgr2=0x7f7c2c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c1036e0 0x7f7c2c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 -- 172.21.15.16:0/3025394437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c1a69c0 con 0x7f7c2c1082a0 2024-09-15T22:44:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c32a3c640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1082a0 0x7f7c2c101410 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400eaa0 tx=0x7f7c2400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c237fe640 1 -- 172.21.15.16:0/3025394437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24004d20 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c237fe640 1 -- 172.21.15.16:0/3025394437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c24005020 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c1a6cb0 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c237fe640 1 -- 172.21.15.16:0/3025394437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c240116d0 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.333+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c1a7170 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.334+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2c103110 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.336+0000 7f7c237fe640 1 -- 172.21.15.16:0/3025394437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7c240051d0 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.336+0000 7f7c237fe640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c180a33b0 0x7f7c180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.336+0000 7f7c237fe640 1 -- 172.21.15.16:0/3025394437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7c240d0190 con 0x7f7c2c1082a0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.336+0000 7f7c3223b640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c180a33b0 0x7f7c180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.337+0000 7f7c3223b640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c180a33b0 0x7f7c180a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800abe0 tx=0x7f7c2800a2d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.341+0000 7f7c237fe640 1 -- 172.21.15.16:0/3025394437 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c24016030 con 0x7f7c2c1082a0 2024-09-15T22:44:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.554+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c2c10a2b0 con 0x7f7c2c1082a0 2024-09-15T22:44:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.556+0000 7f7c237fe640 1 -- 172.21.15.16:0/3025394437 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f7c2409c810 con 0x7f7c2c1082a0 2024-09-15T22:44:10.557 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:44:10.557 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:44:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:44:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c180a33b0 msgr2=0x7f7c180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c180a33b0 0x7f7c180a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c2800abe0 tx=0x7f7c2800a2d0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1082a0 msgr2=0x7f7c2c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1082a0 0x7f7c2c101410 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7c2400eaa0 tx=0x7f7c2400eea0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 shutdown_connections 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c2c1082a0 0x7f7c2c101410 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c2c1036e0 0x7f7c2c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f7c180a33b0 0x7f7c180a5870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 --2- 172.21.15.16:0/3025394437 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7c2c102d30 0x7f7c2c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 >> 172.21.15.16:0/3025394437 conn(0x7f7c2c0fc7d0 msgr2=0x7f7c2c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 shutdown_connections 2024-09-15T22:44:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:10.558+0000 7f7c3449f640 1 -- 172.21.15.16:0/3025394437 wait complete. 2024-09-15T22:44:10.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:10 smithi016 ceph-mon[31492]: pgmap v175: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:10.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:10 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3025394437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:11.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:10 smithi130 ceph-mon[35742]: pgmap v175: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:11.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:10 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3025394437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:11.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:10 smithi059 ceph-mon[34872]: pgmap v175: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-09-15T22:44:11.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:10 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3025394437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:11.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:44:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 -- 172.21.15.16:0/811131336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8101420 msgr2=0x7f1af810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 --2- 172.21.15.16:0/811131336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8101420 0x7f1af810d950 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009cd0 tx=0x7f1af002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 -- 172.21.15.16:0/811131336 shutdown_connections 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 --2- 172.21.15.16:0/811131336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8101420 0x7f1af810d950 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 --2- 172.21.15.16:0/811131336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8100a80 0x7f1af8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 --2- 172.21.15.16:0/811131336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af8106a80 0x7f1af8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 -- 172.21.15.16:0/811131336 >> 172.21.15.16:0/811131336 conn(0x7f1af80fc7a0 msgr2=0x7f1af80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 -- 172.21.15.16:0/811131336 shutdown_connections 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.132+0000 7f1b0022b640 1 -- 172.21.15.16:0/811131336 wait complete. 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 Processor -- start 2024-09-15T22:44:12.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 -- start start 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8100a80 0x7f1af819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8101420 0x7f1af819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af8106a80 0x7f1af819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8110e80 con 0x7f1af8100a80 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8110d00 con 0x7f1af8106a80 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1b0022b640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1af8111000 con 0x7f1af8101420 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1afdfc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8100a80 0x7f1af819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1afdfc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8100a80 0x7f1af819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38764/0 (socket says 172.21.15.16:38764) 2024-09-15T22:44:12.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.133+0000 7f1afdfc7640 1 -- 172.21.15.16:0/4229591701 learned_addr learned my addr 172.21.15.16:0/4229591701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af8106a80 0x7f1af819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8101420 0x7f1af819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afdfc7640 1 -- 172.21.15.16:0/4229591701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8101420 msgr2=0x7f1af819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8101420 0x7f1af819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afdfc7640 1 -- 172.21.15.16:0/4229591701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af8106a80 msgr2=0x7f1af819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af8106a80 0x7f1af819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afdfc7640 1 -- 172.21.15.16:0/4229591701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af819f5f0 con 0x7f1af8100a80 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8100a80 0x7f1af819a640 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1af400e9f0 tx=0x7f1af400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1aeeffd640 1 -- 172.21.15.16:0/4229591701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af400cd80 con 0x7f1af8100a80 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1aeeffd640 1 -- 172.21.15.16:0/4229591701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af4010910 con 0x7f1af8100a80 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1aeeffd640 1 -- 172.21.15.16:0/4229591701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4018990 con 0x7f1af8100a80 2024-09-15T22:44:12.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af819f8e0 con 0x7f1af8100a80 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.134+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af81a71c0 con 0x7f1af8100a80 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1aeeffd640 1 -- 172.21.15.16:0/4229591701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1af4010ac0 con 0x7f1af8100a80 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 Processor -- start 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 -- start start 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982352090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579823525d0 0x557982352a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5579822b82f0 con 0x5579823525d0 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55798225fcf0 con 0x55798235d370 2024-09-15T22:44:12.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1b02604740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5579822b42b0 con 0x557982353b40 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1afdfc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579823525d0 0x557982352a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1afdfc7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579823525d0 0x557982352a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38774/0 (socket says 172.21.15.16:38774) 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1afdfc7640 1 -- 172.21.15.16:0/2806196399 learned_addr learned my addr 172.21.15.16:0/2806196399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.137+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982352090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1aeeffd640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1ae40a3430 0x7f1ae40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1ae40a3430 0x7f1ae40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1aeeffd640 1 -- 172.21.15.16:0/4229591701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1af40d0300 con 0x7f1af8100a80 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afdfc7640 1 -- 172.21.15.16:0/2806196399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 msgr2=0x557982352090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982352090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afdfc7640 1 -- 172.21.15.16:0/2806196399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 msgr2=0x55798235f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afdfc7640 1 -- 172.21.15.16:0/2806196399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579822443a0 con 0x5579823525d0 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579823525d0 0x557982352a10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1af409bf80 tx=0x7f1af4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c536ba55032eb65a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1ae3fff640 1 -- 172.21.15.16:0/2806196399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af409c980 con 0x5579823525d0 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1ae40a3430 0x7f1ae40a58f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1af0004ab0 tx=0x7f1af002a8e0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1ae3fff640 1 -- 172.21.15.16:0/2806196399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af409c280 con 0x5579823525d0 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.138+0000 7f1ae3fff640 1 -- 172.21.15.16:0/2806196399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af40988f0 con 0x5579823525d0 2024-09-15T22:44:12.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 -- 172.21.15.16:0/2806196399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579823525d0 msgr2=0x557982352a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579823525d0 0x557982352a10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1af409bf80 tx=0x7f1af4018420 comp rx=0 tx=0).stop 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 -- 172.21.15.16:0/2806196399 shutdown_connections 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5579823525d0 0x557982352a10 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 --2- 172.21.15.16:0/2806196399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982352090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 -- 172.21.15.16:0/2806196399 >> 172.21.15.16:0/2806196399 conn(0x55798231b310 msgr2=0x55798234ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 -- 172.21.15.16:0/2806196399 shutdown_connections 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.139+0000 7f1b02604740 1 -- 172.21.15.16:0/2806196399 wait complete. 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 -- start start 2024-09-15T22:44:12.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 Processor -- start 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x55798235cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557982353b40 0x55798235a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55798235ffb0 con 0x557982353b40 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557982273060 con 0x55798235d370 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557982360130 con 0x5579823525d0 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1b02604740 10 client.0 _ll_register_callbacks cb 0x557981fc68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1afd7c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557982353b40 0x55798235a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1afd7c6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557982353b40 0x55798235a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38788/0 (socket says 172.21.15.16:38788) 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1afd7c6640 1 -- 172.21.15.16:0/1144406147 learned_addr learned my addr 172.21.15.16:0/1144406147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.140+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x55798235cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1afd7c6640 1 -- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 msgr2=0x55798235cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x55798235cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1afd7c6640 1 -- 172.21.15.16:0/1144406147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 msgr2=0x55798235a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1afd7c6640 1 -- 172.21.15.16:0/1144406147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579820293c0 con 0x557982353b40 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557982353b40 0x55798235a080 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800be30 tx=0x7f1ae8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4bed24664da266 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1ae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1ae17fa640 1 -- 172.21.15.16:0/1144406147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8004230 con 0x557982353b40 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1ae17fa640 1 -- 172.21.15.16:0/1144406147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1ae80043d0 con 0x557982353b40 2024-09-15T22:44:12.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af409c280 con 0x557982353b40 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1ae17fa640 1 -- 172.21.15.16:0/1144406147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ae8024050 con 0x557982353b40 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1b02604740 10 client.16698 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55798236bb80 con 0x557982353b40 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1ae17fa640 1 -- 172.21.15.16:0/1144406147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af409c280 con 0x557982353b40 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1ae17fa640 1 -- 172.21.15.16:0/1144406147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1ae8010a60 con 0x557982353b40 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1b02604740 10 client.16698 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1b02604740 10 client.16698 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.141+0000 7f1b02604740 20 client.16698 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.142+0000 7f1b02604740 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55798236d090 0x55798236f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.142+0000 7f1b02604740 4 client.16698 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.142+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579822c5860 con 0x55798236d090 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.142+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55798236d090 0x55798236f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.142+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55798236d090 0x55798236f4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bf2789967cf3041 server_cookie=89c0d0d4fc9bb56a in_seq=0 out_seq=0 2024-09-15T22:44:12.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.142+0000 7f1ae17fa640 10 client.16698 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1ae17fa640 1 -- 172.21.15.16:0/1144406147 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579822c5860 con 0x55798236d090 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1ae17fa640 10 client.16698 handle_command_reply: tid=0 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 client.16698 shutdown 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557982353b40 msgr2=0x55798235a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557982353b40 0x55798235a080 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800be30 tx=0x7f1ae8012820 comp rx=0 tx=0).stop 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 shutdown_connections 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55798236d090 0x55798236f4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55798235d370 0x55798235a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557982353b40 0x55798235a080 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1ae17fa640 0 client.16698 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 --2- 172.21.15.16:0/1144406147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x55798235cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 >> 172.21.15.16:0/1144406147 conn(0x55798231b310 msgr2=0x557982354990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:12.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 shutdown_connections 2024-09-15T22:44:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.143+0000 7f1b02604740 1 -- 172.21.15.16:0/1144406147 wait complete. 2024-09-15T22:44:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.144+0000 7f1b02604740 20 client.16698 trim_cache size 0 max 16384 2024-09-15T22:44:12.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.156+0000 7f1b02604740 1 Processor -- start 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.156+0000 7f1b02604740 1 -- start start 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1b02604740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579823525d0 0x5579823533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1b02604740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982374da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1b02604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823752e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1b02604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81b3a90 con 0x55798235d370 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1b02604740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81b1360 con 0x5579823525d0 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1b02604740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81af1b0 con 0x557982353b40 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823752e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823752e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38802/0 (socket says 172.21.15.16:38802) 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3967340852 learned_addr learned my addr 172.21.15.16:0/3967340852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982374da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579823525d0 0x5579823533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3967340852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 msgr2=0x557982374da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982374da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3967340852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579823525d0 msgr2=0x5579823533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579823525d0 0x5579823533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3967340852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579823585d0 con 0x55798235d370 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.157+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823752e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009cd0 tx=0x7f1af0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866a436e3b3b1d37 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1ae37fe640 1 -- 172.21.15.16:0/3967340852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0004590 con 0x55798235d370 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1ae37fe640 1 -- 172.21.15.16:0/3967340852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af0036050 con 0x55798235d370 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1ae37fe640 1 -- 172.21.15.16:0/3967340852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af004a690 con 0x55798235d370 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 -- 172.21.15.16:0/3967340852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 msgr2=0x5579823752e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823752e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009cd0 tx=0x7f1af0009360 comp rx=0 tx=0).stop 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 -- 172.21.15.16:0/3967340852 shutdown_connections 2024-09-15T22:44:12.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823752e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557982353b40 0x557982374da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 --2- 172.21.15.16:0/3967340852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5579823525d0 0x5579823533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 -- 172.21.15.16:0/3967340852 >> 172.21.15.16:0/3967340852 conn(0x55798231b310 msgr2=0x557982349a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 -- 172.21.15.16:0/3967340852 shutdown_connections 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.158+0000 7f1b02604740 1 -- 172.21.15.16:0/3967340852 wait complete. 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 -- start start 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 Processor -- start 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x557982365be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557982353b40 0x557982356ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81af820 con 0x55798235d370 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81b0c90 con 0x557982353b40 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1af81afef0 con 0x5579823525d0 2024-09-15T22:44:12.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1b02604740 10 client.0 _ll_register_callbacks cb 0x7f1ae800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1afe7c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.159+0000 7f1afe7c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38808/0 (socket says 172.21.15.16:38808) 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3673391790 learned_addr learned my addr 172.21.15.16:0/3673391790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x557982365be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557982353b40 0x557982356ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 msgr2=0x557982365be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x557982365be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3673391790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557982353b40 msgr2=0x557982356ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557982353b40 0x557982356ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afe7c8640 1 -- 172.21.15.16:0/3673391790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557982357ce0 con 0x55798235d370 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afd7c6640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557982353b40 0x557982356ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1afe7c8640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823574e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1af0005e30 tx=0x7f1af00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8ac924450d3793 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1ae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1ae17fa640 1 -- 172.21.15.16:0/3673391790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af005c060 con 0x55798235d370 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1ae17fa640 1 -- 172.21.15.16:0/3673391790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f1af00305a0 con 0x55798235d370 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1af0036050 con 0x55798235d370 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1ae17fa640 1 -- 172.21.15.16:0/3673391790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1af0031d40 con 0x55798235d370 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1b02604740 10 client.16704 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.160+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579823d7e70 con 0x55798235d370 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1ae17fa640 1 -- 172.21.15.16:0/3673391790 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1af0036050 con 0x55798235d370 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1ae17fa640 1 -- 172.21.15.16:0/3673391790 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1af0030070 con 0x55798235d370 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1b02604740 10 client.16704 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1b02604740 10 client.16704 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1b02604740 20 client.16704 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1b02604740 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5579823d84e0 0x5579823da8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1b02604740 4 client.16704 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579822bfd70 con 0x5579823d84e0 2024-09-15T22:44:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.161+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5579823d84e0 0x5579823da8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1afdfc7640 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5579823d84e0 0x5579823da8d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4eeebf77fcff1840 server_cookie=739c45d362e92878 in_seq=0 out_seq=0 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1ae17fa640 10 client.16704 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1ae17fa640 1 -- 172.21.15.16:0/3673391790 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579822bfd70 con 0x5579823d84e0 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1ae17fa640 10 client.16704 handle_command_reply: tid=0 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 client.16704 shutdown 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 msgr2=0x5579823574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823574e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1af0005e30 tx=0x7f1af00095c0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 shutdown_connections 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5579823d84e0 0x5579823da8d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55798235d370 0x5579823574e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557982353b40 0x557982356ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1ae17fa640 0 client.16704 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 --2- 172.21.15.16:0/3673391790 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5579823525d0 0x557982365be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 >> 172.21.15.16:0/3673391790 conn(0x55798231b310 msgr2=0x557982358950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 shutdown_connections 2024-09-15T22:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 1 -- 172.21.15.16:0/3673391790 wait complete. 2024-09-15T22:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.162+0000 7f1b02604740 20 client.16704 trim_cache size 0 max 16384 2024-09-15T22:44:12.171 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:12.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1ae40a3430 msgr2=0x7f1ae40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1ae40a3430 0x7f1ae40a58f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1af0004ab0 tx=0x7f1af002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8100a80 msgr2=0x7f1af819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8100a80 0x7f1af819a640 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1af400e9f0 tx=0x7f1af400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 shutdown_connections 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af8106a80 0x7f1af819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1ae40a3430 0x7f1ae40a58f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1af8101420 0x7f1af819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 --2- 172.21.15.16:0/4229591701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1af8100a80 0x7f1af819a640 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 >> 172.21.15.16:0/4229591701 conn(0x7f1af80fc7a0 msgr2=0x7f1af810d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 shutdown_connections 2024-09-15T22:44:12.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:12.163+0000 7f1b0022b640 1 -- 172.21.15.16:0/4229591701 wait complete. 2024-09-15T22:44:12.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:44:12.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:12 smithi016 ceph-mon[31492]: pgmap v176: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:13.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:12 smithi130 ceph-mon[35742]: pgmap v176: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:13.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:12 smithi059 ceph-mon[34872]: pgmap v176: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-15T22:44:13.221 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 -- 172.21.15.16:0/2981223625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330101de0 msgr2=0x7f23301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 --2- 172.21.15.16:0/2981223625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330101de0 0x7f23301021c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f232c009900 tx=0x7f232c02a410 comp rx=0 tx=0).stop 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 -- 172.21.15.16:0/2981223625 shutdown_connections 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 --2- 172.21.15.16:0/2981223625 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2330106f10 0x7f233010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 --2- 172.21.15.16:0/2981223625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2330102790 0x7f23301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 --2- 172.21.15.16:0/2981223625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330101de0 0x7f23301021c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 -- 172.21.15.16:0/2981223625 >> 172.21.15.16:0/2981223625 conn(0x7f23300fd8c0 msgr2=0x7f23300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 -- 172.21.15.16:0/2981223625 shutdown_connections 2024-09-15T22:44:13.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.698+0000 7f2338132640 1 -- 172.21.15.16:0/2981223625 wait complete. 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 Processor -- start 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 -- start start 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2330101de0 0x7f2330198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330102790 0x7f23301988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2330106f10 0x7f233019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2330111f70 con 0x7f2330102790 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2330111df0 con 0x7f2330106f10 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2338132640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23301120f0 con 0x7f2330101de0 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330102790 0x7f23301988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330102790 0x7f23301988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38836/0 (socket says 172.21.15.16:38836) 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 -- 172.21.15.16:0/2416123995 learned_addr learned my addr 172.21.15.16:0/2416123995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23366cf640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2330106f10 0x7f233019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f2335ece640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2330101de0 0x7f2330198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 -- 172.21.15.16:0/2416123995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2330101de0 msgr2=0x7f2330198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2330101de0 0x7f2330198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 -- 172.21.15.16:0/2416123995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2330106f10 msgr2=0x7f233019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2330106f10 0x7f233019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23356cd640 1 -- 172.21.15.16:0/2416123995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233019d320 con 0x7f2330102790 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.699+0000 7f23366cf640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2330106f10 0x7f233019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.700+0000 7f23356cd640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330102790 0x7f23301988b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f232000c920 tx=0x7f232000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.700+0000 7f2326ffd640 1 -- 172.21.15.16:0/2416123995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320007c70 con 0x7f2330102790 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.700+0000 7f2326ffd640 1 -- 172.21.15.16:0/2416123995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2320007e10 con 0x7f2330102790 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.700+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233019d610 con 0x7f2330102790 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.700+0000 7f2326ffd640 1 -- 172.21.15.16:0/2416123995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320018890 con 0x7f2330102790 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.700+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23301a4ef0 con 0x7f2330102790 2024-09-15T22:44:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f2326ffd640 1 -- 172.21.15.16:0/2416123995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2320018a70 con 0x7f2330102790 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 Processor -- start 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 -- start start 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e077090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e0775d0 0x55c94e077a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e088b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c94dfdd2f0 con 0x55c94e086780 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c94df84cf0 con 0x55c94e078b40 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.703+0000 7f233a50b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c94dfd92b0 con 0x55c94e0775d0 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e088b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e088b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38838/0 (socket says 172.21.15.16:38838) 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 -- 172.21.15.16:0/371857146 learned_addr learned my addr 172.21.15.16:0/371857146 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23366cf640 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f2335ece640 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e0775d0 0x55c94e077a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f2326ffd640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f231c0a33b0 0x7f231c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 -- 172.21.15.16:0/371857146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e0775d0 msgr2=0x55c94e077a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e0775d0 0x55c94e077a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 -- 172.21.15.16:0/371857146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 msgr2=0x55c94e077090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e077090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 -- 172.21.15.16:0/371857146 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c94df693a0 con 0x55c94e086780 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23366cf640 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e077090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23366cf640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f231c0a33b0 0x7f231c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f2326ffd640 1 -- 172.21.15.16:0/2416123995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f23200d0280 con 0x7f2330102790 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.704+0000 7f23356cd640 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e088b70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f23200cd5e0 tx=0x7f2320020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d7038780f8b81e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f231bfff640 1 -- 172.21.15.16:0/371857146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232009bb60 con 0x55c94e086780 2024-09-15T22:44:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f231bfff640 1 -- 172.21.15.16:0/371857146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232000de20 con 0x55c94e086780 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f231bfff640 1 -- 172.21.15.16:0/371857146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232009c200 con 0x55c94e086780 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f233a50b740 1 -- 172.21.15.16:0/371857146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 msgr2=0x55c94e088b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f23366cf640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f231c0a33b0 0x7f231c0a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2328005fd0 tx=0x7f2328004a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f233a50b740 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e088b70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f23200cd5e0 tx=0x7f2320020000 comp rx=0 tx=0).stop 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f233a50b740 1 -- 172.21.15.16:0/371857146 shutdown_connections 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f233a50b740 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e088b70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f233a50b740 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e0775d0 0x55c94e077a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f233a50b740 1 --2- 172.21.15.16:0/371857146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e077090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.705+0000 7f233a50b740 1 -- 172.21.15.16:0/371857146 >> 172.21.15.16:0/371857146 conn(0x55c94e040310 msgr2=0x55c94e073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 -- 172.21.15.16:0/371857146 shutdown_connections 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 -- 172.21.15.16:0/371857146 wait complete. 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 -- start start 2024-09-15T22:44:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 Processor -- start 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e0775d0 0x55c94e086140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e0833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e0839e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c94e089420 con 0x55c94e086780 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c94df98060 con 0x55c94e0775d0 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c94e0895a0 con 0x55c94e078b40 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f233a50b740 10 client.0 _ll_register_callbacks cb 0x55c94dceb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.706+0000 7f23366cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e0839e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e0839e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38844/0 (socket says 172.21.15.16:38844) 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 -- 172.21.15.16:0/220428182 learned_addr learned my addr 172.21.15.16:0/220428182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f2335ece640 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e0775d0 0x55c94e086140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23356cd640 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e0833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 -- 172.21.15.16:0/220428182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 msgr2=0x55c94e0833f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e0833f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 -- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e0775d0 msgr2=0x55c94e086140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e0775d0 0x55c94e086140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 -- 172.21.15.16:0/220428182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c94dd4e3c0 con 0x55c94e086780 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23366cf640 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e0839e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2328010920 tx=0x7f2328004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f407731a5a17c93 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23197fa640 1 -- 172.21.15.16:0/220428182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2328004b20 con 0x55c94e086780 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f232000de20 con 0x55c94e086780 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23197fa640 1 -- 172.21.15.16:0/220428182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f2328018070 con 0x55c94e086780 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.707+0000 7f23197fa640 1 -- 172.21.15.16:0/220428182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2328031050 con 0x55c94e086780 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 10 client.16716 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c94e094e80 con 0x55c94e086780 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f23197fa640 1 -- 172.21.15.16:0/220428182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232000de20 con 0x55c94e086780 2024-09-15T22:44:13.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f23197fa640 1 -- 172.21.15.16:0/220428182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f232800f040 con 0x55c94e086780 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 10 client.16716 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 10 client.16716 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 20 client.16716 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e096340 0x55c94e098770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 4 client.16716 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c94dfea860 con 0x55c94e096340 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.708+0000 7f2335ece640 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e096340 0x55c94e098770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.709+0000 7f2335ece640 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e096340 0x55c94e098770 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c03d9fa31a8a0b6 server_cookie=600ae358557d6a32 in_seq=0 out_seq=0 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.709+0000 7f23197fa640 10 client.16716 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.709+0000 7f23197fa640 1 -- 172.21.15.16:0/220428182 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c94dfea860 con 0x55c94e096340 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.709+0000 7f23197fa640 10 client.16716 handle_command_reply: tid=0 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 client.16716 shutdown 2024-09-15T22:44:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 msgr2=0x55c94e0839e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e0839e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f2328010920 tx=0x7f2328004ab0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 shutdown_connections 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e096340 0x55c94e098770 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e086780 0x55c94e0839e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e0833f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f23197fa640 0 client.16716 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 --2- 172.21.15.16:0/220428182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e0775d0 0x55c94e086140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 >> 172.21.15.16:0/220428182 conn(0x55c94e040310 msgr2=0x55c94e079960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 shutdown_connections 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 1 -- 172.21.15.16:0/220428182 wait complete. 2024-09-15T22:44:13.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.710+0000 7f233a50b740 20 client.16716 trim_cache size 0 max 16384 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 Processor -- start 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 -- start start 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e0926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e086780 0x55c94e09e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23301af5a0 con 0x55c94e0775d0 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23301ace70 con 0x55c94e078b40 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f233a50b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23301aacc0 con 0x55c94e086780 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f2335ece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e0926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f2335ece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e0926c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38850/0 (socket says 172.21.15.16:38850) 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.723+0000 7f2335ece640 1 -- 172.21.15.16:0/1668641428 learned_addr learned my addr 172.21.15.16:0/1668641428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f2335ece640 1 -- 172.21.15.16:0/1668641428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e086780 msgr2=0x55c94e09e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f23356cd640 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f2335ece640 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e086780 0x55c94e09e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f2335ece640 1 -- 172.21.15.16:0/1668641428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 msgr2=0x55c94e072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f2335ece640 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e072bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f2335ece640 1 -- 172.21.15.16:0/1668641428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c94e081980 con 0x55c94e0775d0 2024-09-15T22:44:13.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f2335ece640 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e0926c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f232c009900 tx=0x7f232c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4780778de85188fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f231b7fe640 1 -- 172.21.15.16:0/1668641428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c035040 con 0x55c94e0775d0 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f231b7fe640 1 -- 172.21.15.16:0/1668641428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232c04a070 con 0x55c94e0775d0 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.724+0000 7f231b7fe640 1 -- 172.21.15.16:0/1668641428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c004920 con 0x55c94e0775d0 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 -- 172.21.15.16:0/1668641428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 msgr2=0x55c94e0926c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e0926c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f232c009900 tx=0x7f232c005d50 comp rx=0 tx=0).stop 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 -- 172.21.15.16:0/1668641428 shutdown_connections 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e086780 0x55c94e09e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e078b40 0x55c94e072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 --2- 172.21.15.16:0/1668641428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e0926c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 -- 172.21.15.16:0/1668641428 >> 172.21.15.16:0/1668641428 conn(0x55c94e040310 msgr2=0x55c94e0724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 -- 172.21.15.16:0/1668641428 shutdown_connections 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 -- 172.21.15.16:0/1668641428 wait complete. 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 -- start start 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.725+0000 7f233a50b740 1 Processor -- start 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f233a50b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e08ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f233a50b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e08ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f233a50b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e086780 0x55c94e08b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f233a50b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23301ab330 con 0x55c94e0775d0 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f233a50b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23301ac7a0 con 0x55c94e086780 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f233a50b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23301aba00 con 0x55c94e078b40 2024-09-15T22:44:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f233a50b740 10 client.0 _ll_register_callbacks cb 0x7f2328009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e08ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f23356cd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e08ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e08ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38858/0 (socket says 172.21.15.16:38858) 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 -- 172.21.15.16:0/2691754432 learned_addr learned my addr 172.21.15.16:0/2691754432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f23366cf640 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e086780 0x55c94e08b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 -- 172.21.15.16:0/2691754432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 msgr2=0x55c94e08ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e08ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 -- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e086780 msgr2=0x55c94e08b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e086780 0x55c94e08b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 -- 172.21.15.16:0/2691754432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c94e08bf10 con 0x55c94e0775d0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f2335ece640 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e08ea10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f232c02a930 tx=0x7f232c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df35f6dca2efe855 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.726+0000 7f23197fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f23197fa640 1 -- 172.21.15.16:0/2691754432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c035070 con 0x55c94e0775d0 2024-09-15T22:44:13.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f23197fa640 1 -- 172.21.15.16:0/2691754432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f232c004250 con 0x55c94e0775d0 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f232c04a070 con 0x55c94e0775d0 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f23197fa640 1 -- 172.21.15.16:0/2691754432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f232c05c050 con 0x55c94e0775d0 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 10 client.16722 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c94e101a70 con 0x55c94e0775d0 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f23197fa640 1 -- 172.21.15.16:0/2691754432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232c04a070 con 0x55c94e0775d0 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f23197fa640 1 -- 172.21.15.16:0/2691754432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f232c031860 con 0x55c94e0775d0 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 10 client.16722 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 10 client.16722 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 20 client.16722 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e102150 0x55c94e104570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 4 client.16722 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c94dfe4d70 con 0x55c94e102150 2024-09-15T22:44:13.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.727+0000 7f23356cd640 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e102150 0x55c94e104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f23356cd640 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e102150 0x55c94e104570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fd519799fa3e21b server_cookie=b83f0177fff99ede in_seq=0 out_seq=0 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f23197fa640 10 client.16722 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f23197fa640 1 -- 172.21.15.16:0/2691754432 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c94dfe4d70 con 0x55c94e102150 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f23197fa640 10 client.16722 handle_command_reply: tid=0 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 client.16722 shutdown 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 msgr2=0x55c94e08ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e08ea10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f232c02a930 tx=0x7f232c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 shutdown_connections 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55c94e102150 0x55c94e104570 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c94e086780 0x55c94e08b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c94e078b40 0x55c94e08ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f23197fa640 0 client.16722 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 --2- 172.21.15.16:0/2691754432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c94e0775d0 0x55c94e08ea10 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.728+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 >> 172.21.15.16:0/2691754432 conn(0x55c94e040310 msgr2=0x55c94e085470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 shutdown_connections 2024-09-15T22:44:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f233a50b740 1 -- 172.21.15.16:0/2691754432 wait complete. 2024-09-15T22:44:13.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f233a50b740 20 client.16722 trim_cache size 0 max 16384 2024-09-15T22:44:13.735 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f231c0a33b0 msgr2=0x7f231c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f231c0a33b0 0x7f231c0a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2328005fd0 tx=0x7f2328004a40 comp rx=0 tx=0).stop 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330102790 msgr2=0x7f23301988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330102790 0x7f23301988b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f232000c920 tx=0x7f232000cde0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 shutdown_connections 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2330106f10 0x7f233019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2330102790 0x7f23301988b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f231c0a33b0 0x7f231c0a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 --2- 172.21.15.16:0/2416123995 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2330101de0 0x7f2330198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 >> 172.21.15.16:0/2416123995 conn(0x7f23300fd8c0 msgr2=0x7f23300fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 shutdown_connections 2024-09-15T22:44:13.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:13.729+0000 7f2338132640 1 -- 172.21.15.16:0/2416123995 wait complete. 2024-09-15T22:44:14.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:44:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:15.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:14 smithi130 ceph-mon[35742]: pgmap v177: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:44:15.047 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:14 smithi016 ceph-mon[31492]: pgmap v177: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:44:15.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:14 smithi059 ceph-mon[34872]: pgmap v177: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:44:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 -- 172.21.15.16:0/3326875812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894101de0 msgr2=0x7f98941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 --2- 172.21.15.16:0/3326875812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894101de0 0x7f98941021c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9884009900 tx=0x7f988402a410 comp rx=0 tx=0).stop 2024-09-15T22:44:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 -- 172.21.15.16:0/3326875812 shutdown_connections 2024-09-15T22:44:15.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 --2- 172.21.15.16:0/3326875812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9894106f10 0x7f989410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 --2- 172.21.15.16:0/3326875812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9894102790 0x7f98941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 --2- 172.21.15.16:0/3326875812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894101de0 0x7f98941021c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 -- 172.21.15.16:0/3326875812 >> 172.21.15.16:0/3326875812 conn(0x7f98940fd8c0 msgr2=0x7f98940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 -- 172.21.15.16:0/3326875812 shutdown_connections 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 -- 172.21.15.16:0/3326875812 wait complete. 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.425+0000 7f98994a9640 1 Processor -- start 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98994a9640 1 -- start start 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98994a9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9894101de0 0x7f98940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98994a9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9894102790 0x7f9894074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98994a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894106f10 0x7f9894074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98994a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9894111f30 con 0x7f9894106f10 2024-09-15T22:44:15.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98994a9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9894111db0 con 0x7f9894101de0 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98994a9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f98941120b0 con 0x7f9894102790 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894106f10 0x7f9894074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894106f10 0x7f9894074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38870/0 (socket says 172.21.15.16:38870) 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 -- 172.21.15.16:0/3495379793 learned_addr learned my addr 172.21.15.16:0/3495379793 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f9892ffd640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9894101de0 0x7f98940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f988a7fc640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9894102790 0x7f9894074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 -- 172.21.15.16:0/3495379793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9894102790 msgr2=0x7f9894074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9894102790 0x7f9894074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 -- 172.21.15.16:0/3495379793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9894101de0 msgr2=0x7f98940744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9894101de0 0x7f98940744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.426+0000 7f98937fe640 1 -- 172.21.15.16:0/3495379793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98941a6980 con 0x7f9894106f10 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.427+0000 7f98937fe640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894106f10 0x7f9894074f70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f988c00eaa0 tx=0x7f988c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.427+0000 7f9890ff9640 1 -- 172.21.15.16:0/3495379793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c00cdb0 con 0x7f9894106f10 2024-09-15T22:44:15.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.427+0000 7f9890ff9640 1 -- 172.21.15.16:0/3495379793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c005020 con 0x7f9894106f10 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.427+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98941a6c70 con 0x7f9894106f10 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.427+0000 7f9890ff9640 1 -- 172.21.15.16:0/3495379793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c0116b0 con 0x7f9894106f10 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.427+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941a7160 con 0x7f9894106f10 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.429+0000 7f9890ff9640 1 -- 172.21.15.16:0/3495379793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f988c005560 con 0x7f9894106f10 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.429+0000 7f989b882740 1 Processor -- start 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.429+0000 7f989b882740 1 -- start start 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f989b882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f989b882740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfaca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f989b882740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfb9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f989b882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b36df122f0 con 0x55b36dfadb40 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f989b882740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b36deb9cf0 con 0x55b36dfac5d0 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f989b882740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b36df0e2b0 con 0x55b36dfb7370 2024-09-15T22:44:15.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38876/0 (socket says 172.21.15.16:38876) 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 -- 172.21.15.16:0/536352523 learned_addr learned my addr 172.21.15.16:0/536352523 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f9890ff9640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f98800a33b0 0x7f98800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f9892ffd640 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfaca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfb9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 -- 172.21.15.16:0/536352523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 msgr2=0x55b36dfb9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfb9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 -- 172.21.15.16:0/536352523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 msgr2=0x55b36dfaca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfaca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f98937fe640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f98800a33b0 0x7f98800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f988a7fc640 1 -- 172.21.15.16:0/536352523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36de9e3a0 con 0x55b36dfadb40 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f9892ffd640 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfaca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:15.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.430+0000 7f9890ff9640 1 -- 172.21.15.16:0/3495379793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f988c0d0290 con 0x7f9894106f10 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.431+0000 7f988a7fc640 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfac090 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f987c0058c0 tx=0x7f987c00b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce147ba00d320020 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.431+0000 7f98897fa640 1 -- 172.21.15.16:0/536352523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c019030 con 0x55b36dfadb40 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.431+0000 7f98937fe640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f98800a33b0 0x7f98800a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f988c09bf10 tx=0x7f988c0054d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.431+0000 7f98897fa640 1 -- 172.21.15.16:0/536352523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987c0119a0 con 0x55b36dfadb40 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.431+0000 7f98897fa640 1 -- 172.21.15.16:0/536352523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c01d7e0 con 0x55b36dfadb40 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 -- 172.21.15.16:0/536352523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 msgr2=0x55b36dfac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfac090 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f987c0058c0 tx=0x7f987c00b6b0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 -- 172.21.15.16:0/536352523 shutdown_connections 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfb9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfaca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 --2- 172.21.15.16:0/536352523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfac090 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 -- 172.21.15.16:0/536352523 >> 172.21.15.16:0/536352523 conn(0x55b36df75310 msgr2=0x55b36dfa8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:15.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 -- 172.21.15.16:0/536352523 shutdown_connections 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 -- 172.21.15.16:0/536352523 wait complete. 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 -- start start 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.432+0000 7f989b882740 1 Processor -- start 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f989b882740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfb6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f989b882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfb3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f989b882740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfb45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f989b882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b36dfba070 con 0x55b36dfadb40 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f989b882740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b36decd060 con 0x55b36dfb7370 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f989b882740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b36dfba1f0 con 0x55b36dfac5d0 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f989b882740 10 client.0 _ll_register_callbacks cb 0x55b36dc208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfb3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfb3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38878/0 (socket says 172.21.15.16:38878) 2024-09-15T22:44:15.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 -- 172.21.15.16:0/372604221 learned_addr learned my addr 172.21.15.16:0/372604221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f988a7fc640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfb6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f988a7fc640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfb45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 -- 172.21.15.16:0/372604221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 msgr2=0x55b36dfb6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfb6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 -- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 msgr2=0x55b36dfb45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfb45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 -- 172.21.15.16:0/372604221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36dc833c0 con 0x55b36dfadb40 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f988a7fc640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfb6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f9892ffd640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfb3fe0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f988402a930 tx=0x7f9884031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc34f44e0080ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.433+0000 7f987a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f987a7fc640 1 -- 172.21.15.16:0/372604221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884035040 con 0x55b36dfadb40 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f987a7fc640 1 -- 172.21.15.16:0/372604221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f9884004240 con 0x55b36dfadb40 2024-09-15T22:44:15.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f987c0119a0 con 0x55b36dfadb40 2024-09-15T22:44:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f987a7fc640 1 -- 172.21.15.16:0/372604221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f988404e050 con 0x55b36dfadb40 2024-09-15T22:44:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 10 client.16734 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b36dfc5ad0 con 0x55b36dfadb40 2024-09-15T22:44:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f987a7fc640 1 -- 172.21.15.16:0/372604221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f987c0119a0 con 0x55b36dfadb40 2024-09-15T22:44:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f987a7fc640 1 -- 172.21.15.16:0/372604221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9884004b70 con 0x55b36dfadb40 2024-09-15T22:44:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 10 client.16734 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:15.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 10 client.16734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 20 client.16734 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36dfc93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 4 client.16734 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.434+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b36df1f860 con 0x55b36dfc6f90 2024-09-15T22:44:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.435+0000 7f988a7fc640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36dfc93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.435+0000 7f987a7fc640 10 client.16734 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:15.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.435+0000 7f988a7fc640 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36dfc93b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d119bf251e4496ab server_cookie=7e6bb4e0a8b335f6 in_seq=0 out_seq=0 2024-09-15T22:44:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f987a7fc640 1 -- 172.21.15.16:0/372604221 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b36df1f860 con 0x55b36dfc6f90 2024-09-15T22:44:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f987a7fc640 10 client.16734 handle_command_reply: tid=0 2024-09-15T22:44:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 client.16734 shutdown 2024-09-15T22:44:15.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 msgr2=0x55b36dfb3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfb3fe0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f988402a930 tx=0x7f9884031920 comp rx=0 tx=0).stop 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 shutdown_connections 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36dfc93b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f987a7fc640 0 client.16734 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfb45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfb3fe0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 --2- 172.21.15.16:0/372604221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfb6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 >> 172.21.15.16:0/372604221 conn(0x55b36df75310 msgr2=0x55b36dfae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 shutdown_connections 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 1 -- 172.21.15.16:0/372604221 wait complete. 2024-09-15T22:44:15.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.436+0000 7f989b882740 20 client.16734 trim_cache size 0 max 16384 2024-09-15T22:44:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.449+0000 7f989b882740 1 Processor -- start 2024-09-15T22:44:15.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.449+0000 7f989b882740 1 -- start start 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f989b882740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfc3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f989b882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfcec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f989b882740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfcf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f989b882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98941b3a80 con 0x55b36dfadb40 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f989b882740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98941b1350 con 0x55b36dfb7370 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f989b882740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f98941af1a0 con 0x55b36dfac5d0 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfcec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfcec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38892/0 (socket says 172.21.15.16:38892) 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 -- 172.21.15.16:0/1989219329 learned_addr learned my addr 172.21.15.16:0/1989219329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfc3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfcf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 -- 172.21.15.16:0/1989219329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 msgr2=0x55b36dfc3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfc3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 -- 172.21.15.16:0/1989219329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 msgr2=0x55b36dfcf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfcf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 -- 172.21.15.16:0/1989219329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36dfb4940 con 0x55b36dfadb40 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfcf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9892ffd640 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfcec80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9884009900 tx=0x7f988402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60d9711543f0b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9888ff9640 1 -- 172.21.15.16:0/1989219329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884035030 con 0x55b36dfadb40 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9888ff9640 1 -- 172.21.15.16:0/1989219329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9884031990 con 0x55b36dfadb40 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.450+0000 7f9888ff9640 1 -- 172.21.15.16:0/1989219329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98840054a0 con 0x55b36dfadb40 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 -- 172.21.15.16:0/1989219329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 msgr2=0x55b36dfcec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfcec80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9884009900 tx=0x7f988402a410 comp rx=0 tx=0).stop 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 -- 172.21.15.16:0/1989219329 shutdown_connections 2024-09-15T22:44:15.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfb7370 0x55b36dfcf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfcec80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 --2- 172.21.15.16:0/1989219329 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfac5d0 0x55b36dfc3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 -- 172.21.15.16:0/1989219329 >> 172.21.15.16:0/1989219329 conn(0x55b36df75310 msgr2=0x55b36dfb30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 -- 172.21.15.16:0/1989219329 shutdown_connections 2024-09-15T22:44:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.451+0000 7f989b882740 1 -- 172.21.15.16:0/1989219329 wait complete. 2024-09-15T22:44:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 -- start start 2024-09-15T22:44:15.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 Processor -- start 2024-09-15T22:44:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfbb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfbb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfbbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98941af810 con 0x55b36dfadb40 2024-09-15T22:44:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98941b0c80 con 0x55b36dfac5d0 2024-09-15T22:44:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f98941afee0 con 0x55b36dfb7370 2024-09-15T22:44:15.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f989b882740 10 client.0 _ll_register_callbacks cb 0x7f9884009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfbb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfbb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38894/0 (socket says 172.21.15.16:38894) 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 -- 172.21.15.16:0/1586597090 learned_addr learned my addr 172.21.15.16:0/1586597090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfbbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfbb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 -- 172.21.15.16:0/1586597090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 msgr2=0x55b36dfbbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfbbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 -- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 msgr2=0x55b36dfbb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfbb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f9892ffd640 1 -- 172.21.15.16:0/1586597090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36e030d00 con 0x55b36dfadb40 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.452+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfbbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f9892ffd640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfbb7c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f988402a930 tx=0x7f9884004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caca22060de406e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f987a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f987a7fc640 1 -- 172.21.15.16:0/1586597090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988405f060 con 0x55b36dfadb40 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f987a7fc640 1 -- 172.21.15.16:0/1586597090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f9884004950 con 0x55b36dfadb40 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9884031990 con 0x55b36dfadb40 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f987a7fc640 1 -- 172.21.15.16:0/1586597090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9884066050 con 0x55b36dfadb40 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 10 client.16740 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b36e031f50 con 0x55b36dfadb40 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f987a7fc640 1 -- 172.21.15.16:0/1586597090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9884031990 con 0x55b36dfadb40 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f987a7fc640 1 -- 172.21.15.16:0/1586597090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f988403c440 con 0x55b36dfadb40 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 10 client.16740 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 20 client.16740 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:15.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36e0345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.453+0000 7f989b882740 4 client.16740 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.454+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b36df19d70 con 0x55b36dfc6f90 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.454+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36e0345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.454+0000 7f988a7fc640 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36e0345f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452afe82feabe86a server_cookie=c4fbdf3054076e6b in_seq=0 out_seq=0 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.454+0000 7f987a7fc640 10 client.16740 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.454+0000 7f987a7fc640 1 -- 172.21.15.16:0/1586597090 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b36df19d70 con 0x55b36dfc6f90 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.454+0000 7f987a7fc640 10 client.16740 handle_command_reply: tid=0 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.454+0000 7f989b882740 1 client.16740 shutdown 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 msgr2=0x55b36dfbb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfbb7c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f988402a930 tx=0x7f9884004fa0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 shutdown_connections 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b36dfc6f90 0x55b36e0345f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f987a7fc640 0 client.16740 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:15.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b36dfb7370 0x55b36dfbbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b36dfadb40 0x55b36dfbb7c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 --2- 172.21.15.16:0/1586597090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b36dfac5d0 0x55b36dfbb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 >> 172.21.15.16:0/1586597090 conn(0x55b36df75310 msgr2=0x55b36dfa74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 shutdown_connections 2024-09-15T22:44:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 1 -- 172.21.15.16:0/1586597090 wait complete. 2024-09-15T22:44:15.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f989b882740 20 client.16740 trim_cache size 0 max 16384 2024-09-15T22:44:15.462 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:15.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f98800a33b0 msgr2=0x7f98800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f98800a33b0 0x7f98800a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f988c09bf10 tx=0x7f988c0054d0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894106f10 msgr2=0x7f9894074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894106f10 0x7f9894074f70 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f988c00eaa0 tx=0x7f988c00ef60 comp rx=0 tx=0).stop 2024-09-15T22:44:15.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 shutdown_connections 2024-09-15T22:44:15.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9894106f10 0x7f9894074f70 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f98800a33b0 0x7f98800a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9894102790 0x7f9894074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 --2- 172.21.15.16:0/3495379793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9894101de0 0x7f98940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.455+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 >> 172.21.15.16:0/3495379793 conn(0x7f98940fd8c0 msgr2=0x7f98940fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:15.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.456+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 shutdown_connections 2024-09-15T22:44:15.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:15.456+0000 7f98994a9640 1 -- 172.21.15.16:0/3495379793 wait complete. 2024-09-15T22:44:17.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:16 smithi130 ceph-mon[35742]: pgmap v178: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:44:17.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:16 smithi016 ceph-mon[31492]: pgmap v178: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:44:17.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:16 smithi059 ceph-mon[34872]: pgmap v178: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:44:17.247 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:44:17.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:44:17.518 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:18.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2285801043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 msgr2=0x7fe418107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2285801043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418107b90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe408009930 tx=0x7fe40802a420 comp rx=0 tx=0).stop 2024-09-15T22:44:18.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2285801043 shutdown_connections 2024-09-15T22:44:18.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2285801043 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4181082a0 0x7fe41810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2285801043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418107b90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2285801043 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe418102d30 0x7fe418103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2285801043 >> 172.21.15.16:0/2285801043 conn(0x7fe4180fc7d0 msgr2=0x7fe4180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2285801043 shutdown_connections 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.972+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2285801043 wait complete. 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 Processor -- start 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 -- start start 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe418102d30 0x7fe418198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4181082a0 0x7fe41819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181120c0 con 0x7fe4181036e0 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe418111f40 con 0x7fe418102d30 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41fdbf640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe418112240 con 0x7fe4181082a0 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38910/0 (socket says 172.21.15.16:38910) 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41d35a640 1 -- 172.21.15.16:0/2225063795 learned_addr learned my addr 172.21.15.16:0/2225063795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41db5b640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe418102d30 0x7fe418198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.973+0000 7fe41e35c640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4181082a0 0x7fe41819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41d35a640 1 -- 172.21.15.16:0/2225063795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4181082a0 msgr2=0x7fe41819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41d35a640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4181082a0 0x7fe41819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41d35a640 1 -- 172.21.15.16:0/2225063795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe418102d30 msgr2=0x7fe418198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41d35a640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe418102d30 0x7fe418198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41d35a640 1 -- 172.21.15.16:0/2225063795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41819d510 con 0x7fe4181036e0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41db5b640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe418102d30 0x7fe418198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41d35a640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418198aa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe408005940 tx=0x7fe4080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe40effd640 1 -- 172.21.15.16:0/2225063795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40803f030 con 0x7fe4181036e0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe40effd640 1 -- 172.21.15.16:0/2225063795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe408003ce0 con 0x7fe4181036e0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41819d7a0 con 0x7fe4181036e0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe40effd640 1 -- 172.21.15.16:0/2225063795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408035070 con 0x7fe4181036e0 2024-09-15T22:44:18.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.974+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4181a50e0 con 0x7fe4181036e0 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.976+0000 7fe40effd640 1 -- 172.21.15.16:0/2225063795 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe408042030 con 0x7fe4181036e0 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.976+0000 7fe422198740 1 Processor -- start 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe422198740 1 -- start start 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe422198740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef2a40 0x55c44fef2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe422198740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef3360 0x55c44ff01c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe422198740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff04540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe422198740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c44fe58280 con 0x55c44ff02150 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe422198740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c44fdffb10 con 0x55c44fef2a40 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe422198740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c44fe54240 con 0x55c44fef3360 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff04540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff04540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38920/0 (socket says 172.21.15.16:38920) 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe41d35a640 1 -- 172.21.15.16:0/2274534303 learned_addr learned my addr 172.21.15.16:0/2274534303 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:18.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe41db5b640 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef3360 0x55c44ff01c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe40effd640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4040a33b0 0x7fe4040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe41e35c640 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef2a40 0x55c44fef2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.977+0000 7fe41e35c640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4040a33b0 0x7fe4040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe41db5b640 1 -- 172.21.15.16:0/2274534303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef2a40 msgr2=0x55c44fef2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe40effd640 1 -- 172.21.15.16:0/2225063795 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe4080f06d0 con 0x7fe4181036e0 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe41db5b640 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef2a40 0x55c44fef2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe41db5b640 1 -- 172.21.15.16:0/2274534303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 msgr2=0x55c44ff04540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe41db5b640 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff04540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe41db5b640 1 -- 172.21.15.16:0/2274534303 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44fe0c1f0 con 0x55c44fef3360 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe41db5b640 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef3360 0x55c44ff01c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe414008b80 tx=0x7fe41400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d13667bf50ecbf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe41e35c640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4040a33b0 0x7fe4040a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe4100070b0 tx=0x7fe4100093d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe403fff640 1 -- 172.21.15.16:0/2274534303 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414014070 con 0x55c44fef3360 2024-09-15T22:44:18.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe403fff640 1 -- 172.21.15.16:0/2274534303 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4140047a0 con 0x55c44fef3360 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.978+0000 7fe403fff640 1 -- 172.21.15.16:0/2274534303 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414010650 con 0x55c44fef3360 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 -- 172.21.15.16:0/2274534303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef3360 msgr2=0x55c44ff01c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef3360 0x55c44ff01c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe414008b80 tx=0x7fe41400e720 comp rx=0 tx=0).stop 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 -- 172.21.15.16:0/2274534303 shutdown_connections 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff04540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef3360 0x55c44ff01c10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 --2- 172.21.15.16:0/2274534303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef2a40 0x55c44fef2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 -- 172.21.15.16:0/2274534303 >> 172.21.15.16:0/2274534303 conn(0x55c44feef0f0 msgr2=0x55c44feef4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:18.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 -- 172.21.15.16:0/2274534303 shutdown_connections 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 -- 172.21.15.16:0/2274534303 wait complete. 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.979+0000 7fe422198740 1 -- start start 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 1 Processor -- start 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44feff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44feff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44feffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c44ff04d20 con 0x55c44fef3360 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c44fe12f10 con 0x55c44ff02150 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c44ff04ea0 con 0x55c44fef2a40 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe422198740 10 client.0 _ll_register_callbacks cb 0x55c44fddba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44feff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44feff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38924/0 (socket says 172.21.15.16:38924) 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 -- 172.21.15.16:0/36540038 learned_addr learned my addr 172.21.15.16:0/36540038 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41db5b640 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44feff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41e35c640 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44feffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 -- 172.21.15.16:0/36540038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 msgr2=0x55c44feff0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44feff0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 -- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 msgr2=0x55c44feffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44feffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.980+0000 7fe41d35a640 1 -- 172.21.15.16:0/36540038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44fbc96a0 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe41d35a640 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44feff5f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe4080edb70 tx=0x7fe4080b8260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d0e34e68b43dca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe4017fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe4017fa640 1 -- 172.21.15.16:0/36540038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408049040 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe4017fa640 1 -- 172.21.15.16:0/36540038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe4080b8cc0 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4140047a0 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe4017fa640 1 -- 172.21.15.16:0/36540038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4080bc420 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 10 client.16752 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c44ff109e0 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe4017fa640 1 -- 172.21.15.16:0/36540038 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4140047a0 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe4017fa640 1 -- 172.21.15.16:0/36540038 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe4080b83a0 con 0x55c44fef3360 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 10 client.16752 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:18.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 20 client.16752 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff11df0 0x55c44ff14210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 4 client.16752 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.981+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c44fe657f0 con 0x55c44ff11df0 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.982+0000 7fe41db5b640 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff11df0 0x55c44ff14210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.982+0000 7fe41db5b640 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff11df0 0x55c44ff14210 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b471a5f8d0cb2b42 server_cookie=3732616dbc6c2fb7 in_seq=0 out_seq=0 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.982+0000 7fe4017fa640 10 client.16752 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe4017fa640 1 -- 172.21.15.16:0/36540038 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c44fe657f0 con 0x55c44ff11df0 2024-09-15T22:44:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe4017fa640 10 client.16752 handle_command_reply: tid=0 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 client.16752 shutdown 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 msgr2=0x55c44feff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44feff5f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe4080edb70 tx=0x7fe4080b8260 comp rx=0 tx=0).stop 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 shutdown_connections 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff11df0 0x55c44ff14210 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44feffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44feff5f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 --2- 172.21.15.16:0/36540038 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44feff0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe4017fa640 0 client.16752 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:18.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 >> 172.21.15.16:0/36540038 conn(0x55c44feef0f0 msgr2=0x55c44fef4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 shutdown_connections 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 1 -- 172.21.15.16:0/36540038 wait complete. 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:17.983+0000 7fe422198740 20 client.16752 trim_cache size 0 max 16384 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.000+0000 7fe422198740 1 Processor -- start 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.000+0000 7fe422198740 1 -- start start 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe422198740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff00ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe422198740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef3360 0x55c44ff19b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe422198740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff1a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe422198740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181af540 con 0x55c44ff02150 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe422198740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181ace10 con 0x55c44fef3360 2024-09-15T22:44:18.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe422198740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181aac60 con 0x55c44fef2a40 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff1a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff1a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38932/0 (socket says 172.21.15.16:38932) 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 -- 172.21.15.16:0/3071767990 learned_addr learned my addr 172.21.15.16:0/3071767990 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41db5b640 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff00ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41d35a640 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef3360 0x55c44ff19b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 -- 172.21.15.16:0/3071767990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 msgr2=0x55c44ff00ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff00ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 -- 172.21.15.16:0/3071767990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef3360 msgr2=0x55c44ff19b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef3360 0x55c44ff19b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 -- 172.21.15.16:0/3071767990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44fefe060 con 0x55c44ff02150 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41db5b640 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff00ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.001+0000 7fe41e35c640 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff1a060 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe41000aee0 tx=0x7fe410009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e90f144e15d6422 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe4037fe640 1 -- 172.21.15.16:0/3071767990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410013040 con 0x55c44ff02150 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe4037fe640 1 -- 172.21.15.16:0/3071767990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe410014c80 con 0x55c44ff02150 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe4037fe640 1 -- 172.21.15.16:0/3071767990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410018700 con 0x55c44ff02150 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 -- 172.21.15.16:0/3071767990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 msgr2=0x55c44ff1a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff1a060 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe41000aee0 tx=0x7fe410009700 comp rx=0 tx=0).stop 2024-09-15T22:44:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 -- 172.21.15.16:0/3071767990 shutdown_connections 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44ff02150 0x55c44ff1a060 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44fef3360 0x55c44ff19b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 --2- 172.21.15.16:0/3071767990 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff00ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 -- 172.21.15.16:0/3071767990 >> 172.21.15.16:0/3071767990 conn(0x55c44feef0f0 msgr2=0x55c44ff00790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 -- 172.21.15.16:0/3071767990 shutdown_connections 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.002+0000 7fe422198740 1 -- 172.21.15.16:0/3071767990 wait complete. 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 -- start start 2024-09-15T22:44:18.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 Processor -- start 2024-09-15T22:44:18.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff066c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44ff06c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44ff75ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181ab2d0 con 0x55c44fef3360 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181ac740 con 0x55c44ff02150 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe4181ab9a0 con 0x55c44fef2a40 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe422198740 10 client.0 _ll_register_callbacks cb 0x7fe408004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44ff06c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe41d35a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44ff06c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38936/0 (socket says 172.21.15.16:38936) 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe41e35c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44ff75ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.003+0000 7fe41d35a640 1 -- 172.21.15.16:0/1653386994 learned_addr learned my addr 172.21.15.16:0/1653386994 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41db5b640 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41e35c640 1 -- 172.21.15.16:0/1653386994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 msgr2=0x55c44ff066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41e35c640 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41e35c640 1 -- 172.21.15.16:0/1653386994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 msgr2=0x55c44ff06c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41e35c640 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44ff06c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41e35c640 1 -- 172.21.15.16:0/1653386994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44ff76100 con 0x55c44ff02150 2024-09-15T22:44:18.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41d35a640 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44ff06c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe41e35c640 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44ff75ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe410004a80 tx=0x7fe410009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7023dafc436e3dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe4017fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe4017fa640 1 -- 172.21.15.16:0/1653386994 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410032060 con 0x55c44ff02150 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe410014c80 con 0x55c44ff02150 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe4017fa640 1 -- 172.21.15.16:0/1653386994 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fe410025630 con 0x55c44ff02150 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe4017fa640 1 -- 172.21.15.16:0/1653386994 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe410013040 con 0x55c44ff02150 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe422198740 10 client.26174 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.004+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c44ff7d470 con 0x55c44ff02150 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe4017fa640 1 -- 172.21.15.16:0/1653386994 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe410014c80 con 0x55c44ff02150 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe4017fa640 1 -- 172.21.15.16:0/1653386994 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe41002d490 con 0x55c44ff02150 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe422198740 10 client.26174 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe422198740 10 client.26174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe422198740 20 client.26174 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe422198740 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff7daf0 0x55c44ff7fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe422198740 4 client.26174 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c44fe5fd00 con 0x55c44ff7daf0 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.005+0000 7fe41db5b640 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff7daf0 0x55c44ff7fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.006+0000 7fe4017fa640 10 client.26174 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.006+0000 7fe41db5b640 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff7daf0 0x55c44ff7fee0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a683c1130de5407c server_cookie=133928ba4e5d8ac8 in_seq=0 out_seq=0 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe4017fa640 1 -- 172.21.15.16:0/1653386994 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c44fe5fd00 con 0x55c44ff7daf0 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe4017fa640 10 client.26174 handle_command_reply: tid=0 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 client.26174 shutdown 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 msgr2=0x55c44ff75ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44ff75ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe410004a80 tx=0x7fe410009b80 comp rx=0 tx=0).stop 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 shutdown_connections 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c44ff7daf0 0x55c44ff7fee0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c44ff02150 0x55c44ff75ac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c44fef3360 0x55c44ff06c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe4017fa640 0 client.26174 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 --2- 172.21.15.16:0/1653386994 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c44fef2a40 0x55c44ff066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 >> 172.21.15.16:0/1653386994 conn(0x55c44feef0f0 msgr2=0x55c44fef4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 shutdown_connections 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 1 -- 172.21.15.16:0/1653386994 wait complete. 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.007+0000 7fe422198740 20 client.26174 trim_cache size 0 max 16384 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "79267bff-f525-467c-8d4f-ec72031b40ca", 2024-09-15T22:44:18.025 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:44:18.026 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:44:18.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4040a33b0 msgr2=0x7fe4040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4040a33b0 0x7fe4040a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe4100070b0 tx=0x7fe4100093d0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 msgr2=0x7fe418198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418198aa0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe408005940 tx=0x7fe4080042f0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 shutdown_connections 2024-09-15T22:44:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe4040a33b0 0x7fe4040a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe4181082a0 0x7fe41819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4181036e0 0x7fe418198aa0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 --2- 172.21.15.16:0/2225063795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe418102d30 0x7fe418198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 >> 172.21.15.16:0/2225063795 conn(0x7fe4180fc7d0 msgr2=0x7fe418105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 shutdown_connections 2024-09-15T22:44:18.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:18.008+0000 7fe41fdbf640 1 -- 172.21.15.16:0/2225063795 wait complete. 2024-09-15T22:44:18.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:44:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:18 smithi130 ceph-mon[35742]: pgmap v179: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:44:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:18 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:44:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:18 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:18 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:44:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:18 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:18 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:44:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:18 smithi016 ceph-mon[31492]: pgmap v179: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:44:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:18 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:44:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:18 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:18 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:44:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:18 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:19.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:18 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:44:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:19.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:18 smithi059 ceph-mon[34872]: pgmap v179: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:44:19.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:18 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:44:19.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:18 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:19.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:18 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:44:19.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:18 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:19.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:18 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 -- 172.21.15.16:0/1385653257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c108300 msgr2=0x7fdc3c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 --2- 172.21.15.16:0/1385653257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c108300 0x7fdc3c10ec70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009930 tx=0x7fdc2c02a420 comp rx=0 tx=0).stop 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 -- 172.21.15.16:0/1385653257 shutdown_connections 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 --2- 172.21.15.16:0/1385653257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c108300 0x7fdc3c10ec70 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 --2- 172.21.15.16:0/1385653257 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc3c103780 0x7fdc3c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 --2- 172.21.15.16:0/1385653257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc3c102dd0 0x7fdc3c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 -- 172.21.15.16:0/1385653257 >> 172.21.15.16:0/1385653257 conn(0x7fdc3c0fc7d0 msgr2=0x7fdc3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 -- 172.21.15.16:0/1385653257 shutdown_connections 2024-09-15T22:44:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 -- 172.21.15.16:0/1385653257 wait complete. 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.603+0000 7fdc413d5640 1 Processor -- start 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc413d5640 1 -- start start 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc413d5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc3c102dd0 0x7fdc3c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc413d5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c103780 0x7fdc3c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc413d5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc3c108300 0x7fdc3c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc413d5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1120d0 con 0x7fdc3c103780 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc413d5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c111f50 con 0x7fdc3c108300 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc413d5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c112250 con 0x7fdc3c102dd0 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c103780 0x7fdc3c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c103780 0x7fdc3c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38954/0 (socket says 172.21.15.16:38954) 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/322184744 learned_addr learned my addr 172.21.15.16:0/322184744 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3affd640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc3c102dd0 0x7fdc3c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc3c108300 0x7fdc3c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/322184744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc3c102dd0 msgr2=0x7fdc3c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc3c102dd0 0x7fdc3c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/322184744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc3c108300 msgr2=0x7fdc3c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc3c108300 0x7fdc3c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/322184744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3c19d540 con 0x7fdc3c103780 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.604+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c103780 0x7fdc3c198ad0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400d750 tx=0x7fdc2400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.605+0000 7fdc33fff640 1 -- 172.21.15.16:0/322184744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24017070 con 0x7fdc3c103780 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.605+0000 7fdc33fff640 1 -- 172.21.15.16:0/322184744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc24005500 con 0x7fdc3c103780 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.605+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc3c19d830 con 0x7fdc3c103780 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.605+0000 7fdc33fff640 1 -- 172.21.15.16:0/322184744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24004e00 con 0x7fdc3c103780 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.605+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc3c1a5110 con 0x7fdc3c103780 2024-09-15T22:44:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.607+0000 7fdc33fff640 1 -- 172.21.15.16:0/322184744 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdc24004fa0 con 0x7fdc3c103780 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.607+0000 7fdc437ae740 1 Processor -- start 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.607+0000 7fdc437ae740 1 -- start start 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc437ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd1092e0 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc437ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd0b0ce0 con 0x55d2cd1ae680 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc437ae740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd1052a0 con 0x55d2cd1a34a0 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc33fff640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdc280a3430 0x7fdc280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38956/0 (socket says 172.21.15.16:38956) 2024-09-15T22:44:19.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 -- 172.21.15.16:0/2996780487 learned_addr learned my addr 172.21.15.16:0/2996780487 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdc280a3430 0x7fdc280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc33fff640 1 -- 172.21.15.16:0/322184744 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fdc240cf920 con 0x7fdc3c103780 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 -- 172.21.15.16:0/2996780487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 msgr2=0x55d2cd1a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 -- 172.21.15.16:0/2996780487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1ae680 msgr2=0x55d2cd1b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3affd640 1 -- 172.21.15.16:0/2996780487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cd095440 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.608+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:19.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.609+0000 7fdc3affd640 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ae140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdc340039f0 tx=0x7fdc34003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e473b3b13e6940a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.609+0000 7fdc317fa640 1 -- 172.21.15.16:0/2996780487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34004b20 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.609+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdc280a3430 0x7fdc280a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c005940 tx=0x7fdc2c0058b0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.609+0000 7fdc317fa640 1 -- 172.21.15.16:0/2996780487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc34004cc0 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.609+0000 7fdc317fa640 1 -- 172.21.15.16:0/2996780487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34012740 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- 172.21.15.16:0/2996780487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 msgr2=0x55d2cd1ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ae140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdc340039f0 tx=0x7fdc34003eb0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- 172.21.15.16:0/2996780487 shutdown_connections 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ae140 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2996780487 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- 172.21.15.16:0/2996780487 >> 172.21.15.16:0/2996780487 conn(0x55d2cd19fa30 msgr2=0x55d2cd19fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- 172.21.15.16:0/2996780487 shutdown_connections 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- 172.21.15.16:0/2996780487 wait complete. 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- start start 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 Processor -- start 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd1b1250 con 0x55d2cd1ae680 2024-09-15T22:44:19.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd0c4050 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d2cd1b13d0 con 0x55d2cd1a34a0 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.610+0000 7fdc437ae740 10 client.0 _ll_register_callbacks cb 0x55d2cd104160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38968/0 (socket says 172.21.15.16:38968) 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/3857907674 learned_addr learned my addr 172.21.15.16:0/3857907674 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3affd640 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/3857907674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 msgr2=0x55d2cd1adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 msgr2=0x55d2cd1ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/3857907674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cce7a3c0 con 0x55d2cd1ae680 2024-09-15T22:44:19.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ab6f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c033000 tx=0x7fdc2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e3834cc7f67298 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc227fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc227fc640 1 -- 172.21.15.16:0/3857907674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c04d030 con 0x55d2cd1ae680 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc227fc640 1 -- 172.21.15.16:0/3857907674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fdc2c005020 con 0x55d2cd1ae680 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc34004cc0 con 0x55d2cd1ae680 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc227fc640 1 -- 172.21.15.16:0/3857907674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc2c058050 con 0x55d2cd1ae680 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc437ae740 10 client.16770 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.611+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2cd1c0f30 con 0x55d2cd1ae680 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc227fc640 1 -- 172.21.15.16:0/3857907674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc34004cc0 con 0x55d2cd1ae680 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc227fc640 1 -- 172.21.15.16:0/3857907674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdc2c03a040 con 0x55d2cd1ae680 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc437ae740 10 client.16770 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc437ae740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc437ae740 20 client.16770 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:19.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd1c2340 0x55d2cd1c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc437ae740 4 client.16770 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2cd116850 con 0x55d2cd1c2340 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.612+0000 7fdc3affd640 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd1c2340 0x55d2cd1c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.613+0000 7fdc227fc640 10 client.16770 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.613+0000 7fdc3affd640 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd1c2340 0x55d2cd1c47e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fba5ac78853fe9b server_cookie=3f049ebb149098a4 in_seq=0 out_seq=0 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.613+0000 7fdc227fc640 1 -- 172.21.15.16:0/3857907674 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2cd116850 con 0x55d2cd1c2340 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.613+0000 7fdc227fc640 10 client.16770 handle_command_reply: tid=0 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.613+0000 7fdc437ae740 1 client.16770 shutdown 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 msgr2=0x55d2cd1ab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ab6f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c033000 tx=0x7fdc2c009260 comp rx=0 tx=0).stop 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 shutdown_connections 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd1c2340 0x55d2cd1c47e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ab6f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3857907674 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc227fc640 0 client.16770 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 >> 172.21.15.16:0/3857907674 conn(0x55d2cd19fa30 msgr2=0x55d2cd19eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 shutdown_connections 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 1 -- 172.21.15.16:0/3857907674 wait complete. 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.614+0000 7fdc437ae740 20 client.16770 trim_cache size 0 max 16384 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 Processor -- start 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 -- start start 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1af5d0 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1acea0 con 0x55d2cd1a34a0 2024-09-15T22:44:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc437ae740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1aacf0 con 0x55d2cd1ae680 2024-09-15T22:44:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc3a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc3a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38984/0 (socket says 172.21.15.16:38984) 2024-09-15T22:44:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.631+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/3602149097 learned_addr learned my addr 172.21.15.16:0/3602149097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/3602149097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1ae680 msgr2=0x55d2cd1ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3affd640 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/3602149097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a34a0 msgr2=0x55d2cd1ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3a7fc640 1 -- 172.21.15.16:0/3602149097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cd1abf70 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3affd640 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ca100 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdc240048e0 tx=0x7fdc2406d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7339a703cfe2a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc30ff9640 1 -- 172.21.15.16:0/3602149097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24020040 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc30ff9640 1 -- 172.21.15.16:0/3602149097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2409b8a0 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.632+0000 7fdc30ff9640 1 -- 172.21.15.16:0/3602149097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc240ccf90 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 -- 172.21.15.16:0/3602149097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 msgr2=0x55d2cd1ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ca100 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdc240048e0 tx=0x7fdc2406d620 comp rx=0 tx=0).stop 2024-09-15T22:44:19.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 -- 172.21.15.16:0/3602149097 shutdown_connections 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1ae680 0x55d2cd1ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1ca100 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 --2- 172.21.15.16:0/3602149097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 -- 172.21.15.16:0/3602149097 >> 172.21.15.16:0/3602149097 conn(0x55d2cd19fa30 msgr2=0x55d2cd1aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 -- 172.21.15.16:0/3602149097 shutdown_connections 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 -- 172.21.15.16:0/3602149097 wait complete. 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 -- start start 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.633+0000 7fdc437ae740 1 Processor -- start 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1baca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1bb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1ab360 con 0x55d2cd1ae680 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1ac7d0 con 0x55d2cd1a3dc0 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3c1aba30 con 0x55d2cd1a34a0 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 10 client.0 _ll_register_callbacks cb 0x7fdc2c041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38994/0 (socket says 172.21.15.16:38994) 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/2026253567 learned_addr learned my addr 172.21.15.16:0/2026253567 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3a7fc640 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1bb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3affd640 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/2026253567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 msgr2=0x55d2cd1baca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1baca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 msgr2=0x55d2cd1bb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1bb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 -- 172.21.15.16:0/2026253567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2cd1b6130 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc3b7fe640 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b5a70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c02a460 tx=0x7fdc2c004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a1dd8f5d2c2e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc227fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc227fc640 1 -- 172.21.15.16:0/2026253567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c061060 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc227fc640 1 -- 172.21.15.16:0/2026253567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fdc2c0048c0 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.634+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc2409b8a0 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc227fc640 1 -- 172.21.15.16:0/2026253567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc2c068050 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 10 client.16776 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2cd22da70 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc227fc640 1 -- 172.21.15.16:0/2026253567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc2409b8a0 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc227fc640 1 -- 172.21.15.16:0/2026253567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdc2c003b00 con 0x55d2cd1ae680 2024-09-15T22:44:19.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 10 client.16776 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 20 client.16776 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd22e1f0 0x55d2cd230610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 4 client.16776 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2cd110d60 con 0x55d2cd22e1f0 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.635+0000 7fdc3affd640 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd22e1f0 0x55d2cd230610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc3affd640 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd22e1f0 0x55d2cd230610 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b1cefe00552dc6 server_cookie=9dee98853f06d4a2 in_seq=0 out_seq=0 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc227fc640 10 client.16776 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc227fc640 1 -- 172.21.15.16:0/2026253567 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d2cd110d60 con 0x55d2cd22e1f0 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc227fc640 10 client.16776 handle_command_reply: tid=0 2024-09-15T22:44:19.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 client.16776 shutdown 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 msgr2=0x55d2cd1b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b5a70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c02a460 tx=0x7fdc2c004500 comp rx=0 tx=0).stop 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 shutdown_connections 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d2cd22e1f0 0x55d2cd230610 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d2cd1ae680 0x55d2cd1b5a70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2cd1a3dc0 0x55d2cd1bb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc227fc640 0 client.16776 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 --2- 172.21.15.16:0/2026253567 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d2cd1a34a0 0x55d2cd1baca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 >> 172.21.15.16:0/2026253567 conn(0x55d2cd19fa30 msgr2=0x55d2cd1a4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 shutdown_connections 2024-09-15T22:44:19.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 1 -- 172.21.15.16:0/2026253567 wait complete. 2024-09-15T22:44:19.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.636+0000 7fdc437ae740 20 client.16776 trim_cache size 0 max 16384 2024-09-15T22:44:19.658 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:44:19.658 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:44:19.658 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:44:19.658 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdc280a3430 msgr2=0x7fdc280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdc280a3430 0x7fdc280a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c005940 tx=0x7fdc2c0058b0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c103780 msgr2=0x7fdc3c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c103780 0x7fdc3c198ad0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400d750 tx=0x7fdc2400dc10 comp rx=0 tx=0).stop 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 shutdown_connections 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc3c108300 0x7fdc3c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc3c103780 0x7fdc3c198ad0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdc280a3430 0x7fdc280a58f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 --2- 172.21.15.16:0/322184744 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdc3c102dd0 0x7fdc3c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 >> 172.21.15.16:0/322184744 conn(0x7fdc3c0fc7d0 msgr2=0x7fdc3c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 shutdown_connections 2024-09-15T22:44:19.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:19.637+0000 7fdc413d5640 1 -- 172.21.15.16:0/322184744 wait complete. 2024-09-15T22:44:20.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:44:20.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:44:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:21.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:20 smithi130 ceph-mon[35742]: pgmap v180: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:44:21.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:20 smithi016 ceph-mon[31492]: pgmap v180: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 -- 172.21.15.16:0/2362534745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1036e0 msgr2=0x7f346c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 --2- 172.21.15.16:0/2362534745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1036e0 0x7f346c107b90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3460009930 tx=0x7f346002a420 comp rx=0 tx=0).stop 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 -- 172.21.15.16:0/2362534745 shutdown_connections 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 --2- 172.21.15.16:0/2362534745 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f346c1082a0 0x7f346c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 --2- 172.21.15.16:0/2362534745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1036e0 0x7f346c107b90 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 --2- 172.21.15.16:0/2362534745 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346c102d30 0x7f346c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 -- 172.21.15.16:0/2362534745 >> 172.21.15.16:0/2362534745 conn(0x7f346c0fc7d0 msgr2=0x7f346c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 -- 172.21.15.16:0/2362534745 shutdown_connections 2024-09-15T22:44:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.145+0000 7f3471325640 1 -- 172.21.15.16:0/2362534745 wait complete. 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 Processor -- start 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 -- start start 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346c102d30 0x7f346c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f346c1036e0 0x7f346c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1082a0 0x7f346c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1120a0 con 0x7f346c1082a0 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f346c111f20 con 0x7f346c102d30 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.146+0000 7f3471325640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f346c112220 con 0x7f346c1036e0 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1082a0 0x7f346c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1082a0 0x7f346c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39004/0 (socket says 172.21.15.16:39004) 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 -- 172.21.15.16:0/3991325635 learned_addr learned my addr 172.21.15.16:0/3991325635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346affd640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346c102d30 0x7f346c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346a7fc640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f346c1036e0 0x7f346c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 -- 172.21.15.16:0/3991325635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f346c1036e0 msgr2=0x7f346c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f346c1036e0 0x7f346c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 -- 172.21.15.16:0/3991325635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346c102d30 msgr2=0x7f346c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346c102d30 0x7f346c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 -- 172.21.15.16:0/3991325635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c1a6b00 con 0x7f346c1082a0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346a7fc640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f346c1036e0 0x7f346c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f346b7fe640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1082a0 0x7f346c101410 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f346400eaa0 tx=0x7f346400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f3453fff640 1 -- 172.21.15.16:0/3991325635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400cdb0 con 0x7f346c1082a0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f3453fff640 1 -- 172.21.15.16:0/3991325635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3464005020 con 0x7f346c1082a0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c1a6df0 con 0x7f346c1082a0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f3453fff640 1 -- 172.21.15.16:0/3991325635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34640116b0 con 0x7f346c1082a0 2024-09-15T22:44:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.147+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a72b0 con 0x7f346c1082a0 2024-09-15T22:44:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.148+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346c103110 con 0x7f346c1082a0 2024-09-15T22:44:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.150+0000 7f3453fff640 1 -- 172.21.15.16:0/3991325635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3464005560 con 0x7f346c1082a0 2024-09-15T22:44:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.150+0000 7f3453fff640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f34540a33b0 0x7f34540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.150+0000 7f346affd640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f34540a33b0 0x7f34540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.150+0000 7f3453fff640 1 -- 172.21.15.16:0/3991325635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f34640d0590 con 0x7f346c1082a0 2024-09-15T22:44:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.151+0000 7f346affd640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f34540a33b0 0x7f34540a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f345c005d00 tx=0x7f345c00b410 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.154+0000 7f3453fff640 1 -- 172.21.15.16:0/3991325635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3464016030 con 0x7f346c1082a0 2024-09-15T22:44:21.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:20 smithi059 ceph-mon[34872]: pgmap v180: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:44:21.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.362+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f346c101ec0 con 0x7f346c1082a0 2024-09-15T22:44:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.362+0000 7f3453fff640 1 -- 172.21.15.16:0/3991325635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f346409cc10 con 0x7f346c1082a0 2024-09-15T22:44:21.365 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:44:21.365 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:44:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:44:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f34540a33b0 msgr2=0x7f34540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f34540a33b0 0x7f34540a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f345c005d00 tx=0x7f345c00b410 comp rx=0 tx=0).stop 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1082a0 msgr2=0x7f346c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1082a0 0x7f346c101410 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f346400eaa0 tx=0x7f346400ef60 comp rx=0 tx=0).stop 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 shutdown_connections 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f346c1082a0 0x7f346c101410 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f34540a33b0 0x7f34540a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f346c1036e0 0x7f346c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 --2- 172.21.15.16:0/3991325635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f346c102d30 0x7f346c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 >> 172.21.15.16:0/3991325635 conn(0x7f346c0fc7d0 msgr2=0x7f346c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 shutdown_connections 2024-09-15T22:44:21.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:21.365+0000 7f3471325640 1 -- 172.21.15.16:0/3991325635 wait complete. 2024-09-15T22:44:22.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:21 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3991325635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:22.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:21 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3991325635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:22.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:44:22.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:21 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3991325635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:22.363 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:22.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:22 smithi016 ceph-mon[31492]: pgmap v181: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 -- 172.21.15.16:0/1233119093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 msgr2=0x7fac6c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 --2- 172.21.15.16:0/1233119093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c1021c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009900 tx=0x7fac5c02a410 comp rx=0 tx=0).stop 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 -- 172.21.15.16:0/1233119093 shutdown_connections 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 --2- 172.21.15.16:0/1233119093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c106f10 0x7fac6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 --2- 172.21.15.16:0/1233119093 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac6c102790 0x7fac6c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 --2- 172.21.15.16:0/1233119093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c1021c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 -- 172.21.15.16:0/1233119093 >> 172.21.15.16:0/1233119093 conn(0x7fac6c0fd8c0 msgr2=0x7fac6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 -- 172.21.15.16:0/1233119093 shutdown_connections 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.889+0000 7fac72536640 1 -- 172.21.15.16:0/1233119093 wait complete. 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.890+0000 7fac72536640 1 Processor -- start 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.890+0000 7fac72536640 1 -- start start 2024-09-15T22:44:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.890+0000 7fac72536640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.890+0000 7fac72536640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c102790 0x7fac6c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.890+0000 7fac72536640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac6c106f10 0x7fac6c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.890+0000 7fac72536640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c111f30 con 0x7fac6c101de0 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.890+0000 7fac72536640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c111db0 con 0x7fac6c102790 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac72536640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1120b0 con 0x7fac6c106f10 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46544/0 (socket says 172.21.15.16:46544) 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 -- 172.21.15.16:0/128637488 learned_addr learned my addr 172.21.15.16:0/128637488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac70ad3640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac6c106f10 0x7fac6c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c102790 0x7fac6c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 -- 172.21.15.16:0/128637488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac6c106f10 msgr2=0x7fac6c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac6c106f10 0x7fac6c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 -- 172.21.15.16:0/128637488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c102790 msgr2=0x7fac6c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c102790 0x7fac6c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 -- 172.21.15.16:0/128637488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c19d3e0 con 0x7fac6c101de0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac70ad3640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac6c106f10 0x7fac6c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac6bfff640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c198430 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fac5c005ab0 tx=0x7fac5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac697fa640 1 -- 172.21.15.16:0/128637488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c03f030 con 0x7fac6c101de0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c19d670 con 0x7fac6c101de0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac697fa640 1 -- 172.21.15.16:0/128637488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac5c003ce0 con 0x7fac6c101de0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac697fa640 1 -- 172.21.15.16:0/128637488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c035070 con 0x7fac6c101de0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.891+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c1a4fb0 con 0x7fac6c101de0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.893+0000 7fac697fa640 1 -- 172.21.15.16:0/128637488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fac5c042030 con 0x7fac6c101de0 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 Processor -- start 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 -- start start 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef4b40 0x560e3bef3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bef3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bf02780 0x560e3bf04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e3be592f0 con 0x560e3bf02780 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560e3be00cf0 con 0x560e3bef35d0 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac7490f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560e3be552b0 con 0x560e3bef4b40 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac697fa640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac540a33b0 0x7fac540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bf02780 0x560e3bf04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bf02780 0x560e3bf04b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46550/0 (socket says 172.21.15.16:46550) 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2237143654 learned_addr learned my addr 172.21.15.16:0/2237143654 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bef3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac70ad3640 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef4b40 0x560e3bef3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.894+0000 7fac70ad3640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac540a33b0 0x7fac540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2237143654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef4b40 msgr2=0x560e3bef3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef4b40 0x560e3bef3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2237143654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 msgr2=0x560e3bef3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac697fa640 1 -- 172.21.15.16:0/128637488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fac5c0f0a40 con 0x7fac6c101de0 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bef3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2237143654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3bde53a0 con 0x560e3bf02780 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac70ad3640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac540a33b0 0x7fac540a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac6400afd0 tx=0x7fac64005c10 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bf02780 0x560e3bf04b70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fac6000bb10 tx=0x7fac6000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9738a779b44eea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac527fc640 1 -- 172.21.15.16:0/2237143654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60017420 con 0x560e3bf02780 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac527fc640 1 -- 172.21.15.16:0/2237143654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac60017aa0 con 0x560e3bf02780 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.895+0000 7fac527fc640 1 -- 172.21.15.16:0/2237143654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6000faf0 con 0x560e3bf02780 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 -- 172.21.15.16:0/2237143654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bf02780 msgr2=0x560e3bf04b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bf02780 0x560e3bf04b70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fac6000bb10 tx=0x7fac6000d340 comp rx=0 tx=0).stop 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 -- 172.21.15.16:0/2237143654 shutdown_connections 2024-09-15T22:44:22.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bf02780 0x560e3bf04b70 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bef3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 --2- 172.21.15.16:0/2237143654 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef4b40 0x560e3bef3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 -- 172.21.15.16:0/2237143654 >> 172.21.15.16:0/2237143654 conn(0x560e3bebc310 msgr2=0x560e3beefc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 -- 172.21.15.16:0/2237143654 shutdown_connections 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 -- 172.21.15.16:0/2237143654 wait complete. 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 -- start start 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.896+0000 7fac7490f740 1 Processor -- start 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac7490f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef35d0 0x560e3bf02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac7490f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3beff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac7490f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bf02780 0x560e3beff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac7490f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e3bf05480 con 0x560e3bef4b40 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac7490f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560e3be14060 con 0x560e3bf02780 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac7490f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x560e3bf05600 con 0x560e3bef35d0 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac7490f740 10 client.0 _ll_register_callbacks cb 0x560e3bb678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3beff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3beff370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46556/0 (socket says 172.21.15.16:46556) 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2454926221 learned_addr learned my addr 172.21.15.16:0/2454926221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef35d0 0x560e3bf02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef35d0 msgr2=0x560e3bf02070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef35d0 0x560e3bf02070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2454926221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bf02780 msgr2=0x560e3beff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac70ad3640 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bf02780 0x560e3beff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bf02780 0x560e3beff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 -- 172.21.15.16:0/2454926221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3bbca3c0 con 0x560e3bef4b40 2024-09-15T22:44:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3beff370 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fac60019040 tx=0x7fac6000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643e6a60ea7bd076 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.897+0000 7fac4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2454926221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60027020 con 0x560e3bef4b40 2024-09-15T22:44:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2454926221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fac60017830 con 0x560e3bef4b40 2024-09-15T22:44:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac60017aa0 con 0x560e3bef4b40 2024-09-15T22:44:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2454926221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac60032050 con 0x560e3bef4b40 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 10 client.16794 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e3bf10e70 con 0x560e3bef4b40 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2454926221 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac60017aa0 con 0x560e3bef4b40 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2454926221 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac60023810 con 0x560e3bef4b40 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 10 client.16794 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 10 client.16794 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 20 client.16794 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf12330 0x560e3bf14720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 4 client.16794 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.898+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e3be66860 con 0x560e3bf12330 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.899+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf12330 0x560e3bf14720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.899+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf12330 0x560e3bf14720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8c4abccecc75128 server_cookie=41b8144eb19b5433 in_seq=0 out_seq=0 2024-09-15T22:44:22.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.899+0000 7fac4b7fe640 10 client.16794 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.900+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2454926221 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e3be66860 con 0x560e3bf12330 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.900+0000 7fac4b7fe640 10 client.16794 handle_command_reply: tid=0 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.900+0000 7fac7490f740 1 client.16794 shutdown 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 msgr2=0x560e3beff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3beff370 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fac60019040 tx=0x7fac6000df80 comp rx=0 tx=0).stop 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 shutdown_connections 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf12330 0x560e3bf14720 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bf02780 0x560e3beff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3beff370 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 --2- 172.21.15.16:0/2454926221 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bef35d0 0x560e3bf02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 >> 172.21.15.16:0/2454926221 conn(0x560e3bebc310 msgr2=0x560e3bef5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac4b7fe640 0 client.16794 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 shutdown_connections 2024-09-15T22:44:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 1 -- 172.21.15.16:0/2454926221 wait complete. 2024-09-15T22:44:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.901+0000 7fac7490f740 20 client.16794 trim_cache size 0 max 16384 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 Processor -- start 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 -- start start 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bf0e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3befe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf19f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1af530 con 0x560e3bef4b40 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1ace00 con 0x560e3bef35d0 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.914+0000 7fac7490f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1aac50 con 0x560e3bf02780 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3befe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3befe350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46572/0 (socket says 172.21.15.16:46572) 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 -- 172.21.15.16:0/4068414420 learned_addr learned my addr 172.21.15.16:0/4068414420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6bfff640 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bf0e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac70ad3640 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf19f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 -- 172.21.15.16:0/4068414420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 msgr2=0x560e3bf19f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf19f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 -- 172.21.15.16:0/4068414420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 msgr2=0x560e3bf0e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bf0e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 -- 172.21.15.16:0/4068414420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e3beef4f0 con 0x560e3bef4b40 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6bfff640 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bf0e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3befe350 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fac60023e20 tx=0x7fac60005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa070dcbe24262a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac51ffb640 1 -- 172.21.15.16:0/4068414420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60027040 con 0x560e3bef4b40 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac51ffb640 1 -- 172.21.15.16:0/4068414420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6000b710 con 0x560e3bef4b40 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.915+0000 7fac51ffb640 1 -- 172.21.15.16:0/4068414420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60000e60 con 0x560e3bef4b40 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 -- 172.21.15.16:0/4068414420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 msgr2=0x560e3befe350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3befe350 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fac60023e20 tx=0x7fac60005780 comp rx=0 tx=0).stop 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 -- 172.21.15.16:0/4068414420 shutdown_connections 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf19f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef4b40 0x560e3befe350 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 --2- 172.21.15.16:0/4068414420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef35d0 0x560e3bf0e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 -- 172.21.15.16:0/4068414420 >> 172.21.15.16:0/4068414420 conn(0x560e3bebc310 msgr2=0x560e3befdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 -- 172.21.15.16:0/4068414420 shutdown_connections 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 -- 172.21.15.16:0/4068414420 wait complete. 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.916+0000 7fac7490f740 1 -- start start 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 1 Processor -- start 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef35d0 0x560e3bf0ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef4b40 0x560e3bf0b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf06200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1ab2c0 con 0x560e3bef35d0 2024-09-15T22:44:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1ac730 con 0x560e3bef4b40 2024-09-15T22:44:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c1ab990 con 0x560e3bf02780 2024-09-15T22:44:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac7490f740 10 client.0 _ll_register_callbacks cb 0x7fac5c0bc790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef35d0 0x560e3bf0ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef35d0 0x560e3bf0ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46574/0 (socket says 172.21.15.16:46574) 2024-09-15T22:44:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 -- 172.21.15.16:0/2819515903 learned_addr learned my addr 172.21.15.16:0/2819515903 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac70ad3640 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf06200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef4b40 0x560e3bf0b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 -- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 msgr2=0x560e3bf06200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf06200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 -- 172.21.15.16:0/2819515903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef4b40 msgr2=0x560e3bf0b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef4b40 0x560e3bf0b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.917+0000 7fac6bfff640 1 -- 172.21.15.16:0/2819515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e3bf06a30 con 0x560e3bef35d0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac6bfff640 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef35d0 0x560e3bf0ac80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0ede90 tx=0x7fac5c03b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ddfa1a552737c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2819515903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0bc320 con 0x560e3bef35d0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2819515903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fac5c0ee0b0 con 0x560e3bef35d0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e3be60d70 con 0x560e3bef35d0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2819515903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac5c049030 con 0x560e3bef35d0 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 10 client.16800 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:22.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e3bf7cb40 con 0x560e3bef35d0 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2819515903 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560e3be60d70 con 0x560e3bef35d0 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2819515903 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac5c0b9030 con 0x560e3bef35d0 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 10 client.16800 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 10 client.16800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 20 client.16800 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf7d210 0x560e3bf7f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 4 client.16800 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.918+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e3bddd170 con 0x560e3bf7d210 2024-09-15T22:44:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.919+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf7d210 0x560e3bf7f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.919+0000 7fac6b7fe640 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf7d210 0x560e3bf7f600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bae016e2b096a7e server_cookie=89e32b609adf4460 in_seq=0 out_seq=0 2024-09-15T22:44:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.919+0000 7fac4b7fe640 10 client.16800 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.919+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2819515903 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e3bddd170 con 0x560e3bf7d210 2024-09-15T22:44:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.919+0000 7fac4b7fe640 10 client.16800 handle_command_reply: tid=0 2024-09-15T22:44:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.919+0000 7fac7490f740 1 client.16800 shutdown 2024-09-15T22:44:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef35d0 msgr2=0x560e3bf0ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef35d0 0x560e3bf0ac80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0ede90 tx=0x7fac5c03b800 comp rx=0 tx=0).stop 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 shutdown_connections 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x560e3bf7d210 0x560e3bf7f600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560e3bf02780 0x560e3bf06200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560e3bef4b40 0x560e3bf0b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 --2- 172.21.15.16:0/2819515903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e3bef35d0 0x560e3bf0ac80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 >> 172.21.15.16:0/2819515903 conn(0x560e3bebc310 msgr2=0x560e3befe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac4b7fe640 0 client.16800 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 shutdown_connections 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 1 -- 172.21.15.16:0/2819515903 wait complete. 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac7490f740 20 client.16800 trim_cache size 0 max 16384 2024-09-15T22:44:22.929 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac540a33b0 msgr2=0x7fac540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac540a33b0 0x7fac540a5870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac6400afd0 tx=0x7fac64005c10 comp rx=0 tx=0).stop 2024-09-15T22:44:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 msgr2=0x7fac6c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c198430 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fac5c005ab0 tx=0x7fac5c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 shutdown_connections 2024-09-15T22:44:22.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fac540a33b0 0x7fac540a5870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fac6c106f10 0x7fac6c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac6c102790 0x7fac6c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.920+0000 7fac72536640 1 --2- 172.21.15.16:0/128637488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac6c101de0 0x7fac6c198430 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.921+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 >> 172.21.15.16:0/128637488 conn(0x7fac6c0fd8c0 msgr2=0x7fac6c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.921+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 shutdown_connections 2024-09-15T22:44:22.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:22.921+0000 7fac72536640 1 -- 172.21.15.16:0/128637488 wait complete. 2024-09-15T22:44:23.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:22 smithi130 ceph-mon[35742]: pgmap v181: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:44:23.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:22 smithi059 ceph-mon[34872]: pgmap v181: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:44:23.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:44:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 -- 172.21.15.16:0/3612743524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c1036e0 msgr2=0x7fb04c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 --2- 172.21.15.16:0/3612743524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c1036e0 0x7fb04c107b90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fb040009930 tx=0x7fb04002a420 comp rx=0 tx=0).stop 2024-09-15T22:44:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 -- 172.21.15.16:0/3612743524 shutdown_connections 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 --2- 172.21.15.16:0/3612743524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb04c1082a0 0x7fb04c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 --2- 172.21.15.16:0/3612743524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c1036e0 0x7fb04c107b90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 --2- 172.21.15.16:0/3612743524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04c102d30 0x7fb04c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 -- 172.21.15.16:0/3612743524 >> 172.21.15.16:0/3612743524 conn(0x7fb04c0fc7d0 msgr2=0x7fb04c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 -- 172.21.15.16:0/3612743524 shutdown_connections 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 -- 172.21.15.16:0/3612743524 wait complete. 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 Processor -- start 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.334+0000 7fb052c76640 1 -- start start 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb052c76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c102d30 0x7fb04c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb052c76640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb04c1036e0 0x7fb04c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb052c76640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04c1082a0 0x7fb04c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb052c76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1120a0 con 0x7fb04c102d30 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb052c76640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c111f20 con 0x7fb04c1082a0 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb052c76640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c112220 con 0x7fb04c1036e0 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c102d30 0x7fb04c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c102d30 0x7fb04c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46580/0 (socket says 172.21.15.16:46580) 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 -- 172.21.15.16:0/4040630604 learned_addr learned my addr 172.21.15.16:0/4040630604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb051213640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04c1082a0 0x7fb04c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb04bfff640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb04c1036e0 0x7fb04c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 -- 172.21.15.16:0/4040630604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb04c1036e0 msgr2=0x7fb04c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb04c1036e0 0x7fb04c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 -- 172.21.15.16:0/4040630604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04c1082a0 msgr2=0x7fb04c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04c1082a0 0x7fb04c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 -- 172.21.15.16:0/4040630604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c19d5f0 con 0x7fb04c102d30 2024-09-15T22:44:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb04bfff640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb04c1036e0 0x7fb04c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.335+0000 7fb050a12640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c102d30 0x7fb04c198640 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c890 tx=0x7fb03c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.336+0000 7fb049ffb640 1 -- 172.21.15.16:0/4040630604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c004220 con 0x7fb04c102d30 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.336+0000 7fb049ffb640 1 -- 172.21.15.16:0/4040630604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb03c0043c0 con 0x7fb04c102d30 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.336+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04c19d8e0 con 0x7fb04c102d30 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.336+0000 7fb049ffb640 1 -- 172.21.15.16:0/4040630604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0049e0 con 0x7fb04c102d30 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.336+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04c1a51c0 con 0x7fb04c102d30 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.338+0000 7fb049ffb640 1 -- 172.21.15.16:0/4040630604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb03c0079b0 con 0x7fb04c102d30 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 Processor -- start 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 -- start start 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a3b40 0x559edb3a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a25d0 0x559edb3a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559edb3082f0 con 0x559edb3a25d0 2024-09-15T22:44:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559edb2afcf0 con 0x559edb3a3b40 2024-09-15T22:44:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb05504f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559edb3042b0 con 0x559edb3ad370 2024-09-15T22:44:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb050a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a25d0 0x559edb3a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb050a12640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a25d0 0x559edb3a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46582/0 (socket says 172.21.15.16:46582) 2024-09-15T22:44:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb050a12640 1 -- 172.21.15.16:0/1504619412 learned_addr learned my addr 172.21.15.16:0/1504619412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb049ffb640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0340a3430 0x7fb0340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb051213640 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a3b40 0x559edb3a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb04bfff640 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb051213640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0340a3430 0x7fb0340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb050a12640 1 -- 172.21.15.16:0/1504619412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 msgr2=0x559edb3af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb050a12640 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb050a12640 1 -- 172.21.15.16:0/1504619412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a3b40 msgr2=0x559edb3a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.339+0000 7fb050a12640 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a3b40 0x559edb3a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb050a12640 1 -- 172.21.15.16:0/1504619412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edb2943a0 con 0x559edb3a25d0 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb051213640 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a3b40 0x559edb3a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb049ffb640 1 -- 172.21.15.16:0/4040630604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb03c0d00a0 con 0x7fb04c102d30 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb050a12640 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a25d0 0x559edb3a2a10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb03c09bfb0 tx=0x7fb03c09bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9afbf3dfbacbd2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb051213640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0340a3430 0x7fb0340a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb04400afd0 tx=0x7fb04400a4d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb032ffd640 1 -- 172.21.15.16:0/1504619412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c01d020 con 0x559edb3a25d0 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb032ffd640 1 -- 172.21.15.16:0/1504619412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb03c0bc110 con 0x559edb3a25d0 2024-09-15T22:44:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.340+0000 7fb032ffd640 1 -- 172.21.15.16:0/1504619412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a4c90 con 0x559edb3a25d0 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 -- 172.21.15.16:0/1504619412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a25d0 msgr2=0x559edb3a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a25d0 0x559edb3a2a10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb03c09bfb0 tx=0x7fb03c09bab0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 -- 172.21.15.16:0/1504619412 shutdown_connections 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a25d0 0x559edb3a2a10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 --2- 172.21.15.16:0/1504619412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a3b40 0x559edb3a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 -- 172.21.15.16:0/1504619412 >> 172.21.15.16:0/1504619412 conn(0x559edb36b310 msgr2=0x559edb39ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 -- 172.21.15.16:0/1504619412 shutdown_connections 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 -- 172.21.15.16:0/1504619412 wait complete. 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 -- start start 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.341+0000 7fb05504f740 1 Processor -- start 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb05504f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3acd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb05504f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3aa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb05504f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb05504f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559edb3affb0 con 0x559edb3a3b40 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb05504f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559edb2c3060 con 0x559edb3a25d0 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb05504f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559edb3b0130 con 0x559edb3ad370 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb05504f740 10 client.0 _ll_register_callbacks cb 0x559edb0168c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46598/0 (socket says 172.21.15.16:46598) 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 -- 172.21.15.16:0/62751914 learned_addr learned my addr 172.21.15.16:0/62751914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb050a12640 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb051213640 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 -- 172.21.15.16:0/62751914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 msgr2=0x559edb3aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 -- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 msgr2=0x559edb3acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 -- 172.21.15.16:0/62751914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edb0793c0 con 0x559edb3a3b40 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb04bfff640 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3aa020 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb040000c00 tx=0x7fb0400318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0cc6c4a568a25f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.342+0000 7fb02bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb02bfff640 1 -- 172.21.15.16:0/62751914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040049060 con 0x559edb3a3b40 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb03c0bc110 con 0x559edb3a3b40 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb02bfff640 1 -- 172.21.15.16:0/62751914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb040004450 con 0x559edb3a3b40 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb02bfff640 1 -- 172.21.15.16:0/62751914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb040050050 con 0x559edb3a3b40 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 10 client.16812 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559edb3bba50 con 0x559edb3a3b40 2024-09-15T22:44:24.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb02bfff640 1 -- 172.21.15.16:0/62751914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb03c0bc110 con 0x559edb3a3b40 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb02bfff640 1 -- 172.21.15.16:0/62751914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb040004cd0 con 0x559edb3a3b40 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 10 client.16812 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 10 client.16812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 20 client.16812 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb3bcf10 0x559edb3bf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 4 client.16812 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.343+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559edb315860 con 0x559edb3bcf10 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.344+0000 7fb050a12640 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb3bcf10 0x559edb3bf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.344+0000 7fb050a12640 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb3bcf10 0x559edb3bf360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2dec4c493192926 server_cookie=5d4336185a1d92d8 in_seq=0 out_seq=0 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.344+0000 7fb02bfff640 10 client.16812 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb02bfff640 1 -- 172.21.15.16:0/62751914 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559edb315860 con 0x559edb3bcf10 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb02bfff640 10 client.16812 handle_command_reply: tid=0 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 client.16812 shutdown 2024-09-15T22:44:24.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 msgr2=0x559edb3aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3aa020 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb040000c00 tx=0x7fb0400318b0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 shutdown_connections 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb3bcf10 0x559edb3bf360 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3aa020 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb02bfff640 0 client.16812 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 --2- 172.21.15.16:0/62751914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 >> 172.21.15.16:0/62751914 conn(0x559edb36b310 msgr2=0x559edb3a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 shutdown_connections 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 1 -- 172.21.15.16:0/62751914 wait complete. 2024-09-15T22:44:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.345+0000 7fb05504f740 20 client.16812 trim_cache size 0 max 16384 2024-09-15T22:44:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 Processor -- start 2024-09-15T22:44:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 -- start start 2024-09-15T22:44:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3a25d0 0x559edb3abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3ad370 0x559edb3a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1af710 con 0x559edb3a3b40 2024-09-15T22:44:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1acfe0 con 0x559edb3ad370 2024-09-15T22:44:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb05504f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1aae30 con 0x559edb3a25d0 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.358+0000 7fb04bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46608/0 (socket says 172.21.15.16:46608) 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 -- 172.21.15.16:0/1326823863 learned_addr learned my addr 172.21.15.16:0/1326823863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb051213640 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3ad370 0x559edb3a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb050a12640 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3a25d0 0x559edb3abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 -- 172.21.15.16:0/1326823863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3a25d0 msgr2=0x559edb3abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3a25d0 0x559edb3abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 -- 172.21.15.16:0/1326823863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3ad370 msgr2=0x559edb3a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3ad370 0x559edb3a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 -- 172.21.15.16:0/1326823863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559edb39e4f0 con 0x559edb3a3b40 2024-09-15T22:44:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb04bfff640 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3a5120 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb04003a800 tx=0x7fb04002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d404c54b3a5a144b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb0327fc640 1 -- 172.21.15.16:0/1326823863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040059070 con 0x559edb3a3b40 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb0327fc640 1 -- 172.21.15.16:0/1326823863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04003aa80 con 0x559edb3a3b40 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.359+0000 7fb0327fc640 1 -- 172.21.15.16:0/1326823863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040049070 con 0x559edb3a3b40 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 -- 172.21.15.16:0/1326823863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 msgr2=0x559edb3a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3a5120 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb04003a800 tx=0x7fb04002a420 comp rx=0 tx=0).stop 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 -- 172.21.15.16:0/1326823863 shutdown_connections 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3ad370 0x559edb3a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3a5120 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 --2- 172.21.15.16:0/1326823863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3a25d0 0x559edb3abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 -- 172.21.15.16:0/1326823863 >> 172.21.15.16:0/1326823863 conn(0x559edb36b310 msgr2=0x559edb3ab880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 -- 172.21.15.16:0/1326823863 shutdown_connections 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 -- 172.21.15.16:0/1326823863 wait complete. 2024-09-15T22:44:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 -- start start 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.360+0000 7fb05504f740 1 Processor -- start 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1ab4a0 con 0x559edb3a3b40 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1ac910 con 0x559edb3a25d0 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1abb70 con 0x559edb3ad370 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 10 client.0 _ll_register_callbacks cb 0x7fb040009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46624/0 (socket says 172.21.15.16:46624) 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 -- 172.21.15.16:0/2071337527 learned_addr learned my addr 172.21.15.16:0/2071337527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb051213640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb050a12640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 -- 172.21.15.16:0/2071337527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 msgr2=0x559edb3b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 -- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 msgr2=0x559edb3b10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3b10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 -- 172.21.15.16:0/2071337527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559edb426c00 con 0x559edb3a3b40 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb051213640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb050a12640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb04bfff640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3b1600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb040057040 tx=0x7fb040004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01019a3e1c9bb51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb02bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb02bfff640 1 -- 172.21.15.16:0/2071337527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040059040 con 0x559edb3a3b40 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb02bfff640 1 -- 172.21.15.16:0/2071337527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fb0400318b0 con 0x559edb3a3b40 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.361+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04003aa80 con 0x559edb3a3b40 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb02bfff640 1 -- 172.21.15.16:0/2071337527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb040049040 con 0x559edb3a3b40 2024-09-15T22:44:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 10 client.16818 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559edb427e40 con 0x559edb3a3b40 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb02bfff640 1 -- 172.21.15.16:0/2071337527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04003aa80 con 0x559edb3a3b40 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb02bfff640 1 -- 172.21.15.16:0/2071337527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb0400035f0 con 0x559edb3a3b40 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 10 client.16818 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 10 client.16818 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 20 client.16818 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb4283c0 0x559edb42a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 4 client.16818 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559edb30fd70 con 0x559edb4283c0 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.362+0000 7fb050a12640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb4283c0 0x559edb42a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb02bfff640 10 client.16818 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb050a12640 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb4283c0 0x559edb42a7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aca7fae15df2a7fa server_cookie=be90d91d34fbabb1 in_seq=0 out_seq=0 2024-09-15T22:44:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb02bfff640 1 -- 172.21.15.16:0/2071337527 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559edb30fd70 con 0x559edb4283c0 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb02bfff640 10 client.16818 handle_command_reply: tid=0 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 client.16818 shutdown 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 msgr2=0x559edb3b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3b1600 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb040057040 tx=0x7fb040004520 comp rx=0 tx=0).stop 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 shutdown_connections 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x559edb4283c0 0x559edb42a7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559edb3ad370 0x559edb3b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb02bfff640 0 client.16818 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559edb3a3b40 0x559edb3b1600 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 --2- 172.21.15.16:0/2071337527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559edb3a25d0 0x559edb3b10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 >> 172.21.15.16:0/2071337527 conn(0x559edb36b310 msgr2=0x559edb3ac0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 shutdown_connections 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 1 -- 172.21.15.16:0/2071337527 wait complete. 2024-09-15T22:44:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.363+0000 7fb05504f740 20 client.16818 trim_cache size 0 max 16384 2024-09-15T22:44:24.373 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0340a3430 msgr2=0x7fb0340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0340a3430 0x7fb0340a58f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb04400afd0 tx=0x7fb04400a4d0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c102d30 msgr2=0x7fb04c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c102d30 0x7fb04c198640 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c890 tx=0x7fb03c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 shutdown_connections 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb04c1082a0 0x7fb04c19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0340a3430 0x7fb0340a58f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb04c1036e0 0x7fb04c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 --2- 172.21.15.16:0/4040630604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c102d30 0x7fb04c198640 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 >> 172.21.15.16:0/4040630604 conn(0x7fb04c0fc7d0 msgr2=0x7fb04c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 shutdown_connections 2024-09-15T22:44:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:24.364+0000 7fb052c76640 1 -- 172.21.15.16:0/4040630604 wait complete. 2024-09-15T22:44:25.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:24 smithi130 ceph-mon[35742]: pgmap v182: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:25.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:24 smithi016 ceph-mon[31492]: pgmap v182: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:25.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:24 smithi059 ceph-mon[34872]: pgmap v182: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:25.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:44:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 -- 172.21.15.16:0/959841354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24102790 msgr2=0x7efd241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 --2- 172.21.15.16:0/959841354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24102790 0x7efd241067e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009900 tx=0x7efd0c02a410 comp rx=0 tx=0).stop 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 -- 172.21.15.16:0/959841354 shutdown_connections 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 --2- 172.21.15.16:0/959841354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd24106f10 0x7efd2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 --2- 172.21.15.16:0/959841354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24102790 0x7efd241067e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 --2- 172.21.15.16:0/959841354 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd24101de0 0x7efd241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 -- 172.21.15.16:0/959841354 >> 172.21.15.16:0/959841354 conn(0x7efd240fd8c0 msgr2=0x7efd240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 -- 172.21.15.16:0/959841354 shutdown_connections 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.998+0000 7efd296b6640 1 -- 172.21.15.16:0/959841354 wait complete. 2024-09-15T22:44:26.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 Processor -- start 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 -- start start 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd24101de0 0x7efd241984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd24102790 0x7efd241989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24106f10 0x7efd2419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd24111f70 con 0x7efd24106f10 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd24111df0 con 0x7efd24101de0 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:25.999+0000 7efd296b6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efd241120f0 con 0x7efd24102790 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24106f10 0x7efd2419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24106f10 0x7efd2419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46640/0 (socket says 172.21.15.16:46640) 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 -- 172.21.15.16:0/2124363246 learned_addr learned my addr 172.21.15.16:0/2124363246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd227fc640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd24102790 0x7efd241989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd22ffd640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd24101de0 0x7efd241984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 -- 172.21.15.16:0/2124363246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd24102790 msgr2=0x7efd241989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd24102790 0x7efd241989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 -- 172.21.15.16:0/2124363246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd24101de0 msgr2=0x7efd241984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd24101de0 0x7efd241984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 -- 172.21.15.16:0/2124363246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2419d460 con 0x7efd24106f10 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd237fe640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24106f10 0x7efd2419cd80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efd1400d750 tx=0x7efd1400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd1bfff640 1 -- 172.21.15.16:0/2124363246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14017070 con 0x7efd24106f10 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd1bfff640 1 -- 172.21.15.16:0/2124363246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14005500 con 0x7efd24106f10 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2419d750 con 0x7efd24106f10 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd1bfff640 1 -- 172.21.15.16:0/2124363246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14004d90 con 0x7efd24106f10 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.000+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd241a5030 con 0x7efd24106f10 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.002+0000 7efd1bfff640 1 -- 172.21.15.16:0/2124363246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efd14004f30 con 0x7efd24106f10 2024-09-15T22:44:26.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 Processor -- start 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 -- start start 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4eb40 0x55b661f4d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 0x55b661f5eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b661eb32f0 con 0x55b661f4eb40 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b661e5acf0 con 0x55b661f4d5d0 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b661eaf2b0 con 0x55b661f5c780 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4eb40 0x55b661f4d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.003+0000 7efd1bfff640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd100a33b0 0x7efd100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4eb40 0x55b661f4d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46654/0 (socket says 172.21.15.16:46654) 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 -- 172.21.15.16:0/4281612945 learned_addr learned my addr 172.21.15.16:0/4281612945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd22ffd640 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 0x55b661f5eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd237fe640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd100a33b0 0x7efd100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 -- 172.21.15.16:0/4281612945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 msgr2=0x55b661f5eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 0x55b661f5eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 -- 172.21.15.16:0/4281612945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 msgr2=0x55b661f4da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f4da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 -- 172.21.15.16:0/4281612945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b661e3f3a0 con 0x55b661f4eb40 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd1bfff640 1 -- 172.21.15.16:0/2124363246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7efd140cf940 con 0x7efd24106f10 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd227fc640 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4eb40 0x55b661f4d090 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x55b661f4e070 tx=0x7efd0c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51645a4761360941 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd197fa640 1 -- 172.21.15.16:0/4281612945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c044030 con 0x55b661f4eb40 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd197fa640 1 -- 172.21.15.16:0/4281612945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c005020 con 0x55b661f4eb40 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd237fe640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd100a33b0 0x7efd100a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd14018000 tx=0x7efd1409f040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.004+0000 7efd197fa640 1 -- 172.21.15.16:0/4281612945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c035040 con 0x55b661f4eb40 2024-09-15T22:44:26.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 -- 172.21.15.16:0/4281612945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4eb40 msgr2=0x55b661f4d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4eb40 0x55b661f4d090 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x55b661f4e070 tx=0x7efd0c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 -- 172.21.15.16:0/4281612945 shutdown_connections 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 0x55b661f5eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f4da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/4281612945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4eb40 0x55b661f4d090 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 -- 172.21.15.16:0/4281612945 >> 172.21.15.16:0/4281612945 conn(0x55b661f16310 msgr2=0x55b661f49c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 -- 172.21.15.16:0/4281612945 shutdown_connections 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 -- 172.21.15.16:0/4281612945 wait complete. 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.005+0000 7efd2ba8f740 1 -- start start 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 1 Processor -- start 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4d5d0 0x55b661f5c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 0x55b661f59480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 0x55b661f59c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b661f5f420 con 0x55b661f4d5d0 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b661e6e060 con 0x55b661f4eb40 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55b661f5f5a0 con 0x55b661f5c780 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd2ba8f740 10 client.0 _ll_register_callbacks cb 0x55b661bc18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4d5d0 0x55b661f5c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4d5d0 0x55b661f5c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46662/0 (socket says 172.21.15.16:46662) 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 -- 172.21.15.16:0/3337907106 learned_addr learned my addr 172.21.15.16:0/3337907106 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 -- 172.21.15.16:0/3337907106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 msgr2=0x55b661f59c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 0x55b661f59c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 -- 172.21.15.16:0/3337907106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 msgr2=0x55b661f59480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 0x55b661f59480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 -- 172.21.15.16:0/3337907106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b661c243c0 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd227fc640 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4d5d0 0x55b661f5c1d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efd0c02aba0 tx=0x7efd0c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bebaf758d0720a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.006+0000 7efd0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3337907106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c04c030 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3337907106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7efd0c005400 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0c005020 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3337907106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd0c059050 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 10 client.16830 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b661f6af30 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3337907106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c005020 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3337907106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efd0c048660 con 0x55b661f4d5d0 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 10 client.16830 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 10 client.16830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 20 client.16830 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:26.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661f6c280 0x55b661f6e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 4 client.16830 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b661ec0860 con 0x55b661f6c280 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.007+0000 7efd22ffd640 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661f6c280 0x55b661f6e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.008+0000 7efd22ffd640 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661f6c280 0x55b661f6e670 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b41fb53c4b63e8 server_cookie=8effeae64b30385b in_seq=0 out_seq=0 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.008+0000 7efd0a7fc640 10 client.16830 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3337907106 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b661ec0860 con 0x55b661f6c280 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd0a7fc640 10 client.16830 handle_command_reply: tid=0 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 client.16830 shutdown 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4d5d0 msgr2=0x55b661f5c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4d5d0 0x55b661f5c1d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efd0c02aba0 tx=0x7efd0c005850 comp rx=0 tx=0).stop 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 shutdown_connections 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661f6c280 0x55b661f6e670 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f5c780 0x55b661f59c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 0x55b661f59480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd0a7fc640 0 client.16830 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3337907106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f4d5d0 0x55b661f5c1d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 >> 172.21.15.16:0/3337907106 conn(0x55b661f16310 msgr2=0x55b661f4f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 shutdown_connections 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3337907106 wait complete. 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.009+0000 7efd2ba8f740 20 client.16830 trim_cache size 0 max 16384 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 Processor -- start 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 -- start start 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f58d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4eb40 0x55b661f67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f566e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd241af520 con 0x55b661f5c780 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd241acdf0 con 0x55b661f4d5d0 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efd241aac40 con 0x55b661f4eb40 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46674/0 (socket says 172.21.15.16:46674) 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 -- 172.21.15.16:0/2044707440 learned_addr learned my addr 172.21.15.16:0/2044707440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f58d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd22ffd640 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4eb40 0x55b661f67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 -- 172.21.15.16:0/2044707440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4eb40 msgr2=0x55b661f67f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4eb40 0x55b661f67f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 -- 172.21.15.16:0/2044707440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 msgr2=0x55b661f58d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f58d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 -- 172.21.15.16:0/2044707440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b661f58430 con 0x55b661f5c780 2024-09-15T22:44:26.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd22ffd640 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4eb40 0x55b661f67f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd227fc640 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f566e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efd0c001eb0 tx=0x7efd0c03c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dac83da3b6fdaa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd18ff9640 1 -- 172.21.15.16:0/2044707440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c04c040 con 0x55b661f5c780 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd18ff9640 1 -- 172.21.15.16:0/2044707440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c0317f0 con 0x55b661f5c780 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.023+0000 7efd18ff9640 1 -- 172.21.15.16:0/2044707440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c035070 con 0x55b661f5c780 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- 172.21.15.16:0/2044707440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 msgr2=0x55b661f566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f566e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efd0c001eb0 tx=0x7efd0c03c040 comp rx=0 tx=0).stop 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- 172.21.15.16:0/2044707440 shutdown_connections 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f566e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4eb40 0x55b661f67f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/2044707440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4d5d0 0x55b661f58d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- 172.21.15.16:0/2044707440 >> 172.21.15.16:0/2044707440 conn(0x55b661f16310 msgr2=0x55b661f67830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- 172.21.15.16:0/2044707440 shutdown_connections 2024-09-15T22:44:26.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- 172.21.15.16:0/2044707440 wait complete. 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- start start 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 Processor -- start 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4d5d0 0x55b661f6f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 0x55b661f6fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f70190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efd241ab2b0 con 0x55b661f5c780 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd241ac720 con 0x55b661f4eb40 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7efd241ab980 con 0x55b661f4d5d0 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.024+0000 7efd2ba8f740 10 client.0 _ll_register_callbacks cb 0x7efd1c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f70190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f70190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46692/0 (socket says 172.21.15.16:46692) 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 -- 172.21.15.16:0/3616626675 learned_addr learned my addr 172.21.15.16:0/3616626675 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4d5d0 0x55b661f6f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd22ffd640 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 0x55b661f6fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 -- 172.21.15.16:0/3616626675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4d5d0 msgr2=0x55b661f6f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4d5d0 0x55b661f6f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 -- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 msgr2=0x55b661f6fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 0x55b661f6fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 -- 172.21.15.16:0/3616626675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b661f59f80 con 0x55b661f5c780 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd227fc640 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f70190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009900 tx=0x7efd0c031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e678fed0896e677 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3616626675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c063040 con 0x55b661f5c780 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3616626675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7efd0c048400 con 0x55b661f5c780 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0c0317f0 con 0x55b661f5c780 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3616626675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd0c035040 con 0x55b661f5c780 2024-09-15T22:44:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd2ba8f740 10 client.16836 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.025+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b661fd6f70 con 0x55b661f5c780 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3616626675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c0317f0 con 0x55b661f5c780 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3616626675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efd0c0046c0 con 0x55b661f5c780 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd2ba8f740 10 client.16836 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd2ba8f740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd2ba8f740 20 client.16836 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661fd7520 0x55b661fd9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd2ba8f740 4 client.16836 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b661ebad70 con 0x55b661fd7520 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd22ffd640 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661fd7520 0x55b661fd9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd0a7fc640 10 client.16836 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.026+0000 7efd22ffd640 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661fd7520 0x55b661fd9910 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403ed53a39479943 server_cookie=c2167b801bb58a93 in_seq=0 out_seq=0 2024-09-15T22:44:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd0a7fc640 1 -- 172.21.15.16:0/3616626675 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b661ebad70 con 0x55b661fd7520 2024-09-15T22:44:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd0a7fc640 10 client.16836 handle_command_reply: tid=0 2024-09-15T22:44:26.058 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:44:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 client.16836 shutdown 2024-09-15T22:44:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 msgr2=0x55b661f70190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f70190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009900 tx=0x7efd0c031670 comp rx=0 tx=0).stop 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 shutdown_connections 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55b661fd7520 0x55b661fd9910 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b661f5c780 0x55b661f70190 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd0a7fc640 0 client.16836 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b661f4eb40 0x55b661f6fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 --2- 172.21.15.16:0/3616626675 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55b661f4d5d0 0x55b661f6f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 >> 172.21.15.16:0/3616626675 conn(0x55b661f16310 msgr2=0x55b661f68050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 shutdown_connections 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 1 -- 172.21.15.16:0/3616626675 wait complete. 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd2ba8f740 20 client.16836 trim_cache size 0 max 16384 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd100a33b0 msgr2=0x7efd100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd296b6640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd100a33b0 0x7efd100a5870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd14018000 tx=0x7efd1409f040 comp rx=0 tx=0).stop 2024-09-15T22:44:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24106f10 msgr2=0x7efd2419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.027+0000 7efd296b6640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24106f10 0x7efd2419cd80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efd1400d750 tx=0x7efd1400dc10 comp rx=0 tx=0).stop 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 shutdown_connections 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd24106f10 0x7efd2419cd80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7efd100a33b0 0x7efd100a5870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7efd24102790 0x7efd241989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 --2- 172.21.15.16:0/2124363246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd24101de0 0x7efd241984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 >> 172.21.15.16:0/2124363246 conn(0x7efd240fd8c0 msgr2=0x7efd240fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 shutdown_connections 2024-09-15T22:44:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:26.028+0000 7efd296b6640 1 -- 172.21.15.16:0/2124363246 wait complete. 2024-09-15T22:44:27.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:26 smithi130 ceph-mon[35742]: pgmap v183: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:27.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:26 smithi016 ceph-mon[31492]: pgmap v183: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:27.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:26 smithi059 ceph-mon[34872]: pgmap v183: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:27.760 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:44:27.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:44:28.032 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 -- 172.21.15.16:0/438086439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00102ef0 msgr2=0x7f3f0010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 --2- 172.21.15.16:0/438086439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00102ef0 0x7f3f0010da40 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009ca0 tx=0x7f3ef802a950 comp rx=0 tx=0).stop 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 -- 172.21.15.16:0/438086439 shutdown_connections 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 --2- 172.21.15.16:0/438086439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00102ef0 0x7f3f0010da40 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 --2- 172.21.15.16:0/438086439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f00102570 0x7f3f001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 --2- 172.21.15.16:0/438086439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f00100e20 0x7f3f00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 -- 172.21.15.16:0/438086439 >> 172.21.15.16:0/438086439 conn(0x7f3f000fc7d0 msgr2=0x7f3f000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 -- 172.21.15.16:0/438086439 shutdown_connections 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 -- 172.21.15.16:0/438086439 wait complete. 2024-09-15T22:44:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 Processor -- start 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.522+0000 7f3f074a4640 1 -- start start 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f074a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00100e20 0x7f3f001984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f074a4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f00102570 0x7f3f00198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f074a4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f00102ef0 0x7f3f0019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f074a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00110f80 con 0x7f3f00100e20 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f074a4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00110e00 con 0x7f3f00102570 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f074a4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3f00111100 con 0x7f3f00102ef0 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00100e20 0x7f3f001984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00100e20 0x7f3f001984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46706/0 (socket says 172.21.15.16:46706) 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 -- 172.21.15.16:0/2291366310 learned_addr learned my addr 172.21.15.16:0/2291366310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05a41640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f00102ef0 0x7f3f0019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 -- 172.21.15.16:0/2291366310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f00102ef0 msgr2=0x7f3f0019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f00102ef0 0x7f3f0019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f00102570 0x7f3f00198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 -- 172.21.15.16:0/2291366310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f00102570 msgr2=0x7f3f00198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f00102570 0x7f3f00198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 -- 172.21.15.16:0/2291366310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0019d480 con 0x7f3f00100e20 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05a41640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f00102ef0 0x7f3f0019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f05240640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00100e20 0x7f3f001984d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00d770 tx=0x7f3efc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3ef67fc640 1 -- 172.21.15.16:0/2291366310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc017070 con 0x7f3f00100e20 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3ef67fc640 1 -- 172.21.15.16:0/2291366310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3efc005500 con 0x7f3f00100e20 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0019d770 con 0x7f3f00100e20 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.523+0000 7f3ef67fc640 1 -- 172.21.15.16:0/2291366310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc004e00 con 0x7f3f00100e20 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.524+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f001a5050 con 0x7f3f00100e20 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3ef67fc640 1 -- 172.21.15.16:0/2291366310 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3efc004fa0 con 0x7f3f00100e20 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 Processor -- start 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 -- start start 2024-09-15T22:44:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce282df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282e480 0x559ce2838800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce2838d40 0x559ce283b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ce2793280 con 0x559ce2838d40 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559ce273ab10 con 0x559ce282db60 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.526+0000 7f3f0987d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559ce278f240 con 0x559ce282e480 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3ef67fc640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3eec0a3430 0x7f3eec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce2838d40 0x559ce283b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f05a41640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce282df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce2838d40 0x559ce283b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46708/0 (socket says 172.21.15.16:46708) 2024-09-15T22:44:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 -- 172.21.15.16:0/3265005458 learned_addr learned my addr 172.21.15.16:0/3265005458 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f05240640 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282e480 0x559ce2838800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f05a41640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3eec0a3430 0x7f3eec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3ef67fc640 1 -- 172.21.15.16:0/2291366310 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f3efc0cfb00 con 0x7f3f00100e20 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 -- 172.21.15.16:0/3265005458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282e480 msgr2=0x559ce2838800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282e480 0x559ce2838800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 -- 172.21.15.16:0/3265005458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 msgr2=0x559ce282df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce282df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 -- 172.21.15.16:0/3265005458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce27471f0 con 0x559ce2838d40 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f05a41640 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce282df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f05a41640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3eec0a3430 0x7f3eec0a58f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8030040 tx=0x7f3ef80095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce2838d40 0x559ce283b130 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009ae0 tx=0x7f3ef000e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed29d9f0f8b4469 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3eeb7fe640 1 -- 172.21.15.16:0/3265005458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0016070 con 0x559ce2838d40 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.527+0000 7f3eeb7fe640 1 -- 172.21.15.16:0/3265005458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef0005500 con 0x559ce2838d40 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3eeb7fe640 1 -- 172.21.15.16:0/3265005458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0011470 con 0x559ce2838d40 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 -- 172.21.15.16:0/3265005458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce2838d40 msgr2=0x559ce283b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce2838d40 0x559ce283b130 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009ae0 tx=0x7f3ef000e2f0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 -- 172.21.15.16:0/3265005458 shutdown_connections 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce2838d40 0x559ce283b130 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282e480 0x559ce2838800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 --2- 172.21.15.16:0/3265005458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce282df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 -- 172.21.15.16:0/3265005458 >> 172.21.15.16:0/3265005458 conn(0x559ce282a0f0 msgr2=0x559ce282a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 -- 172.21.15.16:0/3265005458 shutdown_connections 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.528+0000 7f3f0987d740 1 -- 172.21.15.16:0/3265005458 wait complete. 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 -- start start 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 Processor -- start 2024-09-15T22:44:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce2835b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce28360b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce2838d40 0x559ce28365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559ce283b910 con 0x559ce282e480 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559ce274df10 con 0x559ce2838d40 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559ce283ba90 con 0x559ce282db60 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f0987d740 10 client.0 _ll_register_callbacks cb 0x559ce2716a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f04a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce28360b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f04a3f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce28360b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46714/0 (socket says 172.21.15.16:46714) 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f04a3f640 1 -- 172.21.15.16:0/1035425384 learned_addr learned my addr 172.21.15.16:0/1035425384 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f05240640 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce2835b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.529+0000 7f3f05a41640 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce2838d40 0x559ce28365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f04a3f640 1 -- 172.21.15.16:0/1035425384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 msgr2=0x559ce2835b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce2835b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f04a3f640 1 -- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce2838d40 msgr2=0x559ce28365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce2838d40 0x559ce28365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f04a3f640 1 -- 172.21.15.16:0/1035425384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce25046a0 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce28360b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0004850 tx=0x7f3ef0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdce8ad62ec30774 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3ee8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1035425384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0028060 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef0005500 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1035425384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f3ef0004c50 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1035425384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ef0016040 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f0987d740 10 client.16848 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ce28474c0 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.530+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1035425384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef0005500 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1035425384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ef000f040 con 0x559ce282e480 2024-09-15T22:44:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f0987d740 10 client.16848 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f0987d740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f0987d740 20 client.16848 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce28488d0 0x559ce284acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f0987d740 4 client.16848 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ce27a07f0 con 0x559ce28488d0 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f05240640 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce28488d0 0x559ce284acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3f05240640 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce28488d0 0x559ce284acc0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aa09ed3a3b524b8 server_cookie=8c367976e7e95cc4 in_seq=0 out_seq=0 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.531+0000 7f3ee8ff9640 10 client.16848 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1035425384 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ce27a07f0 con 0x559ce28488d0 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3ee8ff9640 10 client.16848 handle_command_reply: tid=0 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 client.16848 shutdown 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 msgr2=0x559ce28360b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce28360b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0004850 tx=0x7f3ef0005710 comp rx=0 tx=0).stop 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 shutdown_connections 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce28488d0 0x559ce284acc0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce2838d40 0x559ce28365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3ee8ff9640 0 client.16848 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce28360b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1035425384 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce2835b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 >> 172.21.15.16:0/1035425384 conn(0x559ce282a0f0 msgr2=0x559ce2829270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.532+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 shutdown_connections 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.533+0000 7f3f0987d740 1 -- 172.21.15.16:0/1035425384 wait complete. 2024-09-15T22:44:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.533+0000 7f3f0987d740 20 client.16848 trim_cache size 0 max 16384 2024-09-15T22:44:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 Processor -- start 2024-09-15T22:44:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 -- start start 2024-09-15T22:44:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282a0f0 0x559ce28502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce2850830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce2828a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001af670 con 0x559ce282e480 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001acf40 con 0x559ce282db60 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f0987d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001aad90 con 0x559ce282a0f0 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f05a41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce2828a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f05a41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce2828a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46720/0 (socket says 172.21.15.16:46720) 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f05a41640 1 -- 172.21.15.16:0/2952844485 learned_addr learned my addr 172.21.15.16:0/2952844485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f05240640 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282a0f0 0x559ce28502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.550+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce2850830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3f05a41640 1 -- 172.21.15.16:0/2952844485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282a0f0 msgr2=0x559ce28502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3f05a41640 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282a0f0 0x559ce28502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3f05a41640 1 -- 172.21.15.16:0/2952844485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 msgr2=0x559ce2850830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3f05a41640 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce2850830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3f05a41640 1 -- 172.21.15.16:0/2952844485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce2829f20 con 0x559ce282e480 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce2850830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3f05a41640 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce2828a50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8035000 tx=0x7f3ef8046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dead2642f8a4cd44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3eeaffd640 1 -- 172.21.15.16:0/2952844485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8034040 con 0x559ce282e480 2024-09-15T22:44:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3eeaffd640 1 -- 172.21.15.16:0/2952844485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef8003d70 con 0x559ce282e480 2024-09-15T22:44:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.551+0000 7f3eeaffd640 1 -- 172.21.15.16:0/2952844485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8004780 con 0x559ce282e480 2024-09-15T22:44:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 -- 172.21.15.16:0/2952844485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 msgr2=0x559ce2828a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce2828a50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8035000 tx=0x7f3ef8046000 comp rx=0 tx=0).stop 2024-09-15T22:44:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 -- 172.21.15.16:0/2952844485 shutdown_connections 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282e480 0x559ce2828a50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282db60 0x559ce2850830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 --2- 172.21.15.16:0/2952844485 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282a0f0 0x559ce28502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 -- 172.21.15.16:0/2952844485 >> 172.21.15.16:0/2952844485 conn(0x559ce2831560 msgr2=0x559ce2834c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 -- 172.21.15.16:0/2952844485 shutdown_connections 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 -- 172.21.15.16:0/2952844485 wait complete. 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 -- start start 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.552+0000 7f3f0987d740 1 Processor -- start 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282a0f0 0x559ce283caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce283d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f0987d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282e480 0x559ce283d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f0987d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001ab400 con 0x559ce282a0f0 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f0987d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001ac870 con 0x559ce282e480 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f0987d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3f001abad0 con 0x559ce282db60 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f0987d740 10 client.0 _ll_register_callbacks cb 0x7f3ef8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282a0f0 0x559ce283caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282a0f0 0x559ce283caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46722/0 (socket says 172.21.15.16:46722) 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 -- 172.21.15.16:0/1695869973 learned_addr learned my addr 172.21.15.16:0/1695869973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce283d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05a41640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282e480 0x559ce283d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 -- 172.21.15.16:0/1695869973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 msgr2=0x559ce283d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce283d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 -- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282e480 msgr2=0x559ce283d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:28.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282e480 0x559ce283d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 -- 172.21.15.16:0/1695869973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce28b2370 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05a41640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282e480 0x559ce283d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3f05240640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282a0f0 0x559ce283caf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0cd4f0 tx=0x7f3efc018ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f34fa05014332e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3ee8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.553+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1695869973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc09fe30 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1695869973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f3efc009cc0 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ef8003d70 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1695869973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3efc0a46f0 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 10 client.16854 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ce28b3690 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1695869973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef8003d70 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1695869973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3efc0980f0 con 0x559ce282a0f0 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 10 client.16854 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 20 client.16854 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce2838d40 0x559ce28b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 4 client.16854 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559ce279ad00 con 0x559ce2838d40 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.554+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce2838d40 0x559ce28b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.555+0000 7f3f04a3f640 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce2838d40 0x559ce28b5da0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dcd2ce97b4bce66 server_cookie=a44ed526d5df065b in_seq=0 out_seq=0 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.555+0000 7f3ee8ff9640 10 client.16854 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3ee8ff9640 1 -- 172.21.15.16:0/1695869973 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ce279ad00 con 0x559ce2838d40 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3ee8ff9640 10 client.16854 handle_command_reply: tid=0 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 client.16854 shutdown 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282a0f0 msgr2=0x559ce283caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282a0f0 0x559ce283caf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3efc0cd4f0 tx=0x7f3efc018ad0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 shutdown_connections 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ce2838d40 0x559ce28b5da0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559ce282e480 0x559ce283d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559ce282db60 0x559ce283d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 --2- 172.21.15.16:0/1695869973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559ce282a0f0 0x559ce283caf0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 >> 172.21.15.16:0/1695869973 conn(0x559ce2831560 msgr2=0x559ce2828a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3ee8ff9640 0 client.16854 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 shutdown_connections 2024-09-15T22:44:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 1 -- 172.21.15.16:0/1695869973 wait complete. 2024-09-15T22:44:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.556+0000 7f3f0987d740 20 client.16854 trim_cache size 0 max 16384 2024-09-15T22:44:28.564 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:44:28.564 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:44:28.564 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "f39f9b86-8bce-47d7-96d3-8a9dc17fb624", 2024-09-15T22:44:28.564 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:44:28.564 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:44:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3eec0a3430 msgr2=0x7f3eec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3eec0a3430 0x7f3eec0a58f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8030040 tx=0x7f3ef80095c0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00100e20 msgr2=0x7f3f001984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00100e20 0x7f3f001984d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00d770 tx=0x7f3efc00dc30 comp rx=0 tx=0).stop 2024-09-15T22:44:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 shutdown_connections 2024-09-15T22:44:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f3eec0a3430 0x7f3eec0a58f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3f00102ef0 0x7f3f0019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f00102570 0x7f3f00198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 --2- 172.21.15.16:0/2291366310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f00100e20 0x7f3f001984d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 >> 172.21.15.16:0/2291366310 conn(0x7f3f000fc7d0 msgr2=0x7f3f0006a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 shutdown_connections 2024-09-15T22:44:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:28.557+0000 7f3f074a4640 1 -- 172.21.15.16:0/2291366310 wait complete. 2024-09-15T22:44:29.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:28 smithi130 ceph-mon[35742]: pgmap v184: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:29.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:28 smithi016 ceph-mon[31492]: pgmap v184: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:29.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:28 smithi059 ceph-mon[34872]: pgmap v184: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:29.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:44:29.533 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:29.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:29 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:44:29.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:29 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:29.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:29 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:44:29.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:29 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:30.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:29 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:44:30.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:29 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:30.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:29 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:44:30.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:29 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 -- 172.21.15.16:0/1160372900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8106f50 msgr2=0x7fabb810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/1160372900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8106f50 0x7fabb810eaf0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009cd0 tx=0x7fabb002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 -- 172.21.15.16:0/1160372900 shutdown_connections 2024-09-15T22:44:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/1160372900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb8106f50 0x7fabb810eaf0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/1160372900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabb8100ca0 0x7fabb8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/1160372900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb81002f0 0x7fabb81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 -- 172.21.15.16:0/1160372900 >> 172.21.15.16:0/1160372900 conn(0x7fabb8077210 msgr2=0x7fabb80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 -- 172.21.15.16:0/1160372900 shutdown_connections 2024-09-15T22:44:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.112+0000 7fabbfb2e640 1 -- 172.21.15.16:0/1160372900 wait complete. 2024-09-15T22:44:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 Processor -- start 2024-09-15T22:44:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 -- start start 2024-09-15T22:44:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb81002f0 0x7fabb8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabb8100ca0 0x7fabb81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb8106f50 0x7fabb819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8111fa0 con 0x7fabb81002f0 2024-09-15T22:44:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8111e20 con 0x7fabb8100ca0 2024-09-15T22:44:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbfb2e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8112120 con 0x7fabb8106f50 2024-09-15T22:44:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbd8ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb81002f0 0x7fabb8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbd8ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb81002f0 0x7fabb8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46740/0 (socket says 172.21.15.16:46740) 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbd8ca640 1 -- 172.21.15.16:0/2513494700 learned_addr learned my addr 172.21.15.16:0/2513494700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb8106f50 0x7fabb819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.113+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabb8100ca0 0x7fabb81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbd8ca640 1 -- 172.21.15.16:0/2513494700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb8106f50 msgr2=0x7fabb819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb8106f50 0x7fabb819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbd8ca640 1 -- 172.21.15.16:0/2513494700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabb8100ca0 msgr2=0x7fabb81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabb8100ca0 0x7fabb81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbd8ca640 1 -- 172.21.15.16:0/2513494700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb819d410 con 0x7fabb81002f0 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb81002f0 0x7fabb8198460 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fabb400d950 tx=0x7fabb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabaeffd640 1 -- 172.21.15.16:0/2513494700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4017070 con 0x7fabb81002f0 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabaeffd640 1 -- 172.21.15.16:0/2513494700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb4009c30 con 0x7fabb81002f0 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabaeffd640 1 -- 172.21.15.16:0/2513494700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb400f6b0 con 0x7fabb81002f0 2024-09-15T22:44:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb819d700 con 0x7fabb81002f0 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.114+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81a4fe0 con 0x7fabb81002f0 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabaeffd640 1 -- 172.21.15.16:0/2513494700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fabb40109d0 con 0x7fabb81002f0 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 Processor -- start 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 -- start start 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708c4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c4dc0 0x55ac708cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac7082a2e0 con 0x55ac708cf680 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac707d1ce0 con 0x55ac708c44a0 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabc1f07740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ac708262a0 con 0x55ac708c4dc0 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabbd0c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabbd0c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708d1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46752/0 (socket says 172.21.15.16:46752) 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.117+0000 7fabbd0c9640 1 -- 172.21.15.16:0/2211099247 learned_addr learned my addr 172.21.15.16:0/2211099247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c4dc0 0x55ac708cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabaeffd640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faba40a3430 0x7faba40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbd0c9640 1 -- 172.21.15.16:0/2211099247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c4dc0 msgr2=0x55ac708cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c4dc0 0x55ac708cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbd0c9640 1 -- 172.21.15.16:0/2211099247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 msgr2=0x55ac708c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbd0c9640 1 -- 172.21.15.16:0/2211099247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac707b6440 con 0x55ac708cf680 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faba40a3430 0x7faba40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabaeffd640 1 -- 172.21.15.16:0/2513494700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fabb40cfc20 con 0x7fabb81002f0 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708d1a70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faba800bbf0 tx=0x7faba80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bae9332d5e8465 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7faba3fff640 1 -- 172.21.15.16:0/2211099247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8005020 con 0x55ac708cf680 2024-09-15T22:44:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7faba3fff640 1 -- 172.21.15.16:0/2211099247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba80047a0 con 0x55ac708cf680 2024-09-15T22:44:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.118+0000 7faba3fff640 1 -- 172.21.15.16:0/2211099247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80116a0 con 0x55ac708cf680 2024-09-15T22:44:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 -- 172.21.15.16:0/2211099247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 msgr2=0x55ac708d1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708d1a70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faba800bbf0 tx=0x7faba80107a0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faba40a3430 0x7faba40a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb003e040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 -- 172.21.15.16:0/2211099247 shutdown_connections 2024-09-15T22:44:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708d1a70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c4dc0 0x55ac708cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 --2- 172.21.15.16:0/2211099247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 -- 172.21.15.16:0/2211099247 >> 172.21.15.16:0/2211099247 conn(0x55ac708c0a30 msgr2=0x55ac708c0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 -- 172.21.15.16:0/2211099247 shutdown_connections 2024-09-15T22:44:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 -- 172.21.15.16:0/2211099247 wait complete. 2024-09-15T22:44:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.119+0000 7fabc1f07740 1 -- start start 2024-09-15T22:44:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 1 Processor -- start 2024-09-15T22:44:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c44a0 0x55ac708cef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac708d2250 con 0x55ac708cf680 2024-09-15T22:44:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ac707e5050 con 0x55ac708c4dc0 2024-09-15T22:44:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ac708d23d0 con 0x55ac708c44a0 2024-09-15T22:44:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabc1f07740 10 client.0 _ll_register_callbacks cb 0x55ac70825160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabbe0cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabbe0cb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46768/0 (socket says 172.21.15.16:46768) 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabbe0cb640 1 -- 172.21.15.16:0/3581604142 learned_addr learned my addr 172.21.15.16:0/3581604142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c44a0 0x55ac708cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.120+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabbe0cb640 1 -- 172.21.15.16:0/3581604142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c44a0 msgr2=0x55ac708cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c44a0 0x55ac708cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabbe0cb640 1 -- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 msgr2=0x55ac708cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabbe0cb640 1 -- 172.21.15.16:0/3581604142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7059b3c0 con 0x55ac708cf680 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708cc720 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009cd0 tx=0x7fabb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0e68195b7285ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7faba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7faba17fa640 1 -- 172.21.15.16:0/3581604142 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00305a0 con 0x55ac708cf680 2024-09-15T22:44:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7faba17fa640 1 -- 172.21.15.16:0/3581604142 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fabb0030740 con 0x55ac708cf680 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba80047a0 con 0x55ac708cf680 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7faba17fa640 1 -- 172.21.15.16:0/3581604142 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fabb0059050 con 0x55ac708cf680 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabc1f07740 10 client.16866 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac708ddbc0 con 0x55ac708cf680 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.121+0000 7faba17fa640 1 -- 172.21.15.16:0/3581604142 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba80047a0 con 0x55ac708cf680 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7faba17fa640 1 -- 172.21.15.16:0/3581604142 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fabb0030070 con 0x55ac708cf680 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabc1f07740 10 client.16866 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabc1f07740 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabc1f07740 20 client.16866 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabc1f07740 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac708df080 0x55ac708e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabc1f07740 4 client.16866 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac70837850 con 0x55ac708df080 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac708df080 0x55ac708e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7faba17fa640 10 client.16866 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.122+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac708df080 0x55ac708e14d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c745515812b5a06 server_cookie=3915d40fabd43a7b in_seq=0 out_seq=0 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7faba17fa640 1 -- 172.21.15.16:0/3581604142 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac70837850 con 0x55ac708df080 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7faba17fa640 10 client.16866 handle_command_reply: tid=0 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 client.16866 shutdown 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 msgr2=0x55ac708cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708cc720 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009cd0 tx=0x7fabb0004a40 comp rx=0 tx=0).stop 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 shutdown_connections 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac708df080 0x55ac708e14d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708cf680 0x55ac708cc720 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 --2- 172.21.15.16:0/3581604142 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708c44a0 0x55ac708cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 >> 172.21.15.16:0/3581604142 conn(0x55ac708c0a30 msgr2=0x55ac708c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7faba17fa640 0 client.16866 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:30.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.123+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 shutdown_connections 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.124+0000 7fabc1f07740 1 -- 172.21.15.16:0/3581604142 wait complete. 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.124+0000 7fabc1f07740 20 client.16866 trim_cache size 0 max 16384 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 Processor -- start 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 -- start start 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c4dc0 0x55ac708d7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac708d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81b1760 con 0x55ac708c4dc0 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81af030 con 0x55ac708c44a0 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabc1f07740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ace80 con 0x55ac708cf680 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c4dc0 0x55ac708d7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c4dc0 0x55ac708d7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46780/0 (socket says 172.21.15.16:46780) 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 -- 172.21.15.16:0/766427697 learned_addr learned my addr 172.21.15.16:0/766427697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:30.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac708d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 -- 172.21.15.16:0/766427697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 msgr2=0x55ac708d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac708d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 -- 172.21.15.16:0/766427697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 msgr2=0x55ac708db4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 -- 172.21.15.16:0/766427697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac708c0860 con 0x55ac708c4dc0 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac708d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.141+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c4dc0 0x55ac708d7a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7faba800bbf0 tx=0x7faba8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da95f2c3eabb5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7faba37fe640 1 -- 172.21.15.16:0/766427697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba802b070 con 0x55ac708c4dc0 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7faba37fe640 1 -- 172.21.15.16:0/766427697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba80054e0 con 0x55ac708c4dc0 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7faba37fe640 1 -- 172.21.15.16:0/766427697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80116a0 con 0x55ac708c4dc0 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 -- 172.21.15.16:0/766427697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c4dc0 msgr2=0x55ac708d7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c4dc0 0x55ac708d7a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7faba800bbf0 tx=0x7faba8005850 comp rx=0 tx=0).stop 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 -- 172.21.15.16:0/766427697 shutdown_connections 2024-09-15T22:44:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac708d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c4dc0 0x55ac708d7a00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 --2- 172.21.15.16:0/766427697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c44a0 0x55ac708db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 -- 172.21.15.16:0/766427697 >> 172.21.15.16:0/766427697 conn(0x55ac708c0a30 msgr2=0x55ac708caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 -- 172.21.15.16:0/766427697 shutdown_connections 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 -- 172.21.15.16:0/766427697 wait complete. 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.142+0000 7fabc1f07740 1 -- start start 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 1 Processor -- start 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c44a0 0x55ac708cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac7094cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ad4f0 con 0x55ac708c44a0 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ae960 con 0x55ac708c4dc0 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81adbc0 con 0x55ac708cf680 2024-09-15T22:44:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabc1f07740 10 client.0 _ll_register_callbacks cb 0x7fabb0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c44a0 0x55ac708cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c44a0 0x55ac708cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46788/0 (socket says 172.21.15.16:46788) 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 -- 172.21.15.16:0/853088678 learned_addr learned my addr 172.21.15.16:0/853088678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbe0cb640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac7094cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 -- 172.21.15.16:0/853088678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 msgr2=0x55ac7094cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac7094cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 -- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 msgr2=0x55ac708d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd8ca640 1 -- 172.21.15.16:0/853088678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac708c5a20 con 0x55ac708c44a0 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.143+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7fabbd8ca640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c44a0 0x55ac708cfc40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009610 tx=0x7fabb4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bd8fed9775ec3a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7faba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:44:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7faba17fa640 1 -- 172.21.15.16:0/853088678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4020040 con 0x55ac708c44a0 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7faba17fa640 1 -- 172.21.15.16:0/853088678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fabb40cd230 con 0x55ac708c44a0 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba80054e0 con 0x55ac708c44a0 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7faba17fa640 1 -- 172.21.15.16:0/853088678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fabb40a47b0 con 0x55ac708c44a0 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7fabc1f07740 10 client.16872 fetch_fsmap learned FSMap version 26 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ac7094e480 con 0x55ac708c44a0 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7faba17fa640 1 -- 172.21.15.16:0/853088678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faba80054e0 con 0x55ac708c44a0 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.144+0000 7faba17fa640 1 -- 172.21.15.16:0/853088678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fabb409b040 con 0x55ac708c44a0 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabc1f07740 10 client.16872 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabc1f07740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabc1f07740 20 client.16872 populate_metadata read hostname 'smithi016' 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabc1f07740 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac7094eb80 0x55ac70950fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabc1f07740 4 client.16872 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:44:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac70831d60 con 0x55ac7094eb80 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac7094eb80 0x55ac70950fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7faba17fa640 10 client.16872 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabbd0c9640 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac7094eb80 0x55ac70950fa0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f7341a3b226a85 server_cookie=714577bdca5c935b in_seq=0 out_seq=0 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7faba17fa640 1 -- 172.21.15.16:0/853088678 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ac70831d60 con 0x55ac7094eb80 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7faba17fa640 10 client.16872 handle_command_reply: tid=0 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.145+0000 7fabc1f07740 1 client.16872 shutdown 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c44a0 msgr2=0x55ac708cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c44a0 0x55ac708cfc40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009610 tx=0x7fabb4000bc0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 shutdown_connections 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ac7094eb80 0x55ac70950fa0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ac708cf680 0x55ac7094cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ac708c4dc0 0x55ac708d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 --2- 172.21.15.16:0/853088678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac708c44a0 0x55ac708cfc40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7faba17fa640 0 client.16872 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:44:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 >> 172.21.15.16:0/853088678 conn(0x55ac708c0a30 msgr2=0x55ac708bf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 shutdown_connections 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 1 -- 172.21.15.16:0/853088678 wait complete. 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabc1f07740 20 client.16872 trim_cache size 0 max 16384 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stdout: "f39f9b86-8bce-47d7-96d3-8a9dc17fb624": { 2024-09-15T22:44:30.168 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:44:30.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "f39f9b86-8bce-47d7-96d3-8a9dc17fb624", 2024-09-15T22:44:30.169 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:44:30.169 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:44:30.169 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:44:30.169 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:44:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faba40a3430 msgr2=0x7faba40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faba40a3430 0x7faba40a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabb0006010 tx=0x7fabb003e040 comp rx=0 tx=0).stop 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb81002f0 msgr2=0x7fabb8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb81002f0 0x7fabb8198460 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fabb400d950 tx=0x7fabb400de10 comp rx=0 tx=0).stop 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 shutdown_connections 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.146+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7faba40a3430 0x7faba40a58f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.147+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fabb8106f50 0x7fabb819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.147+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fabb8100ca0 0x7fabb81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.147+0000 7fabbfb2e640 1 --2- 172.21.15.16:0/2513494700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabb81002f0 0x7fabb8198460 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.147+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 >> 172.21.15.16:0/2513494700 conn(0x7fabb8077210 msgr2=0x7fabb8105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.147+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 shutdown_connections 2024-09-15T22:44:30.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:44:30.147+0000 7fabbfb2e640 1 -- 172.21.15.16:0/2513494700 wait complete. 2024-09-15T22:44:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:29 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:44:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:29 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:29 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:44:30.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:29 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:44:30.761 INFO:tasks.cephfs.filesystem:scrub status for tag:f39f9b86-8bce-47d7-96d3-8a9dc17fb624 - {'path': '/', 'tag': 'f39f9b86-8bce-47d7-96d3-8a9dc17fb624', 'options': 'recursive,force'} 2024-09-15T22:44:31.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:30 smithi130 ceph-mon[35742]: pgmap v185: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:31.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:30 smithi016 ceph-mon[31492]: pgmap v185: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:31.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:30 smithi059 ceph-mon[34872]: pgmap v185: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:33.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:32 smithi130 ceph-mon[35742]: pgmap v186: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:44:33.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:32 smithi016 ceph-mon[31492]: pgmap v186: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:44:33.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:32 smithi059 ceph-mon[34872]: pgmap v186: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:44:34.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:33 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:44:34.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:33 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:44:34.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:33 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:44:35.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:34 smithi130 ceph-mon[35742]: pgmap v187: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:35.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:34 smithi016 ceph-mon[31492]: pgmap v187: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:35.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:34 smithi059 ceph-mon[34872]: pgmap v187: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:36 smithi016 ceph-mon[31492]: pgmap v188: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:36 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:37.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:36 smithi059 ceph-mon[34872]: pgmap v188: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:36 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:37.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:36 smithi130 ceph-mon[35742]: pgmap v188: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:37.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:36 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:37 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:37 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:38.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:37 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:44:38.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:37 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:37 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:38.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:37 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:44:38.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:37 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:38.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:37 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:38.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:37 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:44:39.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:38 smithi016 ceph-mon[31492]: pgmap v189: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:39.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:38 smithi059 ceph-mon[34872]: pgmap v189: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:39.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:38 smithi130 ceph-mon[35742]: pgmap v189: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:41.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:40 smithi016 ceph-mon[31492]: pgmap v190: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:41.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:40 smithi059 ceph-mon[34872]: pgmap v190: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:41.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:40 smithi130 ceph-mon[35742]: pgmap v190: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:44:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:42 smithi016 ceph-mon[31492]: pgmap v191: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:43.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:42 smithi059 ceph-mon[34872]: pgmap v191: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:43.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:42 smithi130 ceph-mon[35742]: pgmap v191: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:44:45.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:44 smithi016 ceph-mon[31492]: pgmap v192: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:45.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:44 smithi059 ceph-mon[34872]: pgmap v192: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:45.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:44 smithi130 ceph-mon[35742]: pgmap v192: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:47.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:46 smithi016 ceph-mon[31492]: pgmap v193: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:47.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:46 smithi059 ceph-mon[34872]: pgmap v193: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:47.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:46 smithi130 ceph-mon[35742]: pgmap v193: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:49.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:48 smithi016 ceph-mon[31492]: pgmap v194: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:49.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:48 smithi059 ceph-mon[34872]: pgmap v194: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:49.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:48 smithi130 ceph-mon[35742]: pgmap v194: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:51.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:50 smithi016 ceph-mon[31492]: pgmap v195: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:50 smithi059 ceph-mon[34872]: pgmap v195: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:51.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:50 smithi130 ceph-mon[35742]: pgmap v195: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:53.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:52 smithi059 ceph-mon[34872]: pgmap v196: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:53.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:52 smithi130 ceph-mon[35742]: pgmap v196: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:53.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:52 smithi016 ceph-mon[31492]: pgmap v196: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:55.081 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:54 smithi130 ceph-mon[35742]: pgmap v197: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:55.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:54 smithi059 ceph-mon[34872]: pgmap v197: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:55.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:54 smithi016 ceph-mon[31492]: pgmap v197: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:57.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:56 smithi059 ceph-mon[34872]: pgmap v198: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:57.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:56 smithi130 ceph-mon[35742]: pgmap v198: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:57.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:56 smithi016 ceph-mon[31492]: pgmap v198: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:44:59.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:44:58 smithi059 ceph-mon[34872]: pgmap v199: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:59.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:44:58 smithi130 ceph-mon[35742]: pgmap v199: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:44:59.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:44:58 smithi016 ceph-mon[31492]: pgmap v199: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:00.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:45:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:01.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:00 smithi016 ceph-mon[31492]: pgmap v200: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:01.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:00 smithi059 ceph-mon[34872]: pgmap v200: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:01.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:00 smithi130 ceph-mon[35742]: pgmap v200: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 -- 172.21.15.16:0/937511079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784106a70 msgr2=0x7f1784106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 --2- 172.21.15.16:0/937511079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784106a70 0x7f1784106e50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1774009930 tx=0x7f177402a420 comp rx=0 tx=0).stop 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 -- 172.21.15.16:0/937511079 shutdown_connections 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 --2- 172.21.15.16:0/937511079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1784101410 0x7f178410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 --2- 172.21.15.16:0/937511079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1784100a70 0x7f1784100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 --2- 172.21.15.16:0/937511079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784106a70 0x7f1784106e50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 -- 172.21.15.16:0/937511079 >> 172.21.15.16:0/937511079 conn(0x7f17840fc7d0 msgr2=0x7f17840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 -- 172.21.15.16:0/937511079 shutdown_connections 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.504+0000 7f17894a5640 1 -- 172.21.15.16:0/937511079 wait complete. 2024-09-15T22:45:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 Processor -- start 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 -- start start 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784100a70 0x7f17840fce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1784101410 0x7f17840fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1784106a70 0x7f17840fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1784110e70 con 0x7f1784100a70 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1784110cf0 con 0x7f1784106a70 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17894a5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1784110ff0 con 0x7f1784101410 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f1782ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784100a70 0x7f17840fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f1782ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784100a70 0x7f17840fce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55736/0 (socket says 172.21.15.16:55736) 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f1782ffd640 1 -- 172.21.15.16:0/2477343424 learned_addr learned my addr 172.21.15.16:0/2477343424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.505+0000 7f17837fe640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1784106a70 0x7f17840fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f17827fc640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1784101410 0x7f17840fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f1782ffd640 1 -- 172.21.15.16:0/2477343424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1784101410 msgr2=0x7f17840fd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1784101410 0x7f17840fd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f1782ffd640 1 -- 172.21.15.16:0/2477343424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1784106a70 msgr2=0x7f17840fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1784106a70 0x7f17840fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f1782ffd640 1 -- 172.21.15.16:0/2477343424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17841a68e0 con 0x7f1784100a70 2024-09-15T22:45:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784100a70 0x7f17840fce30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1774009230 tx=0x7f1774004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f176bfff640 1 -- 172.21.15.16:0/2477343424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177403f030 con 0x7f1784100a70 2024-09-15T22:45:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f176bfff640 1 -- 172.21.15.16:0/2477343424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774003ad0 con 0x7f1784100a70 2024-09-15T22:45:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17841a6b70 con 0x7f1784100a70 2024-09-15T22:45:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f176bfff640 1 -- 172.21.15.16:0/2477343424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774035070 con 0x7f1784100a70 2024-09-15T22:45:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.506+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17841a7050 con 0x7f1784100a70 2024-09-15T22:45:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.509+0000 7f176bfff640 1 -- 172.21.15.16:0/2477343424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1774042030 con 0x7f1784100a70 2024-09-15T22:45:01.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f176bfff640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f176c0a33b0 0x7f176c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 Processor -- start 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f176bfff640 1 -- 172.21.15.16:0/2477343424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f17740f0580 con 0x7f1784100a70 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 -- start start 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f17827fc640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f176c0a33b0 0x7f176c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4ee950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf4f1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556daf4412e0 con 0x556daf4db3c0 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556daf3e8ce0 con 0x556daf4dbce0 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f178b87e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556daf43d2a0 con 0x556daf4eee90 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f1782ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f1782ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4db7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55748/0 (socket says 172.21.15.16:55748) 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f1782ffd640 1 -- 172.21.15.16:0/1765812387 learned_addr learned my addr 172.21.15.16:0/1765812387 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:01.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.510+0000 7f17827fc640 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f17837fe640 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf4f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f1782ffd640 1 -- 172.21.15.16:0/1765812387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 msgr2=0x556daf4f1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f1782ffd640 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf4f1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f1782ffd640 1 -- 172.21.15.16:0/1765812387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 msgr2=0x556daf4ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f1782ffd640 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4ee950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f1782ffd640 1 -- 172.21.15.16:0/1765812387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556daf3cd440 con 0x556daf4db3c0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f17837fe640 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf4f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f17827fc640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f176c0a33b0 0x7f176c0a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1778005fd0 tx=0x7f177800f730 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f1782ffd640 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4db7a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f17740b8960 tx=0x7f1774005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802b6e2045ea8443 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f17697fa640 1 -- 172.21.15.16:0/1765812387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774049040 con 0x556daf4db3c0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f17697fa640 1 -- 172.21.15.16:0/1765812387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17740c08b0 con 0x556daf4db3c0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.511+0000 7f17697fa640 1 -- 172.21.15.16:0/1765812387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17740b8b70 con 0x556daf4db3c0 2024-09-15T22:45:01.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 -- 172.21.15.16:0/1765812387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 msgr2=0x556daf4db7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4db7a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f17740b8960 tx=0x7f1774005e50 comp rx=0 tx=0).stop 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 -- 172.21.15.16:0/1765812387 shutdown_connections 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf4f1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 --2- 172.21.15.16:0/1765812387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4db7a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 -- 172.21.15.16:0/1765812387 >> 172.21.15.16:0/1765812387 conn(0x556daf4d7a30 msgr2=0x556daf4d7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 -- 172.21.15.16:0/1765812387 shutdown_connections 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 -- 172.21.15.16:0/1765812387 wait complete. 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.512+0000 7f178b87e740 1 -- start start 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 1 Processor -- start 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4db3c0 0x556daf4ee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4dbce0 0x556daf4ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf4ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556daf4f17c0 con 0x556daf4eee90 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556daf3fc050 con 0x556daf4db3c0 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556daf4f1940 con 0x556daf4dbce0 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f178b87e740 10 client.0 _ll_register_callbacks cb 0x556daf43c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf4ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf4ec110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55752/0 (socket says 172.21.15.16:55752) 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f1782ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4db3c0 0x556daf4ee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17837fe640 1 -- 172.21.15.16:0/2322468711 learned_addr learned my addr 172.21.15.16:0/2322468711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17827fc640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4dbce0 0x556daf4ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17837fe640 1 -- 172.21.15.16:0/2322468711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4dbce0 msgr2=0x556daf4ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17837fe640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4dbce0 0x556daf4ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17837fe640 1 -- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4db3c0 msgr2=0x556daf4ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.513+0000 7f17837fe640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4db3c0 0x556daf4ee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f17837fe640 1 -- 172.21.15.16:0/2322468711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556daf1b23c0 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4db3c0 0x556daf4ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f17827fc640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4dbce0 0x556daf4ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f17837fe640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf4ec110 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f177c011a90 tx=0x7f177c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c91ba56b4b1918 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f1766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f1766ff5640 1 -- 172.21.15.16:0/2322468711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c01b3f0 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17740c08b0 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f1766ff5640 1 -- 172.21.15.16:0/2322468711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f177c01b590 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f1766ff5640 1 -- 172.21.15.16:0/2322468711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f177c026050 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f178b87e740 10 client.16884 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556daf4fd1f0 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.514+0000 7f1766ff5640 1 -- 172.21.15.16:0/2322468711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17740c08b0 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f1766ff5640 1 -- 172.21.15.16:0/2322468711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f177c01a950 con 0x556daf4eee90 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f178b87e740 10 client.16884 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:01.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f178b87e740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f178b87e740 20 client.16884 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f178b87e740 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf4fe770 0x556daf500b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f178b87e740 4 client.16884 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556daf44e850 con 0x556daf4fe770 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.515+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf4fe770 0x556daf500b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f1766ff5640 10 client.16884 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf4fe770 0x556daf500b90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f09da3e83874e202 server_cookie=5089fc4163f9c8a5 in_seq=0 out_seq=0 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f1766ff5640 1 -- 172.21.15.16:0/2322468711 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556daf44e850 con 0x556daf4fe770 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f1766ff5640 10 client.16884 handle_command_reply: tid=0 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f178b87e740 1 client.16884 shutdown 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 msgr2=0x556daf4ec110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f178b87e740 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf4ec110 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f177c011a90 tx=0x7f177c011f50 comp rx=0 tx=0).stop 2024-09-15T22:45:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 shutdown_connections 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f178b87e740 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf4fe770 0x556daf500b90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.516+0000 7f178b87e740 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf4ec110 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.517+0000 7f178b87e740 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4dbce0 0x556daf4ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.517+0000 7f178b87e740 1 --2- 172.21.15.16:0/2322468711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4db3c0 0x556daf4ee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.517+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 >> 172.21.15.16:0/2322468711 conn(0x556daf4d7a30 msgr2=0x556daf4d6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.517+0000 7f1766ff5640 0 client.16884 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.517+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 shutdown_connections 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.517+0000 7f178b87e740 1 -- 172.21.15.16:0/2322468711 wait complete. 2024-09-15T22:45:01.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.517+0000 7f178b87e740 20 client.16884 trim_cache size 0 max 16384 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 Processor -- start 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 -- start start 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf506460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf5069a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17841af510 con 0x556daf4db3c0 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17841acde0 con 0x556daf4dbce0 2024-09-15T22:45:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f178b87e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f17841aac30 con 0x556daf4eee90 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f1782ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f1782ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4fa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55758/0 (socket says 172.21.15.16:55758) 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.534+0000 7f1782ffd640 1 -- 172.21.15.16:0/2473531507 learned_addr learned my addr 172.21.15.16:0/2473531507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f17827fc640 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf506460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f17837fe640 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf5069a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1782ffd640 1 -- 172.21.15.16:0/2473531507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 msgr2=0x556daf5069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf5069a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1782ffd640 1 -- 172.21.15.16:0/2473531507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 msgr2=0x556daf506460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf506460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1782ffd640 1 -- 172.21.15.16:0/2473531507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556daf501210 con 0x556daf4db3c0 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4fa990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f17740bc490 tx=0x7f17740bc640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f7f619069c4770 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1768ff9640 1 -- 172.21.15.16:0/2473531507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17740d9020 con 0x556daf4db3c0 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1768ff9640 1 -- 172.21.15.16:0/2473531507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17740c08b0 con 0x556daf4db3c0 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.535+0000 7f1768ff9640 1 -- 172.21.15.16:0/2473531507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774049040 con 0x556daf4db3c0 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 -- 172.21.15.16:0/2473531507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 msgr2=0x556daf4fa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4fa990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f17740bc490 tx=0x7f17740bc640 comp rx=0 tx=0).stop 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 -- 172.21.15.16:0/2473531507 shutdown_connections 2024-09-15T22:45:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4eee90 0x556daf5069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf506460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 --2- 172.21.15.16:0/2473531507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4db3c0 0x556daf4fa990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 -- 172.21.15.16:0/2473531507 >> 172.21.15.16:0/2473531507 conn(0x556daf4d7a30 msgr2=0x556daf4edb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 -- 172.21.15.16:0/2473531507 shutdown_connections 2024-09-15T22:45:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.536+0000 7f178b87e740 1 -- 172.21.15.16:0/2473531507 wait complete. 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 -- start start 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 Processor -- start 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4db3c0 0x556daf4f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf568430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17841ab2a0 con 0x556daf4eee90 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f17841ac710 con 0x556daf4dbce0 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f17841ab970 con 0x556daf4db3c0 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f178b87e740 10 client.0 _ll_register_callbacks cb 0x7f177c0078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f17837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf568430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f17837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf568430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55764/0 (socket says 172.21.15.16:55764) 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f17837fe640 1 -- 172.21.15.16:0/2893015393 learned_addr learned my addr 172.21.15.16:0/2893015393 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f17827fc640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.537+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4db3c0 0x556daf4f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f17837fe640 1 -- 172.21.15.16:0/2893015393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4db3c0 msgr2=0x556daf4f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f17837fe640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4db3c0 0x556daf4f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f17837fe640 1 -- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 msgr2=0x556daf4f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f17837fe640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f17837fe640 1 -- 172.21.15.16:0/2893015393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556daf568970 con 0x556daf4eee90 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f17827fc640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f17837fe640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf568430 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f177c01ad70 tx=0x7f177c011650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37196e1170ad8160 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f1766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f1766ff5640 1 -- 172.21.15.16:0/2893015393 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c02e070 con 0x556daf4eee90 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f1766ff5640 1 -- 172.21.15.16:0/2893015393 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f177c01a3a0 con 0x556daf4eee90 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17740c08b0 con 0x556daf4eee90 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f1766ff5640 1 -- 172.21.15.16:0/2893015393 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f177c03c050 con 0x556daf4eee90 2024-09-15T22:45:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f178b87e740 10 client.16890 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.538+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556daf569bc0 con 0x556daf4eee90 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f1766ff5640 1 -- 172.21.15.16:0/2893015393 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17740c08b0 con 0x556daf4eee90 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f1766ff5640 1 -- 172.21.15.16:0/2893015393 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f177c033800 con 0x556daf4eee90 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f178b87e740 10 client.16890 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f178b87e740 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f178b87e740 20 client.16890 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f178b87e740 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf56a2b0 0x556daf56c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f178b87e740 4 client.16890 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:45:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556daf448d60 con 0x556daf56a2b0 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.539+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf56a2b0 0x556daf56c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f1782ffd640 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf56a2b0 0x556daf56c6d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1e2bb1ef172636 server_cookie=475c10b8bd2f60d in_seq=0 out_seq=0 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f1766ff5640 10 client.16890 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f1766ff5640 1 -- 172.21.15.16:0/2893015393 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556daf448d60 con 0x556daf56a2b0 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f1766ff5640 10 client.16890 handle_command_reply: tid=0 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 client.16890 shutdown 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 msgr2=0x556daf568430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf568430 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f177c01ad70 tx=0x7f177c011650 comp rx=0 tx=0).stop 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 shutdown_connections 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556daf56a2b0 0x556daf56c6d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f1766ff5640 0 client.16890 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556daf4eee90 0x556daf568430 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556daf4dbce0 0x556daf4f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 --2- 172.21.15.16:0/2893015393 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556daf4db3c0 0x556daf4f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 >> 172.21.15.16:0/2893015393 conn(0x556daf4d7a30 msgr2=0x556daf4d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 shutdown_connections 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 1 -- 172.21.15.16:0/2893015393 wait complete. 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.540+0000 7f178b87e740 20 client.16890 trim_cache size 0 max 16384 2024-09-15T22:45:01.549 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:45:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:45:01.550 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:45:01.550 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:45:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f176c0a33b0 msgr2=0x7f176c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f176c0a33b0 0x7f176c0a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1778005fd0 tx=0x7f177800f730 comp rx=0 tx=0).stop 2024-09-15T22:45:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784100a70 msgr2=0x7f17840fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784100a70 0x7f17840fce30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1774009230 tx=0x7f1774004270 comp rx=0 tx=0).stop 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 shutdown_connections 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1784106a70 0x7f17840fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f176c0a33b0 0x7f176c0a5870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1784101410 0x7f17840fd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 --2- 172.21.15.16:0/2477343424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1784100a70 0x7f17840fce30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 >> 172.21.15.16:0/2477343424 conn(0x7f17840fc7d0 msgr2=0x7f178406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 shutdown_connections 2024-09-15T22:45:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:01.541+0000 7f17894a5640 1 -- 172.21.15.16:0/2477343424 wait complete. 2024-09-15T22:45:02.319 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:45:02.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:45:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.036 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:02 smithi016 ceph-mon[31492]: pgmap v201: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:45:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 -- 172.21.15.16:0/2531468066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 msgr2=0x7fc950103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 --2- 172.21.15.16:0/2531468066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc950103120 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009900 tx=0x7fc94c02a480 comp rx=0 tx=0).stop 2024-09-15T22:45:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 -- 172.21.15.16:0/2531468066 shutdown_connections 2024-09-15T22:45:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 --2- 172.21.15.16:0/2531468066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc95010c170 0x7fc95010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 --2- 172.21.15.16:0/2531468066 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc950103660 0x7fc95010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 --2- 172.21.15.16:0/2531468066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc950103120 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 -- 172.21.15.16:0/2531468066 >> 172.21.15.16:0/2531468066 conn(0x7fc9500fc7a0 msgr2=0x7fc9500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 -- 172.21.15.16:0/2531468066 shutdown_connections 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.118+0000 7fc9580a2640 1 -- 172.21.15.16:0/2531468066 wait complete. 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 Processor -- start 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 -- start start 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc95019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc950103660 0x7fc95019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc95010c170 0x7fc95019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc950111f70 con 0x7fc950102d40 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc950111df0 con 0x7fc950103660 2024-09-15T22:45:03.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc9580a2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc9501120f0 con 0x7fc95010c170 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc955e3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc95019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc955e3e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc95019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55774/0 (socket says 172.21.15.16:55774) 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc955e3e640 1 -- 172.21.15.16:0/3465923399 learned_addr learned my addr 172.21.15.16:0/3465923399 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc95663f640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc95010c170 0x7fc95019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc95563d640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc950103660 0x7fc95019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc955e3e640 1 -- 172.21.15.16:0/3465923399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc95010c170 msgr2=0x7fc95019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc955e3e640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc95010c170 0x7fc95019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc955e3e640 1 -- 172.21.15.16:0/3465923399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc950103660 msgr2=0x7fc95019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.119+0000 7fc955e3e640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc950103660 0x7fc95019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.120+0000 7fc955e3e640 1 -- 172.21.15.16:0/3465923399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95019f640 con 0x7fc950102d40 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.120+0000 7fc955e3e640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc95019a690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006010 tx=0x7fc94c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:03.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.120+0000 7fc946ffd640 1 -- 172.21.15.16:0/3465923399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c03f030 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.120+0000 7fc946ffd640 1 -- 172.21.15.16:0/3465923399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94c003ce0 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.120+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95019f8d0 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.120+0000 7fc946ffd640 1 -- 172.21.15.16:0/3465923399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c035070 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.120+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9501a7210 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.121+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc950103120 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.122+0000 7fc946ffd640 1 -- 172.21.15.16:0/3465923399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc94c042030 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.123+0000 7fc946ffd640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc93c0a3430 0x7fc93c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.123+0000 7fc95563d640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc93c0a3430 0x7fc93c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.123+0000 7fc946ffd640 1 -- 172.21.15.16:0/3465923399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7fc94c0f05c0 con 0x7fc950102d40 2024-09-15T22:45:03.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.123+0000 7fc95563d640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc93c0a3430 0x7fc93c0a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc95010d4d0 tx=0x7fc94000b2e0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:03.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.127+0000 7fc946ffd640 1 -- 172.21.15.16:0/3465923399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc94c0bc650 con 0x7fc950102d40 2024-09-15T22:45:03.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:02 smithi059 ceph-mon[34872]: pgmap v201: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:45:03.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:02 smithi130 ceph-mon[35742]: pgmap v201: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:45:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.348+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9501a73e0 con 0x7fc950102d40 2024-09-15T22:45:03.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.348+0000 7fc946ffd640 1 -- 172.21.15.16:0/3465923399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fc94c0c0950 con 0x7fc950102d40 2024-09-15T22:45:03.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:45:03.350 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:45:03.350 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:45:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.350+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc93c0a3430 msgr2=0x7fc93c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.350+0000 7fc9580a2640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc93c0a3430 0x7fc93c0a58f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc95010d4d0 tx=0x7fc94000b2e0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.350+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 msgr2=0x7fc95019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.350+0000 7fc9580a2640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc95019a690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc94c006010 tx=0x7fc94c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 shutdown_connections 2024-09-15T22:45:03.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc93c0a3430 0x7fc93c0a58f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc95010c170 0x7fc95019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc950103660 0x7fc95019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 --2- 172.21.15.16:0/3465923399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc950102d40 0x7fc95019a690 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 >> 172.21.15.16:0/3465923399 conn(0x7fc9500fc7a0 msgr2=0x7fc9500fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 shutdown_connections 2024-09-15T22:45:03.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:03.351+0000 7fc9580a2640 1 -- 172.21.15.16:0/3465923399 wait complete. 2024-09-15T22:45:04.040 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:03 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3465923399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:04.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:45:04.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:03 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3465923399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:04.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:03 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3465923399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:04.334 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 -- 172.21.15.16:0/3833037581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710101de0 msgr2=0x7f77101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 --2- 172.21.15.16:0/3833037581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710101de0 0x7f77101021c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7704009900 tx=0x7f770402a410 comp rx=0 tx=0).stop 2024-09-15T22:45:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 -- 172.21.15.16:0/3833037581 shutdown_connections 2024-09-15T22:45:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 --2- 172.21.15.16:0/3833037581 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7710106f10 0x7f771010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 --2- 172.21.15.16:0/3833037581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7710102790 0x7f77101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 --2- 172.21.15.16:0/3833037581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710101de0 0x7f77101021c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 -- 172.21.15.16:0/3833037581 >> 172.21.15.16:0/3833037581 conn(0x7f77100fd8c0 msgr2=0x7f77100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.743+0000 7f7716a91640 1 -- 172.21.15.16:0/3833037581 shutdown_connections 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.744+0000 7f7716a91640 1 -- 172.21.15.16:0/3833037581 wait complete. 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.744+0000 7f7716a91640 1 Processor -- start 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.744+0000 7f7716a91640 1 -- start start 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f7716a91640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7710101de0 0x7f77100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f7716a91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710102790 0x7f7710074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f7716a91640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7710106f10 0x7f7710074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f7716a91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7710111f70 con 0x7f7710102790 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f7716a91640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7710111df0 con 0x7f7710106f10 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f7716a91640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f77101120f0 con 0x7f7710101de0 2024-09-15T22:45:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710102790 0x7f7710074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f771482d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7710101de0 0x7f77100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710102790 0x7f7710074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55800/0 (socket says 172.21.15.16:55800) 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f771482d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7710101de0 0x7f77100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:50052/0 (socket says 172.21.15.16:50052) 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 -- 172.21.15.16:0/2384479537 learned_addr learned my addr 172.21.15.16:0/2384479537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f771502e640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7710106f10 0x7f7710074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 -- 172.21.15.16:0/2384479537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7710101de0 msgr2=0x7f77100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7710101de0 0x7f77100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 -- 172.21.15.16:0/2384479537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7710106f10 msgr2=0x7f7710074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7710106f10 0x7f7710074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 -- 172.21.15.16:0/2384479537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77101a6990 con 0x7f7710102790 2024-09-15T22:45:04.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.745+0000 7f770ffff640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710102790 0x7f7710074a30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c890 tx=0x7f76fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.746+0000 7f770dffb640 1 -- 172.21.15.16:0/2384479537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc004220 con 0x7f7710102790 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.746+0000 7f770dffb640 1 -- 172.21.15.16:0/2384479537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc0043c0 con 0x7f7710102790 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.746+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77101a6c80 con 0x7f7710102790 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.746+0000 7f770dffb640 1 -- 172.21.15.16:0/2384479537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0049e0 con 0x7f7710102790 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.746+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101a71b0 con 0x7f7710102790 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.748+0000 7f770dffb640 1 -- 172.21.15.16:0/2384479537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f76fc0079b0 con 0x7f7710102790 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 Processor -- start 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 -- start start 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f770dffb640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77000a3430 0x7f77000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 0x5640f06f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640f064b2f0 con 0x5640f06e6b40 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5640f05f2cf0 con 0x5640f06e55d0 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f7718e6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5640f06472b0 con 0x5640f06f0370 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f770dffb640 1 -- 172.21.15.16:0/2384479537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f76fc0cffb0 con 0x7f7710102790 2024-09-15T22:45:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771502e640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77000a3430 0x7f77000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55816/0 (socket says 172.21.15.16:55816) 2024-09-15T22:45:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 -- 172.21.15.16:0/686493798 learned_addr learned my addr 172.21.15.16:0/686493798 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771502e640 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 0x5640f06f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f770ffff640 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 -- 172.21.15.16:0/686493798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 msgr2=0x5640f06f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 0x5640f06f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 -- 172.21.15.16:0/686493798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 msgr2=0x5640f06e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f771482d640 1 -- 172.21.15.16:0/686493798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f05d73a0 con 0x5640f06e6b40 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.749+0000 7f770ffff640 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.750+0000 7f771502e640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77000a3430 0x7f77000a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7708005fd0 tx=0x7f7708014b70 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.750+0000 7f771482d640 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e5090 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7704009900 tx=0x7f7704031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ff0e560010d122 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.750+0000 7f76faffd640 1 -- 172.21.15.16:0/686493798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704040070 con 0x5640f06e6b40 2024-09-15T22:45:04.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.750+0000 7f76faffd640 1 -- 172.21.15.16:0/686493798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704005020 con 0x5640f06e6b40 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.750+0000 7f76faffd640 1 -- 172.21.15.16:0/686493798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704035040 con 0x5640f06e6b40 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.750+0000 7f7718e6a740 1 -- 172.21.15.16:0/686493798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 msgr2=0x5640f06e5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.750+0000 7f7718e6a740 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e5090 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7704009900 tx=0x7f7704031630 comp rx=0 tx=0).stop 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 -- 172.21.15.16:0/686493798 shutdown_connections 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 0x5640f06f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 --2- 172.21.15.16:0/686493798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e5090 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 -- 172.21.15.16:0/686493798 >> 172.21.15.16:0/686493798 conn(0x5640f06ae310 msgr2=0x5640f06e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 -- 172.21.15.16:0/686493798 shutdown_connections 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 -- 172.21.15.16:0/686493798 wait complete. 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 -- start start 2024-09-15T22:45:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.751+0000 7f7718e6a740 1 Processor -- start 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e55d0 0x5640f06efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06f0370 0x5640f06ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f7718e6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640f06f3010 con 0x5640f06e55d0 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f7718e6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5640f0606060 con 0x5640f06f0370 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f7718e6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5640f06f3190 con 0x5640f06e6b40 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f7718e6a740 10 client.0 _ll_register_callbacks cb 0x5640f03598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e55d0 0x5640f06efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e55d0 0x5640f06efcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55820/0 (socket says 172.21.15.16:55820) 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 -- 172.21.15.16:0/238421179 learned_addr learned my addr 172.21.15.16:0/238421179 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f770ffff640 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771502e640 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06f0370 0x5640f06ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 -- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 msgr2=0x5640f06ecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06ecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 -- 172.21.15.16:0/238421179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06f0370 msgr2=0x5640f06ed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06f0370 0x5640f06ed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 -- 172.21.15.16:0/238421179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f03bc3c0 con 0x5640f06e55d0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f771482d640 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e55d0 0x5640f06efcd0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7704005a60 tx=0x7f7704005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d51dff29f574e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.752+0000 7f76effff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f76effff640 1 -- 172.21.15.16:0/238421179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770404b070 con 0x5640f06e55d0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f76effff640 1 -- 172.21.15.16:0/238421179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7704004b60 con 0x5640f06e55d0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7704005020 con 0x5640f06e55d0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f76effff640 1 -- 172.21.15.16:0/238421179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7704057050 con 0x5640f06e55d0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 10 client.16908 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640f06fe9c0 con 0x5640f06e55d0 2024-09-15T22:45:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f76effff640 1 -- 172.21.15.16:0/238421179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704005020 con 0x5640f06e55d0 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f76effff640 1 -- 172.21.15.16:0/238421179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7704044400 con 0x5640f06e55d0 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 10 client.16908 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 10 client.16908 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 20 client.16908 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f06ffd10 0x5640f0702100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 4 client.16908 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.753+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640f0658860 con 0x5640f06ffd10 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.754+0000 7f770ffff640 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f06ffd10 0x5640f0702100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.754+0000 7f770ffff640 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f06ffd10 0x5640f0702100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99eaa2ef8f3cbd server_cookie=af63354576f2b7af in_seq=0 out_seq=0 2024-09-15T22:45:04.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.754+0000 7f76effff640 10 client.16908 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:04.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f76effff640 1 -- 172.21.15.16:0/238421179 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640f0658860 con 0x5640f06ffd10 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f76effff640 10 client.16908 handle_command_reply: tid=0 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 client.16908 shutdown 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e55d0 msgr2=0x5640f06efcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e55d0 0x5640f06efcd0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7704005a60 tx=0x7f7704005540 comp rx=0 tx=0).stop 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 shutdown_connections 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f06ffd10 0x5640f0702100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f76effff640 0 client.16908 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06f0370 0x5640f06ed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06ecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 --2- 172.21.15.16:0/238421179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e55d0 0x5640f06efcd0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.755+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 >> 172.21.15.16:0/238421179 conn(0x5640f06ae310 msgr2=0x5640f06e7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.756+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 shutdown_connections 2024-09-15T22:45:04.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.756+0000 7f7718e6a740 1 -- 172.21.15.16:0/238421179 wait complete. 2024-09-15T22:45:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.756+0000 7f7718e6a740 20 client.16908 trim_cache size 0 max 16384 2024-09-15T22:45:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 Processor -- start 2024-09-15T22:45:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 -- start start 2024-09-15T22:45:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 0x5640f06e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77101b3a10 con 0x5640f06e6b40 2024-09-15T22:45:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77101b12e0 con 0x5640f06e55d0 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f7718e6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f77101af130 con 0x5640f06f0370 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55834/0 (socket says 172.21.15.16:55834) 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 -- 172.21.15.16:0/737907717 learned_addr learned my addr 172.21.15.16:0/737907717 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f771482d640 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 -- 172.21.15.16:0/737907717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 msgr2=0x5640f06e8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 0x5640f06e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 -- 172.21.15.16:0/737907717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 msgr2=0x5640f06e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.769+0000 7f770ffff640 1 -- 172.21.15.16:0/737907717 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f06e2960 con 0x5640f06e6b40 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f770ffff640 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e0c10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0cd9a0 tx=0x7f76fc005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e01d0e89a6dfec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f76fa7fc640 1 -- 172.21.15.16:0/737907717 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc01d040 con 0x5640f06e6b40 2024-09-15T22:45:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f76fa7fc640 1 -- 172.21.15.16:0/737907717 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc09a2e0 con 0x5640f06e6b40 2024-09-15T22:45:04.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f76fa7fc640 1 -- 172.21.15.16:0/737907717 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0a4710 con 0x5640f06e6b40 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 -- 172.21.15.16:0/737907717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 msgr2=0x5640f06e0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e0c10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0cd9a0 tx=0x7f76fc005410 comp rx=0 tx=0).stop 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 -- 172.21.15.16:0/737907717 shutdown_connections 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06f0370 0x5640f06e8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06e6b40 0x5640f06e0c10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 --2- 172.21.15.16:0/737907717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 -- 172.21.15.16:0/737907717 >> 172.21.15.16:0/737907717 conn(0x5640f06ae310 msgr2=0x5640f06e0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.770+0000 7f7718e6a740 1 -- 172.21.15.16:0/737907717 shutdown_connections 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 -- 172.21.15.16:0/737907717 wait complete. 2024-09-15T22:45:04.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 -- start start 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 Processor -- start 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06f4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06f0370 0x5640f06f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77101af7a0 con 0x5640f06f0370 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77101b0c10 con 0x5640f06e55d0 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f77101afe70 con 0x5640f06e6b40 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f7718e6a740 10 client.0 _ll_register_callbacks cb 0x7f7704009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.771+0000 7f771502e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06f0370 0x5640f06f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06f0370 0x5640f06f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55846/0 (socket says 172.21.15.16:55846) 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 -- 172.21.15.16:0/3217748863 learned_addr learned my addr 172.21.15.16:0/3217748863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771482d640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f770ffff640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 -- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 msgr2=0x5640f06f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 -- 172.21.15.16:0/3217748863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 msgr2=0x5640f06f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 -- 172.21.15.16:0/3217748863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f0769910 con 0x5640f06f0370 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771482d640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f771502e640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06f0370 0x5640f06f4ef0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7708012f30 tx=0x7f7708004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ca357abdd2d176 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f76effff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f76effff640 1 -- 172.21.15.16:0/3217748863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708026070 con 0x5640f06f0370 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76fc09a2e0 con 0x5640f06f0370 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f76effff640 1 -- 172.21.15.16:0/3217748863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f7708004750 con 0x5640f06f0370 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f76effff640 1 -- 172.21.15.16:0/3217748863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7708032050 con 0x5640f06f0370 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f7718e6a740 10 client.16914 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.772+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640f076ac10 con 0x5640f06f0370 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f76effff640 1 -- 172.21.15.16:0/3217748863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76fc09a2e0 con 0x5640f06f0370 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f76effff640 1 -- 172.21.15.16:0/3217748863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f770800f040 con 0x5640f06f0370 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f7718e6a740 10 client.16914 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f7718e6a740 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f7718e6a740 20 client.16914 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f7718e6a740 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f076b360 0x5640f076d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f7718e6a740 4 client.16914 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640f0652d70 con 0x5640f076b360 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.773+0000 7f771482d640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f076b360 0x5640f076d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f771482d640 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f076b360 0x5640f076d780 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8ec8b3fbfa56f80 server_cookie=7ef621eea684e690 in_seq=0 out_seq=0 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f76effff640 10 client.16914 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f76effff640 1 -- 172.21.15.16:0/3217748863 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640f0652d70 con 0x5640f076b360 2024-09-15T22:45:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f76effff640 10 client.16914 handle_command_reply: tid=0 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 client.16914 shutdown 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06f0370 msgr2=0x5640f06f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06f0370 0x5640f06f4ef0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7708012f30 tx=0x7f7708004a40 comp rx=0 tx=0).stop 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 shutdown_connections 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5640f076b360 0x5640f076d780 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640f06f0370 0x5640f06f4ef0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640f06e6b40 0x5640f06f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 --2- 172.21.15.16:0/3217748863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640f06e55d0 0x5640f06f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 >> 172.21.15.16:0/3217748863 conn(0x5640f06ae310 msgr2=0x5640f06e0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f76effff640 0 client.16914 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 shutdown_connections 2024-09-15T22:45:04.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 1 -- 172.21.15.16:0/3217748863 wait complete. 2024-09-15T22:45:04.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.774+0000 7f7718e6a740 20 client.16914 trim_cache size 0 max 16384 2024-09-15T22:45:04.783 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77000a3430 msgr2=0x7f77000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77000a3430 0x7f77000a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7708005fd0 tx=0x7f7708014b70 comp rx=0 tx=0).stop 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710102790 msgr2=0x7f7710074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710102790 0x7f7710074a30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00c890 tx=0x7f76fc00cd50 comp rx=0 tx=0).stop 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 shutdown_connections 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7710106f10 0x7f7710074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7710102790 0x7f7710074a30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f77000a3430 0x7f77000a58f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 --2- 172.21.15.16:0/2384479537 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7710101de0 0x7f77100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 >> 172.21.15.16:0/2384479537 conn(0x7f77100fd8c0 msgr2=0x7f77100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 shutdown_connections 2024-09-15T22:45:04.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:04.775+0000 7f7716a91640 1 -- 172.21.15.16:0/2384479537 wait complete. 2024-09-15T22:45:05.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:04 smithi016 ceph-mon[31492]: pgmap v202: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:45:05.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:04 smithi059 ceph-mon[34872]: pgmap v202: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:45:05.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:04 smithi130 ceph-mon[35742]: pgmap v202: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:45:05.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:45:05.805 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 -- 172.21.15.16:0/1808282015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 msgr2=0x7f63481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 --2- 172.21.15.16:0/1808282015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f63481067e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6330009900 tx=0x7f633002a410 comp rx=0 tx=0).stop 2024-09-15T22:45:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 -- 172.21.15.16:0/1808282015 shutdown_connections 2024-09-15T22:45:06.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 --2- 172.21.15.16:0/1808282015 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6348106f10 0x7f634810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 --2- 172.21.15.16:0/1808282015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f63481067e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 --2- 172.21.15.16:0/1808282015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6348101de0 0x7f63481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 -- 172.21.15.16:0/1808282015 >> 172.21.15.16:0/1808282015 conn(0x7f63480fd8c0 msgr2=0x7f63480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 -- 172.21.15.16:0/1808282015 shutdown_connections 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.294+0000 7f634dfef640 1 -- 172.21.15.16:0/1808282015 wait complete. 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 Processor -- start 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 -- start start 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6348101de0 0x7f6348198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f6348198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6348106f10 0x7f634819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6348111f70 con 0x7f6348102790 2024-09-15T22:45:06.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6348111df0 con 0x7f6348101de0 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f634dfef640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63481120f0 con 0x7f6348106f10 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f633effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f6348198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f633effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f6348198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55854/0 (socket says 172.21.15.16:55854) 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f633effd640 1 -- 172.21.15.16:0/2640286220 learned_addr learned my addr 172.21.15.16:0/2640286220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.295+0000 7f63477fe640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6348101de0 0x7f6348198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f6347fff640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6348106f10 0x7f634819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f633effd640 1 -- 172.21.15.16:0/2640286220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6348106f10 msgr2=0x7f634819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f633effd640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6348106f10 0x7f634819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f633effd640 1 -- 172.21.15.16:0/2640286220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6348101de0 msgr2=0x7f6348198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f633effd640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6348101de0 0x7f6348198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f633effd640 1 -- 172.21.15.16:0/2640286220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634819d400 con 0x7f6348102790 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f633effd640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f6348198990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6330006010 tx=0x7f63300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f63457fa640 1 -- 172.21.15.16:0/2640286220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633003f030 con 0x7f6348102790 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634819d690 con 0x7f6348102790 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f63457fa640 1 -- 172.21.15.16:0/2640286220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6330003ce0 con 0x7f6348102790 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f63457fa640 1 -- 172.21.15.16:0/2640286220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330035070 con 0x7f6348102790 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.296+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481a4fd0 con 0x7f6348102790 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.298+0000 7f63457fa640 1 -- 172.21.15.16:0/2640286220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6330042030 con 0x7f6348102790 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 Processor -- start 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 -- start start 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f06944f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f06944fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f06945c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f0693b52f0 con 0x55f06945a370 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f06935ccf0 con 0x55f069450b40 2024-09-15T22:45:06.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63503c8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f0693b12b0 con 0x55f06944f5d0 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f633effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f06945c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f633effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f06945c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55856/0 (socket says 172.21.15.16:55856) 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f633effd640 1 -- 172.21.15.16:0/2546092188 learned_addr learned my addr 172.21.15.16:0/2546092188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f6347fff640 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f06944f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63477fe640 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f06944fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.299+0000 7f63457fa640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63340a33b0 0x7f63340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633effd640 1 -- 172.21.15.16:0/2546092188 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 msgr2=0x55f06944fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633effd640 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f06944fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633effd640 1 -- 172.21.15.16:0/2546092188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 msgr2=0x55f06944f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633effd640 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f06944f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633effd640 1 -- 172.21.15.16:0/2546092188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0693413a0 con 0x55f06945a370 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f6347fff640 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f06944f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:06.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f6347fff640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63340a33b0 0x7f63340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f63457fa640 1 -- 172.21.15.16:0/2640286220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f63300f0660 con 0x7f6348102790 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633effd640 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f06945c760 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f63300ed9c0 tx=0x7f633004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebdb3a815c6a066 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633dffb640 1 -- 172.21.15.16:0/2546092188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63300bc1e0 con 0x55f06945a370 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633dffb640 1 -- 172.21.15.16:0/2546092188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63300bcda0 con 0x55f06945a370 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f6347fff640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63340a33b0 0x7f63340a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6340009fd0 tx=0x7f6340004a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.300+0000 7f633dffb640 1 -- 172.21.15.16:0/2546092188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63300044e0 con 0x55f06945a370 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 -- 172.21.15.16:0/2546092188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 msgr2=0x55f06945c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f06945c760 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f63300ed9c0 tx=0x7f633004b000 comp rx=0 tx=0).stop 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 -- 172.21.15.16:0/2546092188 shutdown_connections 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f06945c760 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f06944fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 --2- 172.21.15.16:0/2546092188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f06944f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 -- 172.21.15.16:0/2546092188 >> 172.21.15.16:0/2546092188 conn(0x55f069418310 msgr2=0x55f06944bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 -- 172.21.15.16:0/2546092188 shutdown_connections 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 -- 172.21.15.16:0/2546092188 wait complete. 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.301+0000 7f63503c8740 1 -- start start 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 1 Processor -- start 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06944f5d0 0x55f069459ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f069456f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06945a370 0x55f0694574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f06945ce30 con 0x55f06944f5d0 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f069370060 con 0x55f069450b40 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f06945cfb0 con 0x55f06945a370 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06944f5d0 0x55f069459ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63503c8740 10 client.0 _ll_register_callbacks cb 0x55f0690c38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06944f5d0 0x55f069459ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55868/0 (socket says 172.21.15.16:55868) 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f63477fe640 1 -- 172.21.15.16:0/3833647333 learned_addr learned my addr 172.21.15.16:0/3833647333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:06.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f633effd640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f069456f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.302+0000 7f6347fff640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06945a370 0x55f0694574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63477fe640 1 -- 172.21.15.16:0/3833647333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06945a370 msgr2=0x55f0694574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63477fe640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06945a370 0x55f0694574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63477fe640 1 -- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 msgr2=0x55f069456f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63477fe640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f069456f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63477fe640 1 -- 172.21.15.16:0/3833647333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0691263c0 con 0x55f06944f5d0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f633effd640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f069456f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63477fe640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06944f5d0 0x55f069459ce0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f633800bad0 tx=0x7f633800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b830c30431f17dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f6326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f6326ffd640 1 -- 172.21.15.16:0/3833647333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338005020 con 0x55f06944f5d0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f6326ffd640 1 -- 172.21.15.16:0/3833647333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f63380051c0 con 0x55f06944f5d0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63300bcda0 con 0x55f06944f5d0 2024-09-15T22:45:06.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f6326ffd640 1 -- 172.21.15.16:0/3833647333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6338005400 con 0x55f06944f5d0 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63503c8740 10 client.16926 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f069468790 con 0x55f06944f5d0 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.303+0000 7f6326ffd640 1 -- 172.21.15.16:0/3833647333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63300bcda0 con 0x55f06944f5d0 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f6326ffd640 1 -- 172.21.15.16:0/3833647333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f633801b740 con 0x55f06944f5d0 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f63503c8740 10 client.16926 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f63503c8740 10 client.16926 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f63503c8740 20 client.16926 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f63503c8740 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f069469c50 0x55f06946c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f63503c8740 4 client.16926 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0693c2860 con 0x55f069469c50 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f633effd640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f069469c50 0x55f06946c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f633effd640 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f069469c50 0x55f06946c060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=861289043d7afc39 server_cookie=f19b26a43ba32304 in_seq=0 out_seq=0 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.304+0000 7f6326ffd640 10 client.16926 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:06.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f6326ffd640 1 -- 172.21.15.16:0/3833647333 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0693c2860 con 0x55f069469c50 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f6326ffd640 10 client.16926 handle_command_reply: tid=0 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 client.16926 shutdown 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06944f5d0 msgr2=0x55f069459ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06944f5d0 0x55f069459ce0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f633800bad0 tx=0x7f633800bed0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 shutdown_connections 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f069469c50 0x55f06946c060 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06945a370 0x55f0694574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f069456f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 --2- 172.21.15.16:0/3833647333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06944f5d0 0x55f069459ce0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 >> 172.21.15.16:0/3833647333 conn(0x55f069418310 msgr2=0x55f069451960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f6326ffd640 0 client.16926 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 shutdown_connections 2024-09-15T22:45:06.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.305+0000 7f63503c8740 1 -- 172.21.15.16:0/3833647333 wait complete. 2024-09-15T22:45:06.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.306+0000 7f63503c8740 20 client.16926 trim_cache size 0 max 16384 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.318+0000 7f63503c8740 1 Processor -- start 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.318+0000 7f63503c8740 1 -- start start 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f63503c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f06944f5d0 0x55f06944be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f63503c8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f069450b40 0x55f069465fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f63503c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069452120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f63503c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63481b1840 con 0x55f06945a370 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f63503c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63481af110 con 0x55f06944f5d0 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f63503c8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63481acf60 con 0x55f069450b40 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069452120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069452120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55870/0 (socket says 172.21.15.16:55870) 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 -- 172.21.15.16:0/2771681033 learned_addr learned my addr 172.21.15.16:0/2771681033 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f633effd640 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f069450b40 0x55f069465fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f63477fe640 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f06944f5d0 0x55f06944be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 -- 172.21.15.16:0/2771681033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f069450b40 msgr2=0x55f069465fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f069450b40 0x55f069465fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 -- 172.21.15.16:0/2771681033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f06944f5d0 msgr2=0x55f06944be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f06944f5d0 0x55f06944be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 -- 172.21.15.16:0/2771681033 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f069450760 con 0x55f06945a370 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.319+0000 7f6347fff640 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069452120 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6340013460 tx=0x7f6340016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b667f495efe3d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f633d7fa640 1 -- 172.21.15.16:0/2771681033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340003970 con 0x55f06945a370 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f633d7fa640 1 -- 172.21.15.16:0/2771681033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6340004020 con 0x55f06945a370 2024-09-15T22:45:06.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f633d7fa640 1 -- 172.21.15.16:0/2771681033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400118f0 con 0x55f06945a370 2024-09-15T22:45:06.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 -- 172.21.15.16:0/2771681033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 msgr2=0x55f069452120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069452120 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6340013460 tx=0x7f6340016000 comp rx=0 tx=0).stop 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 -- 172.21.15.16:0/2771681033 shutdown_connections 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069452120 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f069450b40 0x55f069465fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 --2- 172.21.15.16:0/2771681033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f06944f5d0 0x55f06944be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 -- 172.21.15.16:0/2771681033 >> 172.21.15.16:0/2771681033 conn(0x55f069418310 msgr2=0x55f06944b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 -- 172.21.15.16:0/2771681033 shutdown_connections 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.320+0000 7f63503c8740 1 -- 172.21.15.16:0/2771681033 wait complete. 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 -- start start 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 Processor -- start 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f0694623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f0694628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069453860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63481ad5d0 con 0x55f06945a370 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f63481aea40 con 0x55f069450b40 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f63481adca0 con 0x55f06944f5d0 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63503c8740 10 client.0 _ll_register_callbacks cb 0x7f63380040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f6347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069453860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f6347fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069453860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55874/0 (socket says 172.21.15.16:55874) 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f6347fff640 1 -- 172.21.15.16:0/3067026677 learned_addr learned my addr 172.21.15.16:0/3067026677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f633effd640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f0694628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.321+0000 7f63477fe640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f0694623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6347fff640 1 -- 172.21.15.16:0/3067026677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 msgr2=0x55f0694623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6347fff640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f0694623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6347fff640 1 -- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 msgr2=0x55f0694628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6347fff640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f0694628e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6347fff640 1 -- 172.21.15.16:0/3067026677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f069454090 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f633effd640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f0694628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6347fff640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069453860 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6340004400 tx=0x7f6340004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747f63da5ced6c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6326ffd640 1 -- 172.21.15.16:0/3067026677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340003b80 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6326ffd640 1 -- 172.21.15.16:0/3067026677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f6340003d20 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6340004020 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6326ffd640 1 -- 172.21.15.16:0/3067026677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6340011630 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f63503c8740 10 client.16932 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0694d43f0 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.322+0000 7f6326ffd640 1 -- 172.21.15.16:0/3067026677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6340004020 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f6326ffd640 1 -- 172.21.15.16:0/3067026677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6340026030 con 0x55f06945a370 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63503c8740 10 client.16932 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63503c8740 10 client.16932 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63503c8740 20 client.16932 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63503c8740 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f0694d4a00 0x55f0694d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63503c8740 4 client.16932 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0693bcd70 con 0x55f0694d4a00 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63477fe640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f0694d4a00 0x55f0694d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f6326ffd640 10 client.16932 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.323+0000 7f63477fe640 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f0694d4a00 0x55f0694d6e20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16f48832c461d0f1 server_cookie=e4930d2752407ede in_seq=0 out_seq=0 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f6326ffd640 1 -- 172.21.15.16:0/3067026677 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0693bcd70 con 0x55f0694d4a00 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f6326ffd640 10 client.16932 handle_command_reply: tid=0 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 client.16932 shutdown 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 msgr2=0x55f069453860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069453860 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6340004400 tx=0x7f6340004430 comp rx=0 tx=0).stop 2024-09-15T22:45:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 shutdown_connections 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55f0694d4a00 0x55f0694d6e20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f06945a370 0x55f069453860 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f069450b40 0x55f0694628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f6326ffd640 0 client.16932 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 --2- 172.21.15.16:0/3067026677 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f06944f5d0 0x55f0694623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 >> 172.21.15.16:0/3067026677 conn(0x55f069418310 msgr2=0x55f0694587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 shutdown_connections 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 1 -- 172.21.15.16:0/3067026677 wait complete. 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f63503c8740 20 client.16932 trim_cache size 0 max 16384 2024-09-15T22:45:06.331 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63340a33b0 msgr2=0x7f63340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f634dfef640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63340a33b0 0x7f63340a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6340009fd0 tx=0x7f6340004a40 comp rx=0 tx=0).stop 2024-09-15T22:45:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 msgr2=0x7f6348198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.324+0000 7f634dfef640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f6348198990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6330006010 tx=0x7f63300042f0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 shutdown_connections 2024-09-15T22:45:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f63340a33b0 0x7f63340a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6348106f10 0x7f634819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6348102790 0x7f6348198990 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 --2- 172.21.15.16:0/2640286220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6348101de0 0x7f6348198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 >> 172.21.15.16:0/2640286220 conn(0x7f63480fd8c0 msgr2=0x7f63480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 shutdown_connections 2024-09-15T22:45:06.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:06.325+0000 7f634dfef640 1 -- 172.21.15.16:0/2640286220 wait complete. 2024-09-15T22:45:07.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:06 smithi016 ceph-mon[31492]: pgmap v203: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:45:07.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:45:07.449 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:06 smithi130 ceph-mon[35742]: pgmap v203: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:45:07.449 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:06 smithi059 ceph-mon[34872]: pgmap v203: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:45:07.721 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 -- 172.21.15.16:0/737707267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c100a80 msgr2=0x7f811c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 --2- 172.21.15.16:0/737707267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c100a80 0x7f811c100ee0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f810c009930 tx=0x7f810c02a420 comp rx=0 tx=0).stop 2024-09-15T22:45:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 -- 172.21.15.16:0/737707267 shutdown_connections 2024-09-15T22:45:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 --2- 172.21.15.16:0/737707267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811c101420 0x7f811c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 --2- 172.21.15.16:0/737707267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c100a80 0x7f811c100ee0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 --2- 172.21.15.16:0/737707267 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f811c106a80 0x7f811c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 -- 172.21.15.16:0/737707267 >> 172.21.15.16:0/737707267 conn(0x7f811c0fc7a0 msgr2=0x7f811c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 -- 172.21.15.16:0/737707267 shutdown_connections 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.295+0000 7f8124684640 1 -- 172.21.15.16:0/737707267 wait complete. 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 Processor -- start 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 -- start start 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f811c100a80 0x7f811c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811c101420 0x7f811c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c106a80 0x7f811c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f811c110e40 con 0x7f811c106a80 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f811c110cc0 con 0x7f811c101420 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8124684640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f811c110fc0 con 0x7f811c100a80 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8122c21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c106a80 0x7f811c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8122c21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c106a80 0x7f811c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55886/0 (socket says 172.21.15.16:55886) 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8122c21640 1 -- 172.21.15.16:0/1264176161 learned_addr learned my addr 172.21.15.16:0/1264176161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8121c1f640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811c101420 0x7f811c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.296+0000 7f8122420640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f811c100a80 0x7f811c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8122c21640 1 -- 172.21.15.16:0/1264176161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f811c100a80 msgr2=0x7f811c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8122c21640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f811c100a80 0x7f811c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8122c21640 1 -- 172.21.15.16:0/1264176161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811c101420 msgr2=0x7f811c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8122c21640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811c101420 0x7f811c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8122c21640 1 -- 172.21.15.16:0/1264176161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c19f610 con 0x7f811c106a80 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8122420640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f811c100a80 0x7f811c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8122c21640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c106a80 0x7f811c19ef30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f811400eaa0 tx=0x7f811400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f81137fe640 1 -- 172.21.15.16:0/1264176161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811400cdb0 con 0x7f811c106a80 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811c19f900 con 0x7f811c106a80 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f81137fe640 1 -- 172.21.15.16:0/1264176161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8114005020 con 0x7f811c106a80 2024-09-15T22:45:08.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f81137fe640 1 -- 172.21.15.16:0/1264176161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81140116b0 con 0x7f811c106a80 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.297+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811c1a71e0 con 0x7f811c106a80 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.299+0000 7f81137fe640 1 -- 172.21.15.16:0/1264176161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8114005560 con 0x7f811c106a80 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.299+0000 7f8126a5d740 1 Processor -- start 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8126a5d740 1 -- start start 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823a0b40 0x563a8239f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a8239fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8126a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563a823052f0 con 0x563a823aa370 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8126a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563a822accf0 con 0x563a823a0b40 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8126a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563a823012b0 con 0x563a8239f5d0 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823ac760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55894/0 (socket says 172.21.15.16:55894) 2024-09-15T22:45:08.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 -- 172.21.15.16:0/347102693 learned_addr learned my addr 172.21.15.16:0/347102693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823a0b40 0x563a8239f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8122420640 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a8239fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f81137fe640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f81080a33b0 0x7f81080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8122c21640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f81080a33b0 0x7f81080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 -- 172.21.15.16:0/347102693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 msgr2=0x563a8239fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a8239fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 -- 172.21.15.16:0/347102693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823a0b40 msgr2=0x563a8239f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823a0b40 0x563a8239f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.300+0000 7f8121c1f640 1 -- 172.21.15.16:0/347102693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a822913a0 con 0x563a823aa370 2024-09-15T22:45:08.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.301+0000 7f81137fe640 1 -- 172.21.15.16:0/1264176161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f81140d0300 con 0x7f811c106a80 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.301+0000 7f8121c1f640 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823ac760 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f810c0096c0 tx=0x7f810c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcff350d3a01cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.301+0000 7f8110ff9640 1 -- 172.21.15.16:0/347102693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c035070 con 0x563a823aa370 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.301+0000 7f8122c21640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f81080a33b0 0x7f81080a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8114009270 tx=0x7f81140bc040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.301+0000 7f8110ff9640 1 -- 172.21.15.16:0/347102693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c005020 con 0x563a823aa370 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.301+0000 7f8110ff9640 1 -- 172.21.15.16:0/347102693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c004a00 con 0x563a823aa370 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- 172.21.15.16:0/347102693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 msgr2=0x563a823ac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823ac760 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f810c0096c0 tx=0x7f810c02a420 comp rx=0 tx=0).stop 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- 172.21.15.16:0/347102693 shutdown_connections 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823ac760 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a8239fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 --2- 172.21.15.16:0/347102693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823a0b40 0x563a8239f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- 172.21.15.16:0/347102693 >> 172.21.15.16:0/347102693 conn(0x563a82368310 msgr2=0x563a8239bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- 172.21.15.16:0/347102693 shutdown_connections 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- 172.21.15.16:0/347102693 wait complete. 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- start start 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 Processor -- start 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823a7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563a823ad010 con 0x563a823a0b40 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563a822c0060 con 0x563a823aa370 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.302+0000 7f8126a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563a823ad190 con 0x563a8239f5d0 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8126a5d740 10 client.0 _ll_register_callbacks cb 0x563a820138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55908/0 (socket says 172.21.15.16:55908) 2024-09-15T22:45:08.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 -- 172.21.15.16:0/1289680313 learned_addr learned my addr 172.21.15.16:0/1289680313 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8121c1f640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8121c1f640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 -- 172.21.15.16:0/1289680313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 msgr2=0x563a823a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 -- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 msgr2=0x563a823a7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823a7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 -- 172.21.15.16:0/1289680313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a820763c0 con 0x563a823a0b40 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8121c1f640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823a7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8122420640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a7020 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8118010750 tx=0x7f8118010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6855302dc048e8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8105ffb640 1 -- 172.21.15.16:0/1289680313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811801d060 con 0x563a823a0b40 2024-09-15T22:45:08.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8105ffb640 1 -- 172.21.15.16:0/1289680313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f811800eda0 con 0x563a823a0b40 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.303+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810c005020 con 0x563a823a0b40 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8105ffb640 1 -- 172.21.15.16:0/1289680313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8118024050 con 0x563a823a0b40 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 10 client.16944 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a823b8b10 con 0x563a823a0b40 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8105ffb640 1 -- 172.21.15.16:0/1289680313 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f810c005020 con 0x563a823a0b40 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8105ffb640 1 -- 172.21.15.16:0/1289680313 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f81180164f0 con 0x563a823a0b40 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 10 client.16944 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 20 client.16944 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a823ba020 0x563a823bc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 4 client.16944 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a82312860 con 0x563a823ba020 2024-09-15T22:45:08.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.304+0000 7f8121c1f640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a823ba020 0x563a823bc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.305+0000 7f8105ffb640 10 client.16944 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.305+0000 7f8121c1f640 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a823ba020 0x563a823bc440 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845ec58342797430 server_cookie=28776a53cdee2371 in_seq=0 out_seq=0 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.305+0000 7f8105ffb640 1 -- 172.21.15.16:0/1289680313 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a82312860 con 0x563a823ba020 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.305+0000 7f8105ffb640 10 client.16944 handle_command_reply: tid=0 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.305+0000 7f8126a5d740 1 client.16944 shutdown 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 msgr2=0x563a823a7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a7020 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8118010750 tx=0x7f8118010c10 comp rx=0 tx=0).stop 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 shutdown_connections 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a823ba020 0x563a823bc440 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823a7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a7020 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 --2- 172.21.15.16:0/1289680313 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8105ffb640 0 client.16944 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 >> 172.21.15.16:0/1289680313 conn(0x563a82368310 msgr2=0x563a823a1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 shutdown_connections 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 1 -- 172.21.15.16:0/1289680313 wait complete. 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.306+0000 7f8126a5d740 20 client.16944 trim_cache size 0 max 16384 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 Processor -- start 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 -- start start 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1b3a30 con 0x563a823a0b40 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1b1300 con 0x563a823aa370 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8126a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1af150 con 0x563a8239f5d0 2024-09-15T22:45:08.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55916/0 (socket says 172.21.15.16:55916) 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 -- 172.21.15.16:0/2504267302 learned_addr learned my addr 172.21.15.16:0/2504267302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8121c1f640 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8121c1f640 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 -- 172.21.15.16:0/2504267302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 msgr2=0x563a823b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 -- 172.21.15.16:0/2504267302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 msgr2=0x563a823c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823c1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.319+0000 7f8122420640 1 -- 172.21.15.16:0/2504267302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8239c960 con 0x563a823a0b40 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8122420640 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a6020 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8118016ab0 tx=0x7f811800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458ebe5aa6aca39b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8107fff640 1 -- 172.21.15.16:0/2504267302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118010e60 con 0x563a823a0b40 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8107fff640 1 -- 172.21.15.16:0/2504267302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118016d30 con 0x563a823a0b40 2024-09-15T22:45:08.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8107fff640 1 -- 172.21.15.16:0/2504267302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118028890 con 0x563a823a0b40 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8126a5d740 1 -- 172.21.15.16:0/2504267302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 msgr2=0x563a823a6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a6020 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8118016ab0 tx=0x7f811800bed0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8126a5d740 1 -- 172.21.15.16:0/2504267302 shutdown_connections 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a823aa370 0x563a823c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823a0b40 0x563a823a6020 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2504267302 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a8239f5d0 0x563a823b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.320+0000 7f8126a5d740 1 -- 172.21.15.16:0/2504267302 >> 172.21.15.16:0/2504267302 conn(0x563a82368310 msgr2=0x563a823a5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 -- 172.21.15.16:0/2504267302 shutdown_connections 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 -- 172.21.15.16:0/2504267302 wait complete. 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 -- start start 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 Processor -- start 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a8239f5d0 0x563a823b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a823a0b40 0x563a823a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1af7c0 con 0x563a823aa370 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1b0c30 con 0x563a8239f5d0 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1afe90 con 0x563a823a0b40 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8126a5d740 10 client.0 _ll_register_callbacks cb 0x7f811800b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8121c1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8121c1f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823a40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55928/0 (socket says 172.21.15.16:55928) 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8121c1f640 1 -- 172.21.15.16:0/2793788876 learned_addr learned my addr 172.21.15.16:0/2793788876 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.321+0000 7f8121c1f640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a8239f5d0 0x563a823b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8122420640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a823a0b40 0x563a823a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8121c1f640 1 -- 172.21.15.16:0/2793788876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a823a0b40 msgr2=0x563a823a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8121c1f640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a823a0b40 0x563a823a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8121c1f640 1 -- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a8239f5d0 msgr2=0x563a823b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8121c1f640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a8239f5d0 0x563a823b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8121c1f640 1 -- 172.21.15.16:0/2793788876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a823a2120 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8122420640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a823a0b40 0x563a823a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8121c1f640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823a40d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f810c033000 tx=0x7f811800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa73b5ba7b9d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8105ffb640 1 -- 172.21.15.16:0/2793788876 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c035040 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8105ffb640 1 -- 172.21.15.16:0/2793788876 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9379+0+0 (secure 0 0 0) 0x7f810c004970 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8118016d30 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8105ffb640 1 -- 172.21.15.16:0/2793788876 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f810c065050 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8126a5d740 10 client.16950 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a82424920 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.322+0000 7f8105ffb640 1 -- 172.21.15.16:0/2793788876 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8118016d30 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8105ffb640 1 -- 172.21.15.16:0/2793788876 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f810c03d980 con 0x563a823aa370 2024-09-15T22:45:08.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8126a5d740 10 client.16950 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8126a5d740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8126a5d740 20 client.16950 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a82424bf0 0x563a82426fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8126a5d740 4 client.16950 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a8230cd70 con 0x563a82424bf0 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8122420640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a82424bf0 0x563a82426fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8122420640 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a82424bf0 0x563a82426fe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1693e12f32fabf0b server_cookie=59a508bc5d2e5e8a in_seq=0 out_seq=0 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.323+0000 7f8105ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8105ffb640 1 -- 172.21.15.16:0/2793788876 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8118036020 con 0x563a82424bf0 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8105ffb640 10 client.16950 handle_command_reply: tid=0 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 client.16950 shutdown 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 msgr2=0x563a823a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823a40d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f810c033000 tx=0x7f811800bc50 comp rx=0 tx=0).stop 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 shutdown_connections 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563a82424bf0 0x563a82426fe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8105ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a823aa370 0x563a823a40d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563a823a0b40 0x563a823a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 --2- 172.21.15.16:0/2793788876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563a8239f5d0 0x563a823b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 >> 172.21.15.16:0/2793788876 conn(0x563a82368310 msgr2=0x563a8239a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 shutdown_connections 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 1 -- 172.21.15.16:0/2793788876 wait complete. 2024-09-15T22:45:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8126a5d740 20 client.16950 trim_cache size 0 max 16384 2024-09-15T22:45:08.343 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f81080a33b0 msgr2=0x7f81080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8124684640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f81080a33b0 0x7f81080a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8114009270 tx=0x7f81140bc040 comp rx=0 tx=0).stop 2024-09-15T22:45:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c106a80 msgr2=0x7f811c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.324+0000 7f8124684640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c106a80 0x7f811c19ef30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f811400eaa0 tx=0x7f811400ef60 comp rx=0 tx=0).stop 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 shutdown_connections 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f811c106a80 0x7f811c19ef30 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f811c101420 0x7f811c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f81080a33b0 0x7f81080a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 --2- 172.21.15.16:0/1264176161 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f811c100a80 0x7f811c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 >> 172.21.15.16:0/1264176161 conn(0x7f811c0fc7a0 msgr2=0x7f811c06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 shutdown_connections 2024-09-15T22:45:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:08.325+0000 7f8124684640 1 -- 172.21.15.16:0/1264176161 wait complete. 2024-09-15T22:45:09.092 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:08 smithi016 ceph-mon[31492]: pgmap v204: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:45:09.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:08 smithi059 ceph-mon[34872]: pgmap v204: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:45:09.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:08 smithi130 ceph-mon[35742]: pgmap v204: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:45:10.123 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:45:10.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:45:10.152 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:09 smithi016 ceph-mon[31492]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:45:10.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:09 smithi059 ceph-mon[34872]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:45:10.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:09 smithi130 ceph-mon[35742]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:45:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.890+0000 7f5df49d2640 1 -- 172.21.15.16:0/51475443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df010c170 msgr2=0x7f5df010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.890+0000 7f5df49d2640 1 --2- 172.21.15.16:0/51475443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df010c170 0x7f5df010e610 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009ca0 tx=0x7f5de802a950 comp rx=0 tx=0).stop 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.890+0000 7f5df49d2640 1 -- 172.21.15.16:0/51475443 shutdown_connections 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.890+0000 7f5df49d2640 1 --2- 172.21.15.16:0/51475443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df010c170 0x7f5df010e610 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.890+0000 7f5df49d2640 1 --2- 172.21.15.16:0/51475443 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0103660 0x7f5df010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.890+0000 7f5df49d2640 1 --2- 172.21.15.16:0/51475443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df0102d40 0x7f5df0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.890+0000 7f5df49d2640 1 -- 172.21.15.16:0/51475443 >> 172.21.15.16:0/51475443 conn(0x7f5df00fc7a0 msgr2=0x7f5df00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 -- 172.21.15.16:0/51475443 shutdown_connections 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 -- 172.21.15.16:0/51475443 wait complete. 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 Processor -- start 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 -- start start 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df0102d40 0x7f5df01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0103660 0x7f5df0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df010c170 0x7f5df0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.891+0000 7f5df49d2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0111f70 con 0x7f5df0102d40 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5df49d2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0111df0 con 0x7f5df010c170 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5df49d2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01120f0 con 0x7f5df0103660 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df0102d40 0x7f5df01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df0102d40 0x7f5df01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55946/0 (socket says 172.21.15.16:55946) 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 -- 172.21.15.16:0/4064540516 learned_addr learned my addr 172.21.15.16:0/4064540516 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:10.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5de7fff640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0103660 0x7f5df0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5deed9d640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df010c170 0x7f5df0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 -- 172.21.15.16:0/4064540516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0103660 msgr2=0x7f5df0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0103660 0x7f5df0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 -- 172.21.15.16:0/4064540516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df010c170 msgr2=0x7f5df0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df010c170 0x7f5df0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 -- 172.21.15.16:0/4064540516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df01a67a0 con 0x7f5df0102d40 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5deed9d640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df010c170 0x7f5df0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5dee59c640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df0102d40 0x7f5df01009a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c920 tx=0x7f5de000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5de77fe640 1 -- 172.21.15.16:0/4064540516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0007c70 con 0x7f5df0102d40 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5de77fe640 1 -- 172.21.15.16:0/4064540516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de0007e10 con 0x7f5df0102d40 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df01a6a90 con 0x7f5df0102d40 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5de77fe640 1 -- 172.21.15.16:0/4064540516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0018890 con 0x7f5df0102d40 2024-09-15T22:45:10.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.892+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01a6fd0 con 0x7f5df0102d40 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5de77fe640 1 -- 172.21.15.16:0/4064540516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5de0018a70 con 0x7f5df0102d40 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 Processor -- start 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 -- start start 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b62800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b62d40 0x559533b65130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559533abd280 con 0x559533b57b60 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559533a64b10 con 0x559533b58480 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5df6dab740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559533ab9240 con 0x559533b62d40 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5deed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5deed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b57f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55956/0 (socket says 172.21.15.16:55956) 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.895+0000 7f5deed9d640 1 -- 172.21.15.16:0/3356621090 learned_addr learned my addr 172.21.15.16:0/3356621090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:10.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5dee59c640 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b62800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5de7fff640 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b62d40 0x559533b65130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5de77fe640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5deed9d640 1 -- 172.21.15.16:0/3356621090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b62d40 msgr2=0x559533b65130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5deed9d640 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b62d40 0x559533b65130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5deed9d640 1 -- 172.21.15.16:0/3356621090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 msgr2=0x559533b62800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5deed9d640 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b62800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5deed9d640 1 -- 172.21.15.16:0/3356621090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559533a711f0 con 0x559533b57b60 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5de77fe640 1 -- 172.21.15.16:0/4064540516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9555+0+0 (secure 0 0 0) 0x7f5de00d0280 con 0x7f5df0102d40 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5deed9d640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5deed9d640 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b57f40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5de8006010 tx=0x7f5de802f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14faf2327a7a3145 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5de4ff9640 1 -- 172.21.15.16:0/3356621090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8040040 con 0x559533b57b60 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5de4ff9640 1 -- 172.21.15.16:0/3356621090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de8004520 con 0x559533b57b60 2024-09-15T22:45:10.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.896+0000 7f5de4ff9640 1 -- 172.21.15.16:0/3356621090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8034070 con 0x559533b57b60 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 -- 172.21.15.16:0/3356621090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 msgr2=0x559533b57f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b57f40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5de8006010 tx=0x7f5de802f9c0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5deed9d640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5de804b000 tx=0x7f5de804c000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 -- 172.21.15.16:0/3356621090 shutdown_connections 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b62d40 0x559533b65130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b62800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3356621090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b57f40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 -- 172.21.15.16:0/3356621090 >> 172.21.15.16:0/3356621090 conn(0x559533b540f0 msgr2=0x559533b544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 -- 172.21.15.16:0/3356621090 shutdown_connections 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.897+0000 7f5df6dab740 1 -- 172.21.15.16:0/3356621090 wait complete. 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 -- start start 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 Processor -- start 2024-09-15T22:45:10.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b57b60 0x559533b5fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b60190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b62d40 0x559533b606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559533b65910 con 0x559533b62d40 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559533a77f10 con 0x559533b58480 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559533b65a90 con 0x559533b57b60 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5df6dab740 10 client.0 _ll_register_callbacks cb 0x559533a40a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b62d40 0x559533b606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b62d40 0x559533b606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55958/0 (socket says 172.21.15.16:55958) 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 -- 172.21.15.16:0/2206872908 learned_addr learned my addr 172.21.15.16:0/2206872908 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5dee59c640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b57b60 0x559533b5fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5de7fff640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b60190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 -- 172.21.15.16:0/2206872908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b57b60 msgr2=0x559533b5fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b57b60 0x559533b5fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 -- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 msgr2=0x559533b60190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b60190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5deed9d640 1 -- 172.21.15.16:0/2206872908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953382e6a0 con 0x559533b62d40 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5dee59c640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b57b60 0x559533b5fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.898+0000 7f5de7fff640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b60190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5deed9d640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b62d40 0x559533b606d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5de802fc20 tx=0x7f5de802aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5d3ee9602772c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/2206872908 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8057070 con 0x559533b62d40 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/2206872908 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9555+0+0 (secure 0 0 0) 0x7f5de8004040 con 0x559533b62d40 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de8004520 con 0x559533b62d40 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/2206872908 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5de8063050 con 0x559533b62d40 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5df6dab740 10 client.16962 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559533b65910 con 0x559533b62d40 2024-09-15T22:45:10.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/2206872908 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de8004520 con 0x559533b62d40 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/2206872908 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5de804f6e0 con 0x559533b62d40 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5df6dab740 10 client.16962 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5df6dab740 10 client.16962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.899+0000 7f5df6dab740 20 client.16962 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.900+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b726c0 0x559533b74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.900+0000 7f5df6dab740 4 client.16962 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.900+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559533aca7f0 con 0x559533b726c0 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.900+0000 7f5dee59c640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b726c0 0x559533b74ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.900+0000 7f5dee59c640 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b726c0 0x559533b74ab0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12d75d91a75725e server_cookie=12ae510999db9e0f in_seq=0 out_seq=0 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.900+0000 7f5dd5ffb640 10 client.16962 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/2206872908 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559533aca7f0 con 0x559533b726c0 2024-09-15T22:45:10.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5dd5ffb640 10 client.16962 handle_command_reply: tid=0 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 client.16962 shutdown 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b62d40 msgr2=0x559533b606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b62d40 0x559533b606d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5de802fc20 tx=0x7f5de802aef0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 shutdown_connections 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b726c0 0x559533b74ab0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b62d40 0x559533b606d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b58480 0x559533b60190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2206872908 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b57b60 0x559533b5fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5dd5ffb640 0 client.16962 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 >> 172.21.15.16:0/2206872908 conn(0x559533b540f0 msgr2=0x559533b53270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 shutdown_connections 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 1 -- 172.21.15.16:0/2206872908 wait complete. 2024-09-15T22:45:10.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.901+0000 7f5df6dab740 20 client.16962 trim_cache size 0 max 16384 2024-09-15T22:45:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 Processor -- start 2024-09-15T22:45:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 -- start start 2024-09-15T22:45:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b540f0 0x559533b7a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b57b60 0x559533b7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b58480 0x559533b5ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01b3b00 con 0x559533b58480 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01b13d0 con 0x559533b57b60 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5df6dab740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01af240 con 0x559533b540f0 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5deed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b58480 0x559533b5ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5deed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b58480 0x559533b5ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55974/0 (socket says 172.21.15.16:55974) 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.919+0000 7f5deed9d640 1 -- 172.21.15.16:0/2537508088 learned_addr learned my addr 172.21.15.16:0/2537508088 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5dee59c640 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b540f0 0x559533b7a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5de7fff640 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b57b60 0x559533b7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5deed9d640 1 -- 172.21.15.16:0/2537508088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b540f0 msgr2=0x559533b7a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5deed9d640 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b540f0 0x559533b7a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5deed9d640 1 -- 172.21.15.16:0/2537508088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b57b60 msgr2=0x559533b7a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5deed9d640 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b57b60 0x559533b7a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5deed9d640 1 -- 172.21.15.16:0/2537508088 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559533b65470 con 0x559533b58480 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5de7fff640 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b57b60 0x559533b7a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5deed9d640 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b58480 0x559533b5ceb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009a60 tx=0x7f5de802b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf9215cfc16ed50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5dd7fff640 1 -- 172.21.15.16:0/2537508088 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8060020 con 0x559533b58480 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5dd7fff640 1 -- 172.21.15.16:0/2537508088 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de802fe90 con 0x559533b58480 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.920+0000 7f5dd7fff640 1 -- 172.21.15.16:0/2537508088 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8057040 con 0x559533b58480 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- 172.21.15.16:0/2537508088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b58480 msgr2=0x559533b5ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b58480 0x559533b5ceb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5de8009a60 tx=0x7f5de802b0b0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- 172.21.15.16:0/2537508088 shutdown_connections 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b58480 0x559533b5ceb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b57b60 0x559533b7a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 --2- 172.21.15.16:0/2537508088 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b540f0 0x559533b7a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- 172.21.15.16:0/2537508088 >> 172.21.15.16:0/2537508088 conn(0x559533b54ca0 msgr2=0x559533b61bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- 172.21.15.16:0/2537508088 shutdown_connections 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- 172.21.15.16:0/2537508088 wait complete. 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- start start 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 Processor -- start 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b540f0 0x559533b6b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b58480 0x559533b67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01af890 con 0x559533b57b60 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01b0d00 con 0x559533b540f0 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01aff60 con 0x559533b58480 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.921+0000 7f5df6dab740 10 client.0 _ll_register_callbacks cb 0x7f5de0098470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:10.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55982/0 (socket says 172.21.15.16:55982) 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 -- 172.21.15.16:0/3924028630 learned_addr learned my addr 172.21.15.16:0/3924028630 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5deed9d640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b58480 0x559533b67ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5dee59c640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b540f0 0x559533b6b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 -- 172.21.15.16:0/3924028630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b58480 msgr2=0x559533b67ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b58480 0x559533b67ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 -- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b540f0 msgr2=0x559533b6b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b540f0 0x559533b6b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 -- 172.21.15.16:0/3924028630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559533b680d0 con 0x559533b57b60 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5deed9d640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b58480 0x559533b67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5dee59c640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b540f0 0x559533b6b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5de7fff640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b675a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x559533b55470 tx=0x7f5dd8010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630c5d9b757b4f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/3924028630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd800fb50 con 0x559533b57b60 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/3924028630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9555+0+0 (secure 0 0 0) 0x7f5dd800fcf0 con 0x559533b57b60 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de802fe90 con 0x559533b57b60 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/3924028630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dd8026050 con 0x559533b57b60 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5df6dab740 10 client.16968 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.922+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559533bde040 con 0x559533b57b60 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/3924028630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de802fe90 con 0x559533b57b60 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/3924028630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5dd801b400 con 0x559533b57b60 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5df6dab740 10 client.16968 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5df6dab740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5df6dab740 20 client.16968 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b62d40 0x559533be0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5df6dab740 4 client.16968 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559533ac4d00 con 0x559533b62d40 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.923+0000 7f5dee59c640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b62d40 0x559533be0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.924+0000 7f5dd5ffb640 10 client.16968 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.924+0000 7f5dee59c640 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b62d40 0x559533be0850 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de635d69a189b33f server_cookie=4d5129a1e1c544bc in_seq=0 out_seq=0 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5dd5ffb640 1 -- 172.21.15.16:0/3924028630 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559533ac4d00 con 0x559533b62d40 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5dd5ffb640 10 client.16968 handle_command_reply: tid=0 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 client.16968 shutdown 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 msgr2=0x559533b675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b675a0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x559533b55470 tx=0x7f5dd8010b90 comp rx=0 tx=0).stop 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 shutdown_connections 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559533b62d40 0x559533be0850 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559533b58480 0x559533b67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559533b57b60 0x559533b675a0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 --2- 172.21.15.16:0/3924028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559533b540f0 0x559533b6b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5dd5ffb640 0 client.16968 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 >> 172.21.15.16:0/3924028630 conn(0x559533b54ca0 msgr2=0x559533b595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 shutdown_connections 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 1 -- 172.21.15.16:0/3924028630 wait complete. 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.925+0000 7f5df6dab740 20 client.16968 trim_cache size 0 max 16384 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "56ef5a2f-5d55-4c8c-8695-910bf05824ff", 2024-09-15T22:45:10.934 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ddc0a33b0 msgr2=0x7f5ddc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5de804b000 tx=0x7f5de804c000 comp rx=0 tx=0).stop 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df0102d40 msgr2=0x7f5df01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df0102d40 0x7f5df01009a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5de000c920 tx=0x7f5de000cde0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 shutdown_connections 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5df010c170 0x7f5df0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f5ddc0a33b0 0x7f5ddc0a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f5df0103660 0x7f5df0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 --2- 172.21.15.16:0/4064540516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5df0102d40 0x7f5df01009a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 >> 172.21.15.16:0/4064540516 conn(0x7f5df00fc7a0 msgr2=0x7f5df00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 shutdown_connections 2024-09-15T22:45:10.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:10.926+0000 7f5df49d2640 1 -- 172.21.15.16:0/4064540516 wait complete. 2024-09-15T22:45:11.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:10 smithi016 ceph-mon[31492]: pgmap v206: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:45:11.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:10 smithi059 ceph-mon[34872]: pgmap v206: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:45:11.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:10 smithi130 ceph-mon[35742]: pgmap v206: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:45:11.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:45:12.034 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:12.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:11 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:45:12.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:11 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:12.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:11 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:45:12.153 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:11 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:12.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:11 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:45:12.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:11 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:12.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:11 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:45:12.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:11 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:12.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:11 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:45:12.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:11 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:12.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:11 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:45:12.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:11 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:12.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 -- 172.21.15.16:0/1818502468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141036e0 msgr2=0x7f3714107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 --2- 172.21.15.16:0/1818502468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141036e0 0x7f3714107b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3708009930 tx=0x7f370802a440 comp rx=0 tx=0).stop 2024-09-15T22:45:12.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 -- 172.21.15.16:0/1818502468 shutdown_connections 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 --2- 172.21.15.16:0/1818502468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37141082a0 0x7f371410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 --2- 172.21.15.16:0/1818502468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141036e0 0x7f3714107b90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 --2- 172.21.15.16:0/1818502468 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3714102d30 0x7f3714103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 -- 172.21.15.16:0/1818502468 >> 172.21.15.16:0/1818502468 conn(0x7f37140fc7d0 msgr2=0x7f37140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 -- 172.21.15.16:0/1818502468 shutdown_connections 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 -- 172.21.15.16:0/1818502468 wait complete. 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 Processor -- start 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.505+0000 7f371c36d640 1 -- start start 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371c36d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3714102d30 0x7f3714198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371c36d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37141036e0 0x7f3714198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371c36d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141082a0 0x7f371419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371c36d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37141120a0 con 0x7f37141082a0 2024-09-15T22:45:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371c36d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3714111f20 con 0x7f37141036e0 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371c36d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f3714112220 con 0x7f3714102d30 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141082a0 0x7f371419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141082a0 0x7f371419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35778/0 (socket says 172.21.15.16:35778) 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 -- 172.21.15.16:0/2528871548 learned_addr learned my addr 172.21.15.16:0/2528871548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a109640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3714102d30 0x7f3714198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f3719908640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37141036e0 0x7f3714198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 -- 172.21.15.16:0/2528871548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3714102d30 msgr2=0x7f3714198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3714102d30 0x7f3714198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 -- 172.21.15.16:0/2528871548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37141036e0 msgr2=0x7f3714198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37141036e0 0x7f3714198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 -- 172.21.15.16:0/2528871548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371419d500 con 0x7f37141082a0 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f3719908640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37141036e0 0x7f3714198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f371a90a640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141082a0 0x7f371419ce20 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f370400cce0 tx=0x7f370400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.506+0000 7f37137fe640 1 -- 172.21.15.16:0/2528871548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370400f450 con 0x7f37141082a0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.507+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371419d7f0 con 0x7f37141082a0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.507+0000 7f37137fe640 1 -- 172.21.15.16:0/2528871548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370400fad0 con 0x7f37141082a0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.507+0000 7f37137fe640 1 -- 172.21.15.16:0/2528871548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3704018b60 con 0x7f37141082a0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.507+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37141a50d0 con 0x7f37141082a0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.509+0000 7f37137fe640 1 -- 172.21.15.16:0/2528871548 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3704018d00 con 0x7f37141082a0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.509+0000 7f371e746740 1 Processor -- start 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.509+0000 7f371e746740 1 -- start start 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371e746740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c732374880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371e746740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371e746740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c73237f680 0x55c732381a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371e746740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c7322da2e0 con 0x55c732374dc0 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371e746740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c732281ce0 con 0x55c73237f680 2024-09-15T22:45:12.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371e746740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c7322d62a0 con 0x55c7323744a0 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f37137fe640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f36fc0a33b0 0x7f36fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35784/0 (socket says 172.21.15.16:35784) 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 -- 172.21.15.16:0/3671080689 learned_addr learned my addr 172.21.15.16:0/3671080689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f3719908640 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c732374880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a90a640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f36fc0a33b0 0x7f36fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f37137fe640 1 -- 172.21.15.16:0/2528871548 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9555+0+0 (secure 0 0 0) 0x7f37040d06c0 con 0x7f37141082a0 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 -- 172.21.15.16:0/3671080689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 msgr2=0x55c732374880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c732374880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f3719908640 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c73237f680 0x55c732381a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 -- 172.21.15.16:0/3671080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c73237f680 msgr2=0x55c732381a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c73237f680 0x55c732381a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f371a109640 1 -- 172.21.15.16:0/3671080689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c732266440 con 0x55c732374dc0 2024-09-15T22:45:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.510+0000 7f3719908640 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c73237f680 0x55c732381a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.511+0000 7f371a109640 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237f140 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f370c00ad00 tx=0x7f370c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8323d7713f9d6bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.511+0000 7f371a90a640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f36fc0a33b0 0x7f36fc0a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f370409c4c0 tx=0x7f370406e2b0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.511+0000 7f3710dea640 1 -- 172.21.15.16:0/3671080689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c01d070 con 0x55c732374dc0 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.511+0000 7f3710dea640 1 -- 172.21.15.16:0/3671080689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c018b20 con 0x55c732374dc0 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.511+0000 7f3710dea640 1 -- 172.21.15.16:0/3671080689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c010bc0 con 0x55c732374dc0 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 -- 172.21.15.16:0/3671080689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 msgr2=0x55c73237f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237f140 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f370c00ad00 tx=0x7f370c00f790 comp rx=0 tx=0).stop 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 -- 172.21.15.16:0/3671080689 shutdown_connections 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c73237f680 0x55c732381a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237f140 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 --2- 172.21.15.16:0/3671080689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c732374880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 -- 172.21.15.16:0/3671080689 >> 172.21.15.16:0/3671080689 conn(0x55c732370a30 msgr2=0x55c732370e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 -- 172.21.15.16:0/3671080689 shutdown_connections 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 -- 172.21.15.16:0/3671080689 wait complete. 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 -- start start 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 Processor -- start 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73237ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.512+0000 7f371e746740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371e746740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c73237f680 0x55c73237c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371e746740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c732382250 con 0x55c732374dc0 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371e746740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c732295050 con 0x55c7323744a0 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371e746740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c7323823d0 con 0x55c73237f680 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371e746740 10 client.0 _ll_register_callbacks cb 0x55c7322d5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371a109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371a109640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35796/0 (socket says 172.21.15.16:35796) 2024-09-15T22:45:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.513+0000 7f371a109640 1 -- 172.21.15.16:0/3000874429 learned_addr learned my addr 172.21.15.16:0/3000874429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f3719908640 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73237ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371a109640 1 -- 172.21.15.16:0/3000874429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c73237f680 msgr2=0x55c73237c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f3719908640 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c73237f680 0x55c73237c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371a109640 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c73237f680 0x55c73237c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371a109640 1 -- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 msgr2=0x55c73237ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371a109640 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73237ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371a109640 1 -- 172.21.15.16:0/3000874429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73204b3c0 con 0x55c732374dc0 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371a109640 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237c220 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f370c00a550 tx=0x7f370c0186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0eb64f2f8b6fe71 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f36f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f36f9ffb640 1 -- 172.21.15.16:0/3000874429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c01d040 con 0x55c732374dc0 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f36f9ffb640 1 -- 172.21.15.16:0/3000874429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9555+0+0 (secure 0 0 0) 0x7f370c0108e0 con 0x55c732374dc0 2024-09-15T22:45:12.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f370c018b20 con 0x55c732374dc0 2024-09-15T22:45:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f36f9ffb640 1 -- 172.21.15.16:0/3000874429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f370c031050 con 0x55c732374dc0 2024-09-15T22:45:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371e746740 10 client.16980 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.514+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c73238bb20 con 0x55c732374dc0 2024-09-15T22:45:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f36f9ffb640 1 -- 172.21.15.16:0/3000874429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c018b20 con 0x55c732374dc0 2024-09-15T22:45:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f36f9ffb640 1 -- 172.21.15.16:0/3000874429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f370c020030 con 0x55c732374dc0 2024-09-15T22:45:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f371e746740 10 client.16980 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:12.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f371e746740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f371e746740 20 client.16980 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f371e746740 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c73238cfe0 0x55c73238d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f371e746740 4 client.16980 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7322e7850 con 0x55c73238cfe0 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.515+0000 7f3719908640 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c73238cfe0 0x55c73238d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.516+0000 7f36f9ffb640 10 client.16980 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.516+0000 7f3719908640 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c73238cfe0 0x55c73238d3e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eed6cdb6f583883 server_cookie=106e4db0b1a41906 in_seq=0 out_seq=0 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.516+0000 7f36f9ffb640 1 -- 172.21.15.16:0/3000874429 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7322e7850 con 0x55c73238cfe0 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.516+0000 7f36f9ffb640 10 client.16980 handle_command_reply: tid=0 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.516+0000 7f371e746740 1 client.16980 shutdown 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.516+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 msgr2=0x55c73237c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.516+0000 7f371e746740 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237c220 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f370c00a550 tx=0x7f370c0186f0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 shutdown_connections 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c73238cfe0 0x55c73238d3e0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c73237f680 0x55c73237c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c732374dc0 0x55c73237c220 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 --2- 172.21.15.16:0/3000874429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73237ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f36f9ffb640 0 client.16980 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 >> 172.21.15.16:0/3000874429 conn(0x55c732370a30 msgr2=0x55c732376710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 shutdown_connections 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 1 -- 172.21.15.16:0/3000874429 wait complete. 2024-09-15T22:45:12.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.517+0000 7f371e746740 20 client.16980 trim_cache size 0 max 16384 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 Processor -- start 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 -- start start 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73239f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c732374dc0 0x55c73239f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c732379480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37141af810 con 0x55c73237f680 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37141ad0e0 con 0x55c7323744a0 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371e746740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37141aaf50 con 0x55c732374dc0 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f3719908640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c732379480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f3719908640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c732379480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35804/0 (socket says 172.21.15.16:35804) 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f3719908640 1 -- 172.21.15.16:0/3225829917 learned_addr learned my addr 172.21.15.16:0/3225829917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f371a109640 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c732374dc0 0x55c73239f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.534+0000 7f3719908640 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73239f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f3719908640 1 -- 172.21.15.16:0/3225829917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c732374dc0 msgr2=0x55c73239f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f3719908640 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c732374dc0 0x55c73239f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f3719908640 1 -- 172.21.15.16:0/3225829917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 msgr2=0x55c73239f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f3719908640 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73239f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f3719908640 1 -- 172.21.15.16:0/3225829917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73237f2e0 con 0x55c73237f680 2024-09-15T22:45:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f3719908640 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c732379480 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3708009900 tx=0x7f3708009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3966cba8339094 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f36fbfff640 1 -- 172.21.15.16:0/3225829917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708035040 con 0x55c73237f680 2024-09-15T22:45:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f36fbfff640 1 -- 172.21.15.16:0/3225829917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370804b030 con 0x55c73237f680 2024-09-15T22:45:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.535+0000 7f36fbfff640 1 -- 172.21.15.16:0/3225829917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708004b00 con 0x55c73237f680 2024-09-15T22:45:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 -- 172.21.15.16:0/3225829917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 msgr2=0x55c732379480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c732379480 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3708009900 tx=0x7f3708009260 comp rx=0 tx=0).stop 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 -- 172.21.15.16:0/3225829917 shutdown_connections 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c732379480 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c732374dc0 0x55c73239f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 --2- 172.21.15.16:0/3225829917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7323744a0 0x55c73239f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 -- 172.21.15.16:0/3225829917 >> 172.21.15.16:0/3225829917 conn(0x55c732370a30 msgr2=0x55c73237b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 -- 172.21.15.16:0/3225829917 shutdown_connections 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 -- 172.21.15.16:0/3225829917 wait complete. 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 -- start start 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.536+0000 7f371e746740 1 Processor -- start 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371e746740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c7323862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371e746740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c732374dc0 0x55c732386820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371e746740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c7323fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371e746740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f37141ab5a0 con 0x55c73237f680 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371e746740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37141aca10 con 0x55c732374dc0 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371e746740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f37141abc70 con 0x55c7323744a0 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371e746740 10 client.0 _ll_register_callbacks cb 0x7f370c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c7323fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c7323fa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35818/0 (socket says 172.21.15.16:35818) 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 -- 172.21.15.16:0/1533085207 learned_addr learned my addr 172.21.15.16:0/1533085207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371a109640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c732374dc0 0x55c732386820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c7323862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 -- 172.21.15.16:0/1533085207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 msgr2=0x55c7323862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c7323862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 -- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c732374dc0 msgr2=0x55c732386820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c732374dc0 0x55c732386820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 -- 172.21.15.16:0/1533085207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7323fa8b0 con 0x55c73237f680 2024-09-15T22:45:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f371a109640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c732374dc0 0x55c732386820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f3719908640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c7323fa2c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f370802a960 tx=0x7f370803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff3e7e8ac2ebdc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.537+0000 7f36f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f36f9ffb640 1 -- 172.21.15.16:0/1533085207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3708035070 con 0x55c73237f680 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f36f9ffb640 1 -- 172.21.15.16:0/1533085207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9555+0+0 (secure 0 0 0) 0x7f3708004a30 con 0x55c73237f680 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f370804b030 con 0x55c73237f680 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f36f9ffb640 1 -- 172.21.15.16:0/1533085207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3708064050 con 0x55c73237f680 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 10 client.16986 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c732401c40 con 0x55c73237f680 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f36f9ffb640 1 -- 172.21.15.16:0/1533085207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370804b030 con 0x55c73237f680 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f36f9ffb640 1 -- 172.21.15.16:0/1533085207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f370c019ce0 con 0x55c73237f680 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 10 client.16986 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:12.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 20 client.16986 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c732402230 0x55c732404650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 4 client.16986 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7322e1d60 con 0x55c732402230 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.538+0000 7f371a109640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c732402230 0x55c732404650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371a109640 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c732402230 0x55c732404650 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf19b48ec50af2c server_cookie=dd2ef5e9776d055d in_seq=0 out_seq=0 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f36f9ffb640 10 client.16986 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f36f9ffb640 1 -- 172.21.15.16:0/1533085207 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c7322e1d60 con 0x55c732402230 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f36f9ffb640 10 client.16986 handle_command_reply: tid=0 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 client.16986 shutdown 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 msgr2=0x55c7323fa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c7323fa2c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f370802a960 tx=0x7f370803f040 comp rx=0 tx=0).stop 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 shutdown_connections 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c732402230 0x55c732404650 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c73237f680 0x55c7323fa2c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c732374dc0 0x55c732386820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f36f9ffb640 0 client.16986 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 --2- 172.21.15.16:0/1533085207 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7323744a0 0x55c7323862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 >> 172.21.15.16:0/1533085207 conn(0x55c732370a30 msgr2=0x55c73237da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 shutdown_connections 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 1 -- 172.21.15.16:0/1533085207 wait complete. 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.539+0000 7f371e746740 20 client.16986 trim_cache size 0 max 16384 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:45:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "56ef5a2f-5d55-4c8c-8695-910bf05824ff": { 2024-09-15T22:45:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:45:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "56ef5a2f-5d55-4c8c-8695-910bf05824ff", 2024-09-15T22:45:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:45:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:45:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:45:12.547 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f36fc0a33b0 msgr2=0x7f36fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f36fc0a33b0 0x7f36fc0a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f370409c4c0 tx=0x7f370406e2b0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141082a0 msgr2=0x7f371419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141082a0 0x7f371419ce20 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f370400cce0 tx=0x7f370400a620 comp rx=0 tx=0).stop 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 shutdown_connections 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37141082a0 0x7f371419ce20 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37141036e0 0x7f3714198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f36fc0a33b0 0x7f36fc0a5870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 --2- 172.21.15.16:0/2528871548 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f3714102d30 0x7f3714198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 >> 172.21.15.16:0/2528871548 conn(0x7f37140fc7d0 msgr2=0x7f3714105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 shutdown_connections 2024-09-15T22:45:12.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:12.540+0000 7f371c36d640 1 -- 172.21.15.16:0/2528871548 wait complete. 2024-09-15T22:45:13.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:12 smithi059 ceph-mon[34872]: pgmap v207: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:13.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:12 smithi016 ceph-mon[31492]: pgmap v207: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:13.250 INFO:tasks.cephfs.filesystem:scrub status for tag:56ef5a2f-5d55-4c8c-8695-910bf05824ff - {'path': '/', 'tag': '56ef5a2f-5d55-4c8c-8695-910bf05824ff', 'options': 'recursive,force'} 2024-09-15T22:45:13.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:12 smithi130 ceph-mon[35742]: pgmap v207: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:13 smithi059 ceph-mon[34872]: pgmap v208: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:14.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:13 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:45:14.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:13 smithi130 ceph-mon[35742]: pgmap v208: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:14.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:13 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:45:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:13 smithi016 ceph-mon[31492]: pgmap v208: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:14.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:13 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:45:15.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:15 smithi059 ceph-mon[34872]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:45:15.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:15 smithi130 ceph-mon[35742]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:45:15.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:15 smithi016 ceph-mon[31492]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:45:16.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:16 smithi059 ceph-mon[34872]: pgmap v210: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-15T22:45:16.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:16 smithi130 ceph-mon[35742]: pgmap v210: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-15T22:45:16.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:16 smithi016 ceph-mon[31492]: pgmap v210: 129 pgs: 129 active+clean; 21 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 127 B/s wr, 4 op/s 2024-09-15T22:45:18.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:18 smithi059 ceph-mon[34872]: pgmap v211: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 247 B/s wr, 7 op/s 2024-09-15T22:45:18.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:18 smithi130 ceph-mon[35742]: pgmap v211: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 247 B/s wr, 7 op/s 2024-09-15T22:45:18.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:18 smithi016 ceph-mon[31492]: pgmap v211: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 247 B/s wr, 7 op/s 2024-09-15T22:45:20.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:20 smithi016 ceph-mon[31492]: pgmap v212: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:20.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:20 smithi059 ceph-mon[34872]: pgmap v212: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:20.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:20 smithi130 ceph-mon[35742]: pgmap v212: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:45:22.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:22 smithi059 ceph-mon[34872]: pgmap v213: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:45:22.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:22 smithi130 ceph-mon[35742]: pgmap v213: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:45:22.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:22 smithi016 ceph-mon[31492]: pgmap v213: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:45:24.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:24 smithi059 ceph-mon[34872]: pgmap v214: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:45:24.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:24 smithi130 ceph-mon[35742]: pgmap v214: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:45:24.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:24 smithi016 ceph-mon[31492]: pgmap v214: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:45:26.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:26 smithi059 ceph-mon[34872]: pgmap v215: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:45:26.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:26 smithi130 ceph-mon[35742]: pgmap v215: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:45:26.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:26 smithi016 ceph-mon[31492]: pgmap v215: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:45:28.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:28 smithi059 ceph-mon[34872]: pgmap v216: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:45:28.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:28 smithi130 ceph-mon[35742]: pgmap v216: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:45:28.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:28 smithi016 ceph-mon[31492]: pgmap v216: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:45:30.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:30 smithi059 ceph-mon[34872]: pgmap v217: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:30.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:30 smithi130 ceph-mon[35742]: pgmap v217: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:30.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:30 smithi016 ceph-mon[31492]: pgmap v217: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:32.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:32 smithi059 ceph-mon[34872]: pgmap v218: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:32.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:32 smithi130 ceph-mon[35742]: pgmap v218: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:32.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:32 smithi016 ceph-mon[31492]: pgmap v218: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:34.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:34 smithi059 ceph-mon[34872]: pgmap v219: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:34.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:34 smithi130 ceph-mon[35742]: pgmap v219: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:34.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:34 smithi016 ceph-mon[31492]: pgmap v219: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:36.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:36 smithi059 ceph-mon[34872]: pgmap v220: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:36.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:36 smithi130 ceph-mon[35742]: pgmap v220: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:36.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:36 smithi016 ceph-mon[31492]: pgmap v220: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:37.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:37 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:45:37.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:37 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:45:37.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:37 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:45:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:38 smithi059 ceph-mon[34872]: pgmap v221: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:45:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:45:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:45:38.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:38 smithi130 ceph-mon[35742]: pgmap v221: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:38.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:45:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:45:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:45:38.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:38 smithi016 ceph-mon[31492]: pgmap v221: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:45:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:45:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:45:40.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:40 smithi059 ceph-mon[34872]: pgmap v222: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:40.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:40 smithi130 ceph-mon[35742]: pgmap v222: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:40.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:40 smithi016 ceph-mon[31492]: pgmap v222: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:42.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:42 smithi059 ceph-mon[34872]: pgmap v223: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:42.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:42 smithi130 ceph-mon[35742]: pgmap v223: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:42.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:42 smithi016 ceph-mon[31492]: pgmap v223: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:43.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:45:43.515 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 -- 172.21.15.16:0/28821658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80745c0 msgr2=0x7f20d80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 --2- 172.21.15.16:0/28821658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80745c0 0x7f20d80749a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c802a480 comp rx=0 tx=0).stop 2024-09-15T22:45:44.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 -- 172.21.15.16:0/28821658 shutdown_connections 2024-09-15T22:45:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 --2- 172.21.15.16:0/28821658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20d810e420 0x7f20d8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 --2- 172.21.15.16:0/28821658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20d8074ee0 0x7f20d810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 --2- 172.21.15.16:0/28821658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d80745c0 0x7f20d80749a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 -- 172.21.15.16:0/28821658 >> 172.21.15.16:0/28821658 conn(0x7f20d80fd8c0 msgr2=0x7f20d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 -- 172.21.15.16:0/28821658 shutdown_connections 2024-09-15T22:45:44.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 -- 172.21.15.16:0/28821658 wait complete. 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.123+0000 7f20de96c640 1 Processor -- start 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20de96c640 1 -- start start 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20de96c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20d80745c0 0x7f20d819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20de96c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d8074ee0 0x7f20d819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20de96c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20d810e420 0x7f20d819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20de96c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8114170 con 0x7f20d8074ee0 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20de96c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20d8113ff0 con 0x7f20d80745c0 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20de96c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81142f0 con 0x7f20d810e420 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d8074ee0 0x7f20d819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d8074ee0 0x7f20d819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40564/0 (socket says 172.21.15.16:40564) 2024-09-15T22:45:44.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 -- 172.21.15.16:0/1290230830 learned_addr learned my addr 172.21.15.16:0/1290230830 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20d7fff640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20d80745c0 0x7f20d819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20dcf09640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20d810e420 0x7f20d819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 -- 172.21.15.16:0/1290230830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20d810e420 msgr2=0x7f20d819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20d810e420 0x7f20d819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 -- 172.21.15.16:0/1290230830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20d80745c0 msgr2=0x7f20d819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20d80745c0 0x7f20d819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.124+0000 7f20cffff640 1 -- 172.21.15.16:0/1290230830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d819f620 con 0x7f20d8074ee0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.125+0000 7f20cffff640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d8074ee0 0x7f20d819abb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d950 tx=0x7f20c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.125+0000 7f20d5ffb640 1 -- 172.21.15.16:0/1290230830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0017070 con 0x7f20d8074ee0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.125+0000 7f20d5ffb640 1 -- 172.21.15.16:0/1290230830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c0009c30 con 0x7f20d8074ee0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.125+0000 7f20d5ffb640 1 -- 172.21.15.16:0/1290230830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c000f6b0 con 0x7f20d8074ee0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.125+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d819f910 con 0x7f20d8074ee0 2024-09-15T22:45:44.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.125+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d81a71f0 con 0x7f20d8074ee0 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.128+0000 7f20d5ffb640 1 -- 172.21.15.16:0/1290230830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f20c00109d0 con 0x7f20d8074ee0 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.128+0000 7f20e0d45740 1 Processor -- start 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.128+0000 7f20e0d45740 1 -- start start 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.128+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74528880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74533140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74535a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20e0d45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f7448e2e0 con 0x564f74528dc0 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20e0d45740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f74435ce0 con 0x564f745284a0 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20e0d45740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564f7448a2a0 con 0x564f74533680 2024-09-15T22:45:44.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74533140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74533140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40578/0 (socket says 172.21.15.16:40578) 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 -- 172.21.15.16:0/2865488596 learned_addr learned my addr 172.21.15.16:0/2865488596 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20dcf09640 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74528880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d5ffb640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20c40a33b0 0x7f20c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20cffff640 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74535a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 -- 172.21.15.16:0/2865488596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 msgr2=0x564f74535a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74535a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 -- 172.21.15.16:0/2865488596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 msgr2=0x564f74528880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74528880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 -- 172.21.15.16:0/2865488596 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f7441a440 con 0x564f74528dc0 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20dcf09640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20c40a33b0 0x7f20c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d5ffb640 1 -- 172.21.15.16:0/1290230830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f20c00cfd10 con 0x7f20d8074ee0 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.129+0000 7f20d7fff640 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74533140 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658eb973642924a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:44.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20ce7fc640 1 -- 172.21.15.16:0/2865488596 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8040070 con 0x564f74528dc0 2024-09-15T22:45:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20ce7fc640 1 -- 172.21.15.16:0/2865488596 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c80319f0 con 0x564f74528dc0 2024-09-15T22:45:44.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20ce7fc640 1 -- 172.21.15.16:0/2865488596 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8035040 con 0x564f74528dc0 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 -- 172.21.15.16:0/2865488596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 msgr2=0x564f74533140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20dcf09640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20c40a33b0 0x7f20c40a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20d806b460 tx=0x7f20d0004a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74533140 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c8005e80 comp rx=0 tx=0).stop 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 -- 172.21.15.16:0/2865488596 shutdown_connections 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74535a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74533140 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2865488596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74528880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 -- 172.21.15.16:0/2865488596 >> 172.21.15.16:0/2865488596 conn(0x564f74524a30 msgr2=0x564f74524e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:44.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 -- 172.21.15.16:0/2865488596 shutdown_connections 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.130+0000 7f20e0d45740 1 -- 172.21.15.16:0/2865488596 wait complete. 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 -- start start 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 Processor -- start 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74532f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74530200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74530740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564f74536250 con 0x564f74528dc0 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f74449050 con 0x564f745284a0 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564f745363d0 con 0x564f74533680 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20e0d45740 10 client.0 _ll_register_callbacks cb 0x564f74489160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74530200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74530200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40590/0 (socket says 172.21.15.16:40590) 2024-09-15T22:45:44.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.131+0000 7f20cffff640 1 -- 172.21.15.16:0/131986456 learned_addr learned my addr 172.21.15.16:0/131986456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20d7fff640 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74532f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20dcf09640 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74530740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20cffff640 1 -- 172.21.15.16:0/131986456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 msgr2=0x564f74530740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20cffff640 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74530740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20cffff640 1 -- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 msgr2=0x564f74532f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20cffff640 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74532f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20cffff640 1 -- 172.21.15.16:0/131986456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f741ff3c0 con 0x564f74528dc0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20cffff640 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74530200 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009610 tx=0x7f20c009b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78819c5928b4338 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20b77fe640 1 -- 172.21.15.16:0/131986456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0020070 con 0x564f74528dc0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20b77fe640 1 -- 172.21.15.16:0/131986456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f20c009b0e0 con 0x564f74528dc0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c80319f0 con 0x564f74528dc0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20b77fe640 1 -- 172.21.15.16:0/131986456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20c00cd4d0 con 0x564f74528dc0 2024-09-15T22:45:44.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20e0d45740 10 client.16998 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f74541ba0 con 0x564f74528dc0 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.132+0000 7f20b77fe640 1 -- 172.21.15.16:0/131986456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c80319f0 con 0x564f74528dc0 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20b77fe640 1 -- 172.21.15.16:0/131986456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20c00ba2b0 con 0x564f74528dc0 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20e0d45740 10 client.16998 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20e0d45740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20e0d45740 20 client.16998 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20e0d45740 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f74543060 0x564f745454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20e0d45740 4 client.16998 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f7449b850 con 0x564f74543060 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.133+0000 7f20d7fff640 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f74543060 0x564f745454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.134+0000 7f20d7fff640 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f74543060 0x564f745454b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6c9ba2d70a14ef server_cookie=4623bc560fb5c167 in_seq=0 out_seq=0 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.134+0000 7f20b77fe640 10 client.16998 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:44.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.134+0000 7f20b77fe640 1 -- 172.21.15.16:0/131986456 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f7449b850 con 0x564f74543060 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.134+0000 7f20b77fe640 10 client.16998 handle_command_reply: tid=0 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.134+0000 7f20e0d45740 1 client.16998 shutdown 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 msgr2=0x564f74530200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74530200 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009610 tx=0x7f20c009b8d0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 shutdown_connections 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f74543060 0x564f745454b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74533680 0x564f74530740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f74530200 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 --2- 172.21.15.16:0/131986456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f745284a0 0x564f74532f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 >> 172.21.15.16:0/131986456 conn(0x564f74524a30 msgr2=0x564f7452a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20b77fe640 0 client.16998 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 shutdown_connections 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 1 -- 172.21.15.16:0/131986456 wait complete. 2024-09-15T22:45:44.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.135+0000 7f20e0d45740 20 client.16998 trim_cache size 0 max 16384 2024-09-15T22:45:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 Processor -- start 2024-09-15T22:45:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 -- start start 2024-09-15T22:45:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f745284a0 0x564f7453f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74528dc0 0x564f7454ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7454b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81b3b70 con 0x564f745284a0 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81b1440 con 0x564f74533680 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20e0d45740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81af2b0 con 0x564f74528dc0 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f745284a0 0x564f7453f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f745284a0 0x564f7453f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40600/0 (socket says 172.21.15.16:40600) 2024-09-15T22:45:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20d7fff640 1 -- 172.21.15.16:0/2266040545 learned_addr learned my addr 172.21.15.16:0/2266040545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.152+0000 7f20cffff640 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74528dc0 0x564f7454ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20d7fff640 1 -- 172.21.15.16:0/2266040545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74528dc0 msgr2=0x564f7454ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20d7fff640 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74528dc0 0x564f7454ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20dcf09640 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7454b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20d7fff640 1 -- 172.21.15.16:0/2266040545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 msgr2=0x564f7454b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20d7fff640 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7454b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20d7fff640 1 -- 172.21.15.16:0/2266040545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f74524860 con 0x564f745284a0 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20dcf09640 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7454b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20d7fff640 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f745284a0 0x564f7453f4b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808547c25a415373 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20cdffb640 1 -- 172.21.15.16:0/2266040545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8040040 con 0x564f745284a0 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20cdffb640 1 -- 172.21.15.16:0/2266040545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c8005520 con 0x564f745284a0 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.153+0000 7f20cdffb640 1 -- 172.21.15.16:0/2266040545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8035040 con 0x564f745284a0 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 -- 172.21.15.16:0/2266040545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f745284a0 msgr2=0x564f7453f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f745284a0 0x564f7453f4b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009900 tx=0x7f20c802a480 comp rx=0 tx=0).stop 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 -- 172.21.15.16:0/2266040545 shutdown_connections 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7454b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f74528dc0 0x564f7454ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 --2- 172.21.15.16:0/2266040545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f745284a0 0x564f7453f4b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 -- 172.21.15.16:0/2266040545 >> 172.21.15.16:0/2266040545 conn(0x564f74524a30 msgr2=0x564f7452f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 -- 172.21.15.16:0/2266040545 shutdown_connections 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 -- 172.21.15.16:0/2266040545 wait complete. 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.154+0000 7f20e0d45740 1 -- start start 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 1 Processor -- start 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f745284a0 0x564f7453bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f7452ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7452d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81af900 con 0x564f74528dc0 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81b0d70 con 0x564f74533680 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f20d81affd0 con 0x564f745284a0 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20e0d45740 10 client.0 _ll_register_callbacks cb 0x7f20c000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f7452ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20cffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f7452ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40616/0 (socket says 172.21.15.16:40616) 2024-09-15T22:45:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20cffff640 1 -- 172.21.15.16:0/3913781306 learned_addr learned my addr 172.21.15.16:0/3913781306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20d7fff640 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f745284a0 0x564f7453bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.155+0000 7f20dcf09640 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7452d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20cffff640 1 -- 172.21.15.16:0/3913781306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f745284a0 msgr2=0x564f7453bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20cffff640 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f745284a0 0x564f7453bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20cffff640 1 -- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 msgr2=0x564f7452d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20cffff640 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7452d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20cffff640 1 -- 172.21.15.16:0/3913781306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f7452dac0 con 0x564f74528dc0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20cffff640 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f7452ce70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f20c00860b0 tx=0x7f20c0097900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6270cdde2f6bef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20b77fe640 1 -- 172.21.15.16:0/3913781306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00b2020 con 0x564f74528dc0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20b77fe640 1 -- 172.21.15.16:0/3913781306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f20c0097be0 con 0x564f74528dc0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20c8005520 con 0x564f74528dc0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20b77fe640 1 -- 172.21.15.16:0/3913781306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20c009ba60 con 0x564f74528dc0 2024-09-15T22:45:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20e0d45740 10 client.17004 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f745adc10 con 0x564f74528dc0 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.156+0000 7f20b77fe640 1 -- 172.21.15.16:0/3913781306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c8005520 con 0x564f74528dc0 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20b77fe640 1 -- 172.21.15.16:0/3913781306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20c009ab50 con 0x564f74528dc0 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20e0d45740 10 client.17004 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20e0d45740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20e0d45740 20 client.17004 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20e0d45740 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f745ae3e0 0x564f745b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20e0d45740 4 client.17004 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:45:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f74545840 con 0x564f745ae3e0 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20d7fff640 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f745ae3e0 0x564f745b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20d7fff640 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f745ae3e0 0x564f745b0800 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c034aea2849b39d server_cookie=8ebd39a41733f35d in_seq=0 out_seq=0 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.157+0000 7f20b77fe640 10 client.17004 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20b77fe640 1 -- 172.21.15.16:0/3913781306 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564f74545840 con 0x564f745ae3e0 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20b77fe640 10 client.17004 handle_command_reply: tid=0 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 client.17004 shutdown 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 msgr2=0x564f7452ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f7452ce70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f20c00860b0 tx=0x7f20c0097900 comp rx=0 tx=0).stop 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 shutdown_connections 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564f745ae3e0 0x564f745b0800 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f74533680 0x564f7452d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564f74528dc0 0x564f7452ce70 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 --2- 172.21.15.16:0/3913781306 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564f745284a0 0x564f7453bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20b77fe640 0 client.17004 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 >> 172.21.15.16:0/3913781306 conn(0x564f74524a30 msgr2=0x564f7452afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 shutdown_connections 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 1 -- 172.21.15.16:0/3913781306 wait complete. 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.158+0000 7f20e0d45740 20 client.17004 trim_cache size 0 max 16384 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:45:44.165 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:45:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20c40a33b0 msgr2=0x7f20c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20c40a33b0 0x7f20c40a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f20d806b460 tx=0x7f20d0004a40 comp rx=0 tx=0).stop 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d8074ee0 msgr2=0x7f20d819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d8074ee0 0x7f20d819abb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f20c000d950 tx=0x7f20c000de10 comp rx=0 tx=0).stop 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 shutdown_connections 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f20c40a33b0 0x7f20c40a5870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f20d810e420 0x7f20d819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20d8074ee0 0x7f20d819abb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 --2- 172.21.15.16:0/1290230830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20d80745c0 0x7f20d819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 >> 172.21.15.16:0/1290230830 conn(0x7f20d80fd8c0 msgr2=0x7f20d80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 shutdown_connections 2024-09-15T22:45:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:44.159+0000 7f20de96c640 1 -- 172.21.15.16:0/1290230830 wait complete. 2024-09-15T22:45:44.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:44 smithi016 ceph-mon[31492]: pgmap v224: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:44.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:44 smithi059 ceph-mon[34872]: pgmap v224: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:44.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:44 smithi130 ceph-mon[35742]: pgmap v224: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:44.894 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:45:44.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:45:45.155 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:45.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 -- 172.21.15.16:0/3203029900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 msgr2=0x7fc268106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 --2- 172.21.15.16:0/3203029900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc268106e60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc260009930 tx=0x7fc26002a420 comp rx=0 tx=0).stop 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 -- 172.21.15.16:0/3203029900 shutdown_connections 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 --2- 172.21.15.16:0/3203029900 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc268101420 0x7fc26810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 --2- 172.21.15.16:0/3203029900 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc268100a80 0x7fc268100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 --2- 172.21.15.16:0/3203029900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc268106e60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 -- 172.21.15.16:0/3203029900 >> 172.21.15.16:0/3203029900 conn(0x7fc2680fc7a0 msgr2=0x7fc2680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 -- 172.21.15.16:0/3203029900 shutdown_connections 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.628+0000 7fc26f163640 1 -- 172.21.15.16:0/3203029900 wait complete. 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 Processor -- start 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 -- start start 2024-09-15T22:45:45.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc268100a80 0x7fc268198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc268101420 0x7fc2681989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc26819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc268110e80 con 0x7fc268106a80 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc268110d00 con 0x7fc268101420 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26f163640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fc268111000 con 0x7fc268100a80 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.629+0000 7fc26d700640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc26819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc26819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40626/0 (socket says 172.21.15.16:40626) 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 -- 172.21.15.16:0/3365973965 learned_addr learned my addr 172.21.15.16:0/3365973965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc267fff640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc268101420 0x7fc2681989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26ceff640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc268100a80 0x7fc268198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 -- 172.21.15.16:0/3365973965 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc268100a80 msgr2=0x7fc268198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc268100a80 0x7fc268198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 -- 172.21.15.16:0/3365973965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc268101420 msgr2=0x7fc2681989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc268101420 0x7fc2681989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 -- 172.21.15.16:0/3365973965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26819d430 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26ceff640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc268100a80 0x7fc268198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26d700640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc26819cd50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d950 tx=0x7fc25000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc265ffb640 1 -- 172.21.15.16:0/3365973965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250017030 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc265ffb640 1 -- 172.21.15.16:0/3365973965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc250004020 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26819d720 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc265ffb640 1 -- 172.21.15.16:0/3365973965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250003c00 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.630+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2681a5000 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.631+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc268100ee0 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.632+0000 7fc265ffb640 1 -- 172.21.15.16:0/3365973965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc25001a070 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.633+0000 7fc265ffb640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc2540a3430 0x7fc2540a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.633+0000 7fc265ffb640 1 -- 172.21.15.16:0/3365973965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fc2500cf900 con 0x7fc268106a80 2024-09-15T22:45:45.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.633+0000 7fc26ceff640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc2540a3430 0x7fc2540a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.634+0000 7fc26ceff640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc2540a3430 0x7fc2540a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc260000c00 tx=0x7fc260005a20 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:45.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.637+0000 7fc265ffb640 1 -- 172.21.15.16:0/3365973965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc250021070 con 0x7fc268106a80 2024-09-15T22:45:45.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.851+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2681a52e0 con 0x7fc268106a80 2024-09-15T22:45:45.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.852+0000 7fc265ffb640 1 -- 172.21.15.16:0/3365973965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fc25009bed0 con 0x7fc268106a80 2024-09-15T22:45:45.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:45:45.854 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:45:45.854 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc2540a3430 msgr2=0x7fc2540a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc2540a3430 0x7fc2540a58f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc260000c00 tx=0x7fc260005a20 comp rx=0 tx=0).stop 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 msgr2=0x7fc26819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc26819cd50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d950 tx=0x7fc25000de10 comp rx=0 tx=0).stop 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 shutdown_connections 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc268106a80 0x7fc26819cd50 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc268101420 0x7fc2681989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fc2540a3430 0x7fc2540a58f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 --2- 172.21.15.16:0/3365973965 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fc268100a80 0x7fc268198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 >> 172.21.15.16:0/3365973965 conn(0x7fc2680fc7a0 msgr2=0x7fc26806a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:45.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 shutdown_connections 2024-09-15T22:45:45.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:45.854+0000 7fc26f163640 1 -- 172.21.15.16:0/3365973965 wait complete. 2024-09-15T22:45:46.544 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:46 smithi016 ceph-mon[31492]: pgmap v225: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:46.544 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:46 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3365973965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:46.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:45:46.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:46 smithi059 ceph-mon[34872]: pgmap v225: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:46.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:46 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3365973965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:46.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:46 smithi130 ceph-mon[35742]: pgmap v225: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:46 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3365973965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:47.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 -- 172.21.15.16:0/1695217041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7841008d0 msgr2=0x7fb784100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1695217041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7841008d0 0x7fb784100cb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb774009900 tx=0x7fb77402a410 comp rx=0 tx=0).stop 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 -- 172.21.15.16:0/1695217041 shutdown_connections 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1695217041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb78410e480 0x7fb784110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1695217041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb784101280 0x7fb78410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1695217041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7841008d0 0x7fb784100cb0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 -- 172.21.15.16:0/1695217041 >> 172.21.15.16:0/1695217041 conn(0x7fb7840fc7a0 msgr2=0x7fb7840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 -- 172.21.15.16:0/1695217041 shutdown_connections 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 -- 172.21.15.16:0/1695217041 wait complete. 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.288+0000 7fb78a82c640 1 Processor -- start 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 -- start start 2024-09-15T22:45:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7841008d0 0x7fb78419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784101280 0x7fb78419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb78410e480 0x7fb78419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841141b0 con 0x7fb784101280 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb784114030 con 0x7fb7841008d0 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb784114330 con 0x7fb78410e480 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784101280 0x7fb78419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784101280 0x7fb78419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40656/0 (socket says 172.21.15.16:40656) 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 -- 172.21.15.16:0/1931151799 learned_addr learned my addr 172.21.15.16:0/1931151799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb788dc9640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb78410e480 0x7fb78419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb783fff640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7841008d0 0x7fb78419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 -- 172.21.15.16:0/1931151799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb78410e480 msgr2=0x7fb78419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb78410e480 0x7fb78419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 -- 172.21.15.16:0/1931151799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7841008d0 msgr2=0x7fb78419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7841008d0 0x7fb78419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 -- 172.21.15.16:0/1931151799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78419f680 con 0x7fb784101280 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb783fff640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7841008d0 0x7fb78419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784101280 0x7fb78419ac10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c890 tx=0x7fb76c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb781ffb640 1 -- 172.21.15.16:0/1931151799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c004220 con 0x7fb784101280 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb781ffb640 1 -- 172.21.15.16:0/1931151799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c0043c0 con 0x7fb784101280 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.289+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78419f970 con 0x7fb784101280 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.290+0000 7fb781ffb640 1 -- 172.21.15.16:0/1931151799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0049e0 con 0x7fb784101280 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.290+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7841a7250 con 0x7fb784101280 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.292+0000 7fb781ffb640 1 -- 172.21.15.16:0/1931151799 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb76c0079b0 con 0x7fb784101280 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.292+0000 7fb78cc05740 1 Processor -- start 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.292+0000 7fb78cc05740 1 -- start start 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb78cc05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563766e902f0 con 0x563766f2bb40 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb78cc05740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563766e37cf0 con 0x563766f35370 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb78cc05740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563766e8c2b0 con 0x563766f2a5d0 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40658/0 (socket says 172.21.15.16:40658) 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 -- 172.21.15.16:0/4027297492 learned_addr learned my addr 172.21.15.16:0/4027297492 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb783fff640 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb77bfff640 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 -- 172.21.15.16:0/4027297492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 msgr2=0x563766f2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 -- 172.21.15.16:0/4027297492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 msgr2=0x563766f37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 -- 172.21.15.16:0/4027297492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563766e1c3a0 con 0x563766f2bb40 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb781ffb640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7700a3430 0x7fb7700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb783fff640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7700a3430 0x7fb7700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb788dc9640 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2a090 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb77c008ce0 tx=0x7fb77c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dcc59a3ef24a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.293+0000 7fb781ffb640 1 -- 172.21.15.16:0/1931151799 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fb76c0d0200 con 0x7fb784101280 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb77a7fc640 1 -- 172.21.15.16:0/4027297492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c014070 con 0x563766f2bb40 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb77a7fc640 1 -- 172.21.15.16:0/4027297492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77c003b70 con 0x563766f2bb40 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb77a7fc640 1 -- 172.21.15.16:0/4027297492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c010700 con 0x563766f2bb40 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 -- 172.21.15.16:0/4027297492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 msgr2=0x563766f2a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb783fff640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7700a3430 0x7fb7700a58f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb774031550 tx=0x7fb774005a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2a090 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb77c008ce0 tx=0x7fb77c00e720 comp rx=0 tx=0).stop 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 -- 172.21.15.16:0/4027297492 shutdown_connections 2024-09-15T22:45:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 --2- 172.21.15.16:0/4027297492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2a090 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 -- 172.21.15.16:0/4027297492 >> 172.21.15.16:0/4027297492 conn(0x563766ef3310 msgr2=0x563766f26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.294+0000 7fb78cc05740 1 -- 172.21.15.16:0/4027297492 shutdown_connections 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 -- 172.21.15.16:0/4027297492 wait complete. 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 -- start start 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 Processor -- start 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f34cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f31ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f32600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563766f38070 con 0x563766f2bb40 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563766e4b060 con 0x563766f35370 2024-09-15T22:45:47.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563766f381f0 con 0x563766f2a5d0 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb78cc05740 10 client.0 _ll_register_callbacks cb 0x563766b9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.295+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f31ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f31ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40664/0 (socket says 172.21.15.16:40664) 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 -- 172.21.15.16:0/1516661849 learned_addr learned my addr 172.21.15.16:0/1516661849 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb788dc9640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f32600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb783fff640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f34cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 -- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 msgr2=0x563766f34cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f34cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 -- 172.21.15.16:0/1516661849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 msgr2=0x563766f32600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f32600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 -- 172.21.15.16:0/1516661849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563766c013c0 con 0x563766f2bb40 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb788dc9640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f32600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:47.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f31ff0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb76c000c00 tx=0x7fb76c00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7037d1cf01eebc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb7637fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb7637fe640 1 -- 172.21.15.16:0/1516661849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c01d030 con 0x563766f2bb40 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb7637fe640 1 -- 172.21.15.16:0/1516661849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fb76c0cd7c0 con 0x563766f2bb40 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.296+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77c003b70 con 0x563766f2bb40 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb7637fe640 1 -- 172.21.15.16:0/1516661849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb76c0cd9c0 con 0x563766f2bb40 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 10 client.17022 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563766f43b40 con 0x563766f2bb40 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb7637fe640 1 -- 172.21.15.16:0/1516661849 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77c003b70 con 0x563766f2bb40 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb7637fe640 1 -- 172.21.15.16:0/1516661849 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb76c0a48b0 con 0x563766f2bb40 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 10 client.17022 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 10 client.17022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 20 client.17022 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766f45000 0x563766f47410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 4 client.17022 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563766e9d860 con 0x563766f45000 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.297+0000 7fb783fff640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766f45000 0x563766f47410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.298+0000 7fb783fff640 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766f45000 0x563766f47410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ba5ce23d66a606b server_cookie=7102507e59ddbc3b in_seq=0 out_seq=0 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.298+0000 7fb7637fe640 10 client.17022 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.298+0000 7fb7637fe640 1 -- 172.21.15.16:0/1516661849 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563766e9d860 con 0x563766f45000 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.298+0000 7fb7637fe640 10 client.17022 handle_command_reply: tid=0 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.298+0000 7fb78cc05740 1 client.17022 shutdown 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 msgr2=0x563766f31ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f31ff0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb76c000c00 tx=0x7fb76c00a5f0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 shutdown_connections 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766f45000 0x563766f47410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f35370 0x563766f32600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f31ff0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1516661849 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2a5d0 0x563766f34cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb7637fe640 0 client.17022 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 >> 172.21.15.16:0/1516661849 conn(0x563766ef3310 msgr2=0x563766f2c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 shutdown_connections 2024-09-15T22:45:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 1 -- 172.21.15.16:0/1516661849 wait complete. 2024-09-15T22:45:47.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.299+0000 7fb78cc05740 20 client.17022 trim_cache size 0 max 16384 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 Processor -- start 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 -- start start 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f41310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2bb40 0x563766f4cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f35370 0x563766f4d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b3c10 con 0x563766f35370 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b14e0 con 0x563766f2a5d0 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb78cc05740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841af350 con 0x563766f2bb40 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f35370 0x563766f4d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f35370 0x563766f4d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40672/0 (socket says 172.21.15.16:40672) 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 -- 172.21.15.16:0/1161633829 learned_addr learned my addr 172.21.15.16:0/1161633829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:47.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb77bfff640 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2bb40 0x563766f4cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb783fff640 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f41310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 -- 172.21.15.16:0/1161633829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2bb40 msgr2=0x563766f4cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2bb40 0x563766f4cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 -- 172.21.15.16:0/1161633829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 msgr2=0x563766f41310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f41310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.312+0000 7fb788dc9640 1 -- 172.21.15.16:0/1161633829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563766f32c00 con 0x563766f35370 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.313+0000 7fb788dc9640 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f35370 0x563766f4d1d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb77c004860 tx=0x7fb77c0083f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec0f54ea14eb709 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.313+0000 7fb779ffb640 1 -- 172.21.15.16:0/1161633829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c028070 con 0x563766f35370 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.313+0000 7fb779ffb640 1 -- 172.21.15.16:0/1161633829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77c003b70 con 0x563766f35370 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.313+0000 7fb779ffb640 1 -- 172.21.15.16:0/1161633829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c014050 con 0x563766f35370 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- 172.21.15.16:0/1161633829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f35370 msgr2=0x563766f4d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f35370 0x563766f4d1d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb77c004860 tx=0x7fb77c0083f0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- 172.21.15.16:0/1161633829 shutdown_connections 2024-09-15T22:45:47.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f35370 0x563766f4d1d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f2bb40 0x563766f4cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 --2- 172.21.15.16:0/1161633829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f41310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- 172.21.15.16:0/1161633829 >> 172.21.15.16:0/1161633829 conn(0x563766ef3310 msgr2=0x563766f310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- 172.21.15.16:0/1161633829 shutdown_connections 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- 172.21.15.16:0/1161633829 wait complete. 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- start start 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 Processor -- start 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f2eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f35370 0x563766f2f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841af9a0 con 0x563766f2bb40 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b0e10 con 0x563766f2a5d0 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.314+0000 7fb78cc05740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb7841b0070 con 0x563766f35370 2024-09-15T22:45:47.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb78cc05740 10 client.0 _ll_register_callbacks cb 0x7fb76c0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40680/0 (socket says 172.21.15.16:40680) 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 -- 172.21.15.16:0/167808484 learned_addr learned my addr 172.21.15.16:0/167808484 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb783fff640 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f2eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb788dc9640 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f35370 0x563766f2f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 -- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f35370 msgr2=0x563766f2f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f35370 0x563766f2f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 -- 172.21.15.16:0/167808484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 msgr2=0x563766f2eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f2eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 -- 172.21.15.16:0/167808484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563766faec60 con 0x563766f2bb40 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb77bfff640 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2f430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb76c098a50 tx=0x7fb76c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b1a6d555bbe369 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb7637fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb7637fe640 1 -- 172.21.15.16:0/167808484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0b4030 con 0x563766f2bb40 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb7637fe640 1 -- 172.21.15.16:0/167808484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fb76c09b720 con 0x563766f2bb40 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77c003b70 con 0x563766f2bb40 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb7637fe640 1 -- 172.21.15.16:0/167808484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb76c09bc10 con 0x563766f2bb40 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb78cc05740 10 client.17028 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.315+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563766fafe40 con 0x563766f2bb40 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb7637fe640 1 -- 172.21.15.16:0/167808484 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77c003b70 con 0x563766f2bb40 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb7637fe640 1 -- 172.21.15.16:0/167808484 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb76c09af90 con 0x563766f2bb40 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb78cc05740 10 client.17028 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb78cc05740 10 client.17028 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb78cc05740 20 client.17028 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb78cc05740 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766fb0530 0x563766fb2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb78cc05740 4 client.17028 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563766e97d70 con 0x563766fb0530 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.316+0000 7fb783fff640 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766fb0530 0x563766fb2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb783fff640 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766fb0530 0x563766fb2950 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff2d2ffb62a16915 server_cookie=5aa7cfa193c30cc5 in_seq=0 out_seq=0 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb7637fe640 10 client.17028 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb7637fe640 1 -- 172.21.15.16:0/167808484 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563766e97d70 con 0x563766fb0530 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb7637fe640 10 client.17028 handle_command_reply: tid=0 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 client.17028 shutdown 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 msgr2=0x563766f2f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2f430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb76c098a50 tx=0x7fb76c007f30 comp rx=0 tx=0).stop 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 shutdown_connections 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x563766fb0530 0x563766fb2950 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563766f35370 0x563766f2f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563766f2bb40 0x563766f2f430 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 --2- 172.21.15.16:0/167808484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563766f2a5d0 0x563766f2eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb7637fe640 0 client.17028 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:47.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 >> 172.21.15.16:0/167808484 conn(0x563766ef3310 msgr2=0x563766f25500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 shutdown_connections 2024-09-15T22:45:47.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 1 -- 172.21.15.16:0/167808484 wait complete. 2024-09-15T22:45:47.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.317+0000 7fb78cc05740 20 client.17028 trim_cache size 0 max 16384 2024-09-15T22:45:47.327 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7700a3430 msgr2=0x7fb7700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7700a3430 0x7fb7700a58f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb774031550 tx=0x7fb774005a40 comp rx=0 tx=0).stop 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784101280 msgr2=0x7fb78419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784101280 0x7fb78419ac10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c890 tx=0x7fb76c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 shutdown_connections 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb7700a3430 0x7fb7700a58f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb78410e480 0x7fb78419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb784101280 0x7fb78419ac10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 --2- 172.21.15.16:0/1931151799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7841008d0 0x7fb78419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 >> 172.21.15.16:0/1931151799 conn(0x7fb7840fc7a0 msgr2=0x7fb7840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 shutdown_connections 2024-09-15T22:45:47.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:47.318+0000 7fb78a82c640 1 -- 172.21.15.16:0/1931151799 wait complete. 2024-09-15T22:45:48.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:45:48.388 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:48.650 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:48 smithi016 ceph-mon[31492]: pgmap v226: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:48.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:48 smithi059 ceph-mon[34872]: pgmap v226: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:48.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:48 smithi130 ceph-mon[35742]: pgmap v226: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 -- 172.21.15.16:0/3427881004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390101de0 msgr2=0x7fb3901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 --2- 172.21.15.16:0/3427881004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390101de0 0x7fb3901021c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009900 tx=0x7fb38c02a410 comp rx=0 tx=0).stop 2024-09-15T22:45:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 -- 172.21.15.16:0/3427881004 shutdown_connections 2024-09-15T22:45:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 --2- 172.21.15.16:0/3427881004 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390106f10 0x7fb39010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 --2- 172.21.15.16:0/3427881004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb390102790 0x7fb3901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 --2- 172.21.15.16:0/3427881004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390101de0 0x7fb3901021c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 -- 172.21.15.16:0/3427881004 >> 172.21.15.16:0/3427881004 conn(0x7fb3900fd8c0 msgr2=0x7fb3900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 -- 172.21.15.16:0/3427881004 shutdown_connections 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.848+0000 7fb3982a1640 1 -- 172.21.15.16:0/3427881004 wait complete. 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 Processor -- start 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 -- start start 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb390101de0 0x7fb3901985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390102790 0x7fb390198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390106f10 0x7fb39019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb390112010 con 0x7fb390102790 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb390111e90 con 0x7fb390101de0 2024-09-15T22:45:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb3982a1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb390112190 con 0x7fb390106f10 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390102790 0x7fb390198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390102790 0x7fb390198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40696/0 (socket says 172.21.15.16:40696) 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 -- 172.21.15.16:0/2543515556 learned_addr learned my addr 172.21.15.16:0/2543515556 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39603d640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb390101de0 0x7fb3901985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39683e640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390106f10 0x7fb39019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 -- 172.21.15.16:0/2543515556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390106f10 msgr2=0x7fb39019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390106f10 0x7fb39019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 -- 172.21.15.16:0/2543515556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb390101de0 msgr2=0x7fb3901985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb390101de0 0x7fb3901985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39583c640 1 -- 172.21.15.16:0/2543515556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39019d560 con 0x7fb390102790 2024-09-15T22:45:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39603d640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb390101de0 0x7fb3901985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.849+0000 7fb39683e640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390106f10 0x7fb39019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.850+0000 7fb39583c640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390102790 0x7fb390198af0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b4a0 tx=0x7fb38000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.850+0000 7fb3877fe640 1 -- 172.21.15.16:0/2543515556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380014070 con 0x7fb390102790 2024-09-15T22:45:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.850+0000 7fb3877fe640 1 -- 172.21.15.16:0/2543515556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb380010af0 con 0x7fb390102790 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.850+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39019d850 con 0x7fb390102790 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.850+0000 7fb3877fe640 1 -- 172.21.15.16:0/2543515556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380017d10 con 0x7fb390102790 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.850+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3901a5130 con 0x7fb390102790 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.852+0000 7fb3877fe640 1 -- 172.21.15.16:0/2543515556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb380017070 con 0x7fb390102790 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 Processor -- start 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 -- start start 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d371090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d371a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d382b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562b0d2d72f0 con 0x562b0d380780 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562b0d27ecf0 con 0x562b0d372b40 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39a67a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562b0d2d32b0 con 0x562b0d3715d0 2024-09-15T22:45:48.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39583c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d382b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39583c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d382b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40702/0 (socket says 172.21.15.16:40702) 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39583c640 1 -- 172.21.15.16:0/642161670 learned_addr learned my addr 172.21.15.16:0/642161670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb3877fe640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb37c0a3430 0x7fb37c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39683e640 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d371090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.853+0000 7fb39603d640 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d371a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb3877fe640 1 -- 172.21.15.16:0/2543515556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fb3800cf3c0 con 0x7fb390102790 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39683e640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb37c0a3430 0x7fb37c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39583c640 1 -- 172.21.15.16:0/642161670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 msgr2=0x562b0d371a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39583c640 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d371a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39583c640 1 -- 172.21.15.16:0/642161670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 msgr2=0x562b0d371090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39583c640 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d371090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39583c640 1 -- 172.21.15.16:0/642161670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0d2633a0 con 0x562b0d380780 2024-09-15T22:45:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39583c640 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d382b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb3800cc750 tx=0x7fb3800047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9058940a0d5e487f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb384dea640 1 -- 172.21.15.16:0/642161670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38001e070 con 0x562b0d380780 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb39683e640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb37c0a3430 0x7fb37c0a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb388013870 tx=0x7fb388004a40 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb384dea640 1 -- 172.21.15.16:0/642161670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3800cce50 con 0x562b0d380780 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.854+0000 7fb384dea640 1 -- 172.21.15.16:0/642161670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380097a10 con 0x562b0d380780 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 -- 172.21.15.16:0/642161670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 msgr2=0x562b0d382b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d382b70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb3800cc750 tx=0x7fb3800047d0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 -- 172.21.15.16:0/642161670 shutdown_connections 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d382b70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d371a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 --2- 172.21.15.16:0/642161670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d371090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 -- 172.21.15.16:0/642161670 >> 172.21.15.16:0/642161670 conn(0x562b0d33a310 msgr2=0x562b0d36dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 -- 172.21.15.16:0/642161670 shutdown_connections 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 -- 172.21.15.16:0/642161670 wait complete. 2024-09-15T22:45:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 -- start start 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.855+0000 7fb39a67a740 1 Processor -- start 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d3715d0 0x562b0d3801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d372b40 0x562b0d37d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562b0d3833c0 con 0x562b0d380780 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562b0d292060 con 0x562b0d3715d0 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562b0d383540 con 0x562b0d372b40 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 10 client.0 _ll_register_callbacks cb 0x562b0cfe58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40716/0 (socket says 172.21.15.16:40716) 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 -- 172.21.15.16:0/2111354582 learned_addr learned my addr 172.21.15.16:0/2111354582 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39583c640 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d372b40 0x562b0d37d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39603d640 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d3715d0 0x562b0d3801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 -- 172.21.15.16:0/2111354582 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d372b40 msgr2=0x562b0d37d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d372b40 0x562b0d37d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 -- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d3715d0 msgr2=0x562b0d3801f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d3715d0 0x562b0d3801f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 -- 172.21.15.16:0/2111354582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0d0483c0 con 0x562b0d380780 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39683e640 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37dbe0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb388010f70 tx=0x7fb388004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d00ec52fb4c5a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb379ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb379ffb640 1 -- 172.21.15.16:0/2111354582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388022070 con 0x562b0d380780 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb379ffb640 1 -- 172.21.15.16:0/2111354582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fb38801e800 con 0x562b0d380780 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.856+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3800cce50 con 0x562b0d380780 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb379ffb640 1 -- 172.21.15.16:0/2111354582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb38802e050 con 0x562b0d380780 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 10 client.17040 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b0d38ef50 con 0x562b0d380780 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb379ffb640 1 -- 172.21.15.16:0/2111354582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3800cce50 con 0x562b0d380780 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb379ffb640 1 -- 172.21.15.16:0/2111354582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb38801d070 con 0x562b0d380780 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 10 client.17040 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 10 client.17040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 20 client.17040 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d390360 0x562b0d392770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 4 client.17040 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b0d2e4860 con 0x562b0d390360 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.857+0000 7fb39603d640 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d390360 0x562b0d392770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.858+0000 7fb379ffb640 10 client.17040 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.858+0000 7fb39603d640 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d390360 0x562b0d392770 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c45ee421522e2e74 server_cookie=d505d97a91fdf7e8 in_seq=0 out_seq=0 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.858+0000 7fb379ffb640 1 -- 172.21.15.16:0/2111354582 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b0d2e4860 con 0x562b0d390360 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.858+0000 7fb379ffb640 10 client.17040 handle_command_reply: tid=0 2024-09-15T22:45:48.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 client.17040 shutdown 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 msgr2=0x562b0d37dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37dbe0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb388010f70 tx=0x7fb388004b50 comp rx=0 tx=0).stop 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 shutdown_connections 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d390360 0x562b0d392770 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37dbe0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d372b40 0x562b0d37d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb379ffb640 0 client.17040 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 --2- 172.21.15.16:0/2111354582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d3715d0 0x562b0d3801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 >> 172.21.15.16:0/2111354582 conn(0x562b0d33a310 msgr2=0x562b0d373990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 shutdown_connections 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 1 -- 172.21.15.16:0/2111354582 wait complete. 2024-09-15T22:45:48.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.859+0000 7fb39a67a740 20 client.17040 trim_cache size 0 max 16384 2024-09-15T22:45:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 Processor -- start 2024-09-15T22:45:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 -- start start 2024-09-15T22:45:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d372b40 0x562b0d38c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d380780 0x562b0d398040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901af940 con 0x562b0d372b40 2024-09-15T22:45:48.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901ad210 con 0x562b0d380780 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.872+0000 7fb39a67a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901ab080 con 0x562b0d3715d0 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d372b40 0x562b0d38c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d372b40 0x562b0d38c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40726/0 (socket says 172.21.15.16:40726) 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 -- 172.21.15.16:0/3626797224 learned_addr learned my addr 172.21.15.16:0/3626797224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39683e640 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d380780 0x562b0d398040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39603d640 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 -- 172.21.15.16:0/3626797224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 msgr2=0x562b0d37cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 -- 172.21.15.16:0/3626797224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d380780 msgr2=0x562b0d398040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d380780 0x562b0d398040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 -- 172.21.15.16:0/3626797224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b0d36dde0 con 0x562b0d372b40 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb39583c640 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d372b40 0x562b0d38c660 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb380097800 tx=0x7fb38009b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453344cc3d07948c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:48.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb37bfff640 1 -- 172.21.15.16:0/3626797224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38001e070 con 0x562b0d372b40 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb37bfff640 1 -- 172.21.15.16:0/3626797224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3800cce50 con 0x562b0d372b40 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.873+0000 7fb37bfff640 1 -- 172.21.15.16:0/3626797224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380017430 con 0x562b0d372b40 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 -- 172.21.15.16:0/3626797224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d372b40 msgr2=0x562b0d38c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d372b40 0x562b0d38c660 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb380097800 tx=0x7fb38009b190 comp rx=0 tx=0).stop 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 -- 172.21.15.16:0/3626797224 shutdown_connections 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d380780 0x562b0d398040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d372b40 0x562b0d38c660 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 --2- 172.21.15.16:0/3626797224 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 -- 172.21.15.16:0/3626797224 >> 172.21.15.16:0/3626797224 conn(0x562b0d33a310 msgr2=0x562b0d37c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 -- 172.21.15.16:0/3626797224 shutdown_connections 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 -- 172.21.15.16:0/3626797224 wait complete. 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.874+0000 7fb39a67a740 1 -- start start 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 1 Processor -- start 2024-09-15T22:45:48.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d37ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901ab6d0 con 0x562b0d380780 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901acb40 con 0x562b0d372b40 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb3901abda0 con 0x562b0d3715d0 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39a67a740 10 client.0 _ll_register_callbacks cb 0x7fb388013450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40728/0 (socket says 172.21.15.16:40728) 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 -- 172.21.15.16:0/1904575214 learned_addr learned my addr 172.21.15.16:0/1904575214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39603d640 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39583c640 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d37ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 -- 172.21.15.16:0/1904575214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 msgr2=0x562b0d37a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 -- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 msgr2=0x562b0d37ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d37ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.875+0000 7fb39683e640 1 -- 172.21.15.16:0/1904575214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b0d3fa170 con 0x562b0d380780 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39683e640 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37b090 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb388026b40 tx=0x7fb388007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3062066f25a15e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb379ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb379ffb640 1 -- 172.21.15.16:0/1904575214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388036020 con 0x562b0d380780 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb379ffb640 1 -- 172.21.15.16:0/1904575214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fb3880267c0 con 0x562b0d380780 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3800cce50 con 0x562b0d380780 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb379ffb640 1 -- 172.21.15.16:0/1904575214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb388041050 con 0x562b0d380780 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 10 client.17046 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b0d3fb430 con 0x562b0d380780 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb379ffb640 1 -- 172.21.15.16:0/1904575214 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3800cce50 con 0x562b0d380780 2024-09-15T22:45:48.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb379ffb640 1 -- 172.21.15.16:0/1904575214 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb38801d050 con 0x562b0d380780 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 10 client.17046 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 10 client.17046 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 20 client.17046 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d3fbae0 0x562b0d3fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 4 client.17046 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.876+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b0d2ded70 con 0x562b0d3fbae0 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39603d640 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d3fbae0 0x562b0d3fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39603d640 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d3fbae0 0x562b0d3fdf00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6143e02759546c49 server_cookie=2b319838dd3bdb15 in_seq=0 out_seq=0 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb379ffb640 10 client.17046 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb379ffb640 1 -- 172.21.15.16:0/1904575214 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b0d2ded70 con 0x562b0d3fbae0 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb379ffb640 10 client.17046 handle_command_reply: tid=0 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 client.17046 shutdown 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 msgr2=0x562b0d37b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37b090 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb388026b40 tx=0x7fb388007e20 comp rx=0 tx=0).stop 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 shutdown_connections 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562b0d3fbae0 0x562b0d3fdf00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b0d380780 0x562b0d37b090 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562b0d372b40 0x562b0d37ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb379ffb640 0 client.17046 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 --2- 172.21.15.16:0/1904575214 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562b0d3715d0 0x562b0d37a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.877+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 >> 172.21.15.16:0/1904575214 conn(0x562b0d33a310 msgr2=0x562b0d37ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 shutdown_connections 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb39a67a740 1 -- 172.21.15.16:0/1904575214 wait complete. 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb39a67a740 20 client.17046 trim_cache size 0 max 16384 2024-09-15T22:45:48.887 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb37c0a3430 msgr2=0x7fb37c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb37c0a3430 0x7fb37c0a58f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb388013870 tx=0x7fb388004a40 comp rx=0 tx=0).stop 2024-09-15T22:45:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390102790 msgr2=0x7fb390198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390102790 0x7fb390198af0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fb38000b4a0 tx=0x7fb38000b960 comp rx=0 tx=0).stop 2024-09-15T22:45:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 shutdown_connections 2024-09-15T22:45:48.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb37c0a3430 0x7fb37c0a58f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb390106f10 0x7fb39019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb390102790 0x7fb390198af0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 --2- 172.21.15.16:0/2543515556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb390101de0 0x7fb3901985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 >> 172.21.15.16:0/2543515556 conn(0x7fb3900fd8c0 msgr2=0x7fb3900fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 shutdown_connections 2024-09-15T22:45:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:48.878+0000 7fb3982a1640 1 -- 172.21.15.16:0/2543515556 wait complete. 2024-09-15T22:45:49.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:45:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:50.528 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:50 smithi016 ceph-mon[31492]: pgmap v227: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 -- 172.21.15.16:0/1939342311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 msgr2=0x7fab5410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 --2- 172.21.15.16:0/1939342311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5410e610 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009ca0 tx=0x7fab4c02a950 comp rx=0 tx=0).stop 2024-09-15T22:45:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 -- 172.21.15.16:0/1939342311 shutdown_connections 2024-09-15T22:45:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 --2- 172.21.15.16:0/1939342311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5410e610 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 --2- 172.21.15.16:0/1939342311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab54103660 0x7fab5410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 --2- 172.21.15.16:0/1939342311 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab54102d40 0x7fab54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 -- 172.21.15.16:0/1939342311 >> 172.21.15.16:0/1939342311 conn(0x7fab540fc7a0 msgr2=0x7fab540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 -- 172.21.15.16:0/1939342311 shutdown_connections 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.608+0000 7fab59e33640 1 -- 172.21.15.16:0/1939342311 wait complete. 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 Processor -- start 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 -- start start 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab54102d40 0x7fab54198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab54103660 0x7fab541988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab54111f70 con 0x7fab5410c170 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab54111df0 con 0x7fab54102d40 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab59e33640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab541120f0 con 0x7fab54103660 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40750/0 (socket says 172.21.15.16:40750) 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.609+0000 7fab53fff640 1 -- 172.21.15.16:0/3087988697 learned_addr learned my addr 172.21.15.16:0/3087988697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab54103660 0x7fab541988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab537fe640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab54102d40 0x7fab54198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab53fff640 1 -- 172.21.15.16:0/3087988697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab54103660 msgr2=0x7fab541988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab53fff640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab54103660 0x7fab541988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab53fff640 1 -- 172.21.15.16:0/3087988697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab54102d40 msgr2=0x7fab54198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab53fff640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab54102d40 0x7fab54198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab53fff640 1 -- 172.21.15.16:0/3087988697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5419d340 con 0x7fab5410c170 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab537fe640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab54102d40 0x7fab54198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab53fff640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5419cc60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fab4c006010 tx=0x7fab4c02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab50ff9640 1 -- 172.21.15.16:0/3087988697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c034040 con 0x7fab5410c170 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5419d5d0 con 0x7fab5410c170 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab50ff9640 1 -- 172.21.15.16:0/3087988697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4c003e00 con 0x7fab5410c170 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab50ff9640 1 -- 172.21.15.16:0/3087988697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0456e0 con 0x7fab5410c170 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.610+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab541a4f10 con 0x7fab5410c170 2024-09-15T22:45:50.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab50ff9640 1 -- 172.21.15.16:0/3087988697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fab4c037020 con 0x7fab5410c170 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 Processor -- start 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 -- start start 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9b8b40 0x559cca9b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9c6780 0x559cca9c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cca91d2f0 con 0x559cca9b75d0 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559cca8c4cf0 con 0x559cca9c6780 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab5c20c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559cca9192b0 con 0x559cca9b8b40 2024-09-15T22:45:50.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40760/0 (socket says 172.21.15.16:40760) 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 -- 172.21.15.16:0/3873169156 learned_addr learned my addr 172.21.15.16:0/3873169156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9b8b40 0x559cca9b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9c6780 0x559cca9c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 -- 172.21.15.16:0/3873169156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9b8b40 msgr2=0x559cca9b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9b8b40 0x559cca9b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 -- 172.21.15.16:0/3873169156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9c6780 msgr2=0x559cca9c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9c6780 0x559cca9c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab537fe640 1 -- 172.21.15.16:0/3873169156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cca8a93a0 con 0x559cca9b75d0 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.613+0000 7fab50ff9640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fab3c0a33b0 0x7fab3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.614+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fab3c0a33b0 0x7fab3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.614+0000 7fab537fe640 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9b7a10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fab4400cca0 tx=0x7fab4400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f11776beb3be32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.614+0000 7fab50ff9640 1 -- 172.21.15.16:0/3087988697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fab4c0f1090 con 0x7fab5410c170 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.614+0000 7fab39ffb640 1 -- 172.21.15.16:0/3873169156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab44005220 con 0x559cca9b75d0 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.614+0000 7fab39ffb640 1 -- 172.21.15.16:0/3873169156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab44004c80 con 0x559cca9b75d0 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.614+0000 7fab39ffb640 1 -- 172.21.15.16:0/3873169156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440108b0 con 0x559cca9b75d0 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 -- 172.21.15.16:0/3873169156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 msgr2=0x559cca9b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fab3c0a33b0 0x7fab3c0a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x559cca9b8070 tx=0x7fab4800a250 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9b7a10 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fab4400cca0 tx=0x7fab4400e720 comp rx=0 tx=0).stop 2024-09-15T22:45:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 -- 172.21.15.16:0/3873169156 shutdown_connections 2024-09-15T22:45:50.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9c6780 0x559cca9c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9b7a10 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3873169156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9b8b40 0x559cca9b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 -- 172.21.15.16:0/3873169156 >> 172.21.15.16:0/3873169156 conn(0x559cca980310 msgr2=0x559cca9b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 -- 172.21.15.16:0/3873169156 shutdown_connections 2024-09-15T22:45:50.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 -- 172.21.15.16:0/3873169156 wait complete. 2024-09-15T22:45:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 -- start start 2024-09-15T22:45:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.615+0000 7fab5c20c740 1 Processor -- start 2024-09-15T22:45:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559cca9c9480 con 0x559cca9b75d0 2024-09-15T22:45:50.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559cca8d8060 con 0x559cca9b8b40 2024-09-15T22:45:50.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559cca9c9600 con 0x559cca9c6780 2024-09-15T22:45:50.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 10 client.0 _ll_register_callbacks cb 0x559cca62b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:50.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40776/0 (socket says 172.21.15.16:40776) 2024-09-15T22:45:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 -- 172.21.15.16:0/1505740542 learned_addr learned my addr 172.21.15.16:0/1505740542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab53fff640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab52ffd640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 -- 172.21.15.16:0/1505740542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 msgr2=0x559cca9c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 -- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 msgr2=0x559cca9c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 -- 172.21.15.16:0/1505740542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cca68e3c0 con 0x559cca9b75d0 2024-09-15T22:45:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab52ffd640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab537fe640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c6170 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fab4400ec20 tx=0x7fab440054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5aa5f4df20b795 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab32ffd640 1 -- 172.21.15.16:0/1505740542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4402a060 con 0x559cca9b75d0 2024-09-15T22:45:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab32ffd640 1 -- 172.21.15.16:0/1505740542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fab440048d0 con 0x559cca9b75d0 2024-09-15T22:45:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.616+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab44004c80 con 0x559cca9b75d0 2024-09-15T22:45:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab32ffd640 1 -- 172.21.15.16:0/1505740542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab44004ad0 con 0x559cca9b75d0 2024-09-15T22:45:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 10 client.17058 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559cca9d4f20 con 0x559cca9b75d0 2024-09-15T22:45:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab32ffd640 1 -- 172.21.15.16:0/1505740542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab44004c80 con 0x559cca9b75d0 2024-09-15T22:45:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab32ffd640 1 -- 172.21.15.16:0/1505740542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fab44025500 con 0x559cca9b75d0 2024-09-15T22:45:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 10 client.17058 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 20 client.17058 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559cca9d6320 0x559cca9d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 4 client.17058 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cca92a860 con 0x559cca9d6320 2024-09-15T22:45:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.617+0000 7fab52ffd640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559cca9d6320 0x559cca9d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.618+0000 7fab32ffd640 10 client.17058 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.618+0000 7fab52ffd640 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559cca9d6320 0x559cca9d8710 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ae4b2f6edb1d64a server_cookie=13697b4396918193 in_seq=0 out_seq=0 2024-09-15T22:45:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.618+0000 7fab32ffd640 1 -- 172.21.15.16:0/1505740542 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cca92a860 con 0x559cca9d6320 2024-09-15T22:45:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.618+0000 7fab32ffd640 10 client.17058 handle_command_reply: tid=0 2024-09-15T22:45:50.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.618+0000 7fab5c20c740 1 client.17058 shutdown 2024-09-15T22:45:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 msgr2=0x559cca9c6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c6170 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fab4400ec20 tx=0x7fab440054a0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 shutdown_connections 2024-09-15T22:45:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559cca9d6320 0x559cca9d8710 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1505740542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c6170 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab32ffd640 0 client.17058 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 >> 172.21.15.16:0/1505740542 conn(0x559cca980310 msgr2=0x559cca9b9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 shutdown_connections 2024-09-15T22:45:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 1 -- 172.21.15.16:0/1505740542 wait complete. 2024-09-15T22:45:50.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.619+0000 7fab5c20c740 20 client.17058 trim_cache size 0 max 16384 2024-09-15T22:45:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 Processor -- start 2024-09-15T22:45:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 -- start start 2024-09-15T22:45:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b75d0 0x559cca9d2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b8b40 0x559cca9b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9ddd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab541af620 con 0x559cca9b8b40 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab541acef0 con 0x559cca9b75d0 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab5c20c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab541aad60 con 0x559cca9c6780 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b8b40 0x559cca9b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b8b40 0x559cca9b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40786/0 (socket says 172.21.15.16:40786) 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 -- 172.21.15.16:0/3488433941 learned_addr learned my addr 172.21.15.16:0/3488433941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab537fe640 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b75d0 0x559cca9d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 -- 172.21.15.16:0/3488433941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 msgr2=0x559cca9ddd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab53fff640 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9ddd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9ddd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 -- 172.21.15.16:0/3488433941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b75d0 msgr2=0x559cca9d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b75d0 0x559cca9d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab52ffd640 1 -- 172.21.15.16:0/3488433941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cca9b4960 con 0x559cca9b8b40 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.632+0000 7fab537fe640 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b75d0 0x559cca9d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.633+0000 7fab53fff640 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9ddd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.633+0000 7fab52ffd640 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b8b40 0x559cca9b2bf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x559cca9c6750 tx=0x7fab48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf4c6c7367cac38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.633+0000 7fab397fa640 1 -- 172.21.15.16:0/3488433941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4801d990 con 0x559cca9b8b40 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.633+0000 7fab397fa640 1 -- 172.21.15.16:0/3488433941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4801db30 con 0x559cca9b8b40 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.633+0000 7fab397fa640 1 -- 172.21.15.16:0/3488433941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4801e630 con 0x559cca9b8b40 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.633+0000 7fab5c20c740 1 -- 172.21.15.16:0/3488433941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b8b40 msgr2=0x559cca9b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.633+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b8b40 0x559cca9b2bf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x559cca9c6750 tx=0x7fab48004730 comp rx=0 tx=0).stop 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- 172.21.15.16:0/3488433941 shutdown_connections 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9ddd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b8b40 0x559cca9b2bf0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 --2- 172.21.15.16:0/3488433941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b75d0 0x559cca9d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- 172.21.15.16:0/3488433941 >> 172.21.15.16:0/3488433941 conn(0x559cca980310 msgr2=0x559cca9b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- 172.21.15.16:0/3488433941 shutdown_connections 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- 172.21.15.16:0/3488433941 wait complete. 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- start start 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 Processor -- start 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab541ab3b0 con 0x559cca9b75d0 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fab541ac820 con 0x559cca9b8b40 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fab541aba80 con 0x559cca9c6780 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.634+0000 7fab5c20c740 10 client.0 _ll_register_callbacks cb 0x7fab44014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40790/0 (socket says 172.21.15.16:40790) 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 -- 172.21.15.16:0/1457236386 learned_addr learned my addr 172.21.15.16:0/1457236386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab53fff640 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab52ffd640 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 -- 172.21.15.16:0/1457236386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 msgr2=0x559cca9c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 -- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 msgr2=0x559cca9c08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:50.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 -- 172.21.15.16:0/1457236386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cca9d2c60 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab537fe640 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c03a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fab44005d70 tx=0x7fab44025700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114f390d0b18782a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab32ffd640 1 -- 172.21.15.16:0/1457236386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4400ec50 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab32ffd640 1 -- 172.21.15.16:0/1457236386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fab44005600 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4801db30 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab32ffd640 1 -- 172.21.15.16:0/1457236386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab44040050 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab5c20c740 10 client.17064 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.635+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ccaa40cd0 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab32ffd640 1 -- 172.21.15.16:0/1457236386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4801db30 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab32ffd640 1 -- 172.21.15.16:0/1457236386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fab440035f0 con 0x559cca9b75d0 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab5c20c740 10 client.17064 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab5c20c740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab5c20c740 20 client.17064 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ccaa41400 0x559ccaa43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab5c20c740 4 client.17064 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cca924d70 con 0x559ccaa41400 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.636+0000 7fab52ffd640 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ccaa41400 0x559ccaa43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab32ffd640 10 client.17064 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab52ffd640 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ccaa41400 0x559ccaa43820 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761e094d0372d8a9 server_cookie=649d82ecaf888d50 in_seq=0 out_seq=0 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab32ffd640 1 -- 172.21.15.16:0/1457236386 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cca924d70 con 0x559ccaa41400 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab32ffd640 10 client.17064 handle_command_reply: tid=0 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 client.17064 shutdown 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 msgr2=0x559cca9c03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c03a0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fab44005d70 tx=0x7fab44025700 comp rx=0 tx=0).stop 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 shutdown_connections 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559ccaa41400 0x559ccaa43820 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab32ffd640 0 client.17064 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559cca9c6780 0x559cca9c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559cca9b8b40 0x559cca9c08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 --2- 172.21.15.16:0/1457236386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559cca9b75d0 0x559cca9c03a0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 >> 172.21.15.16:0/1457236386 conn(0x559cca980310 msgr2=0x559cca9b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 shutdown_connections 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 1 -- 172.21.15.16:0/1457236386 wait complete. 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.637+0000 7fab5c20c740 20 client.17064 trim_cache size 0 max 16384 2024-09-15T22:45:50.659 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fab3c0a33b0 msgr2=0x7fab3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fab3c0a33b0 0x7fab3c0a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x559cca9b8070 tx=0x7fab4800a250 comp rx=0 tx=0).stop 2024-09-15T22:45:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 msgr2=0x7fab5419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5419cc60 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fab4c006010 tx=0x7fab4c02f710 comp rx=0 tx=0).stop 2024-09-15T22:45:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 shutdown_connections 2024-09-15T22:45:50.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab5410c170 0x7fab5419cc60 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fab3c0a33b0 0x7fab3c0a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fab54103660 0x7fab541988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 --2- 172.21.15.16:0/3087988697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab54102d40 0x7fab54198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 >> 172.21.15.16:0/3087988697 conn(0x7fab540fc7a0 msgr2=0x7fab540fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 shutdown_connections 2024-09-15T22:45:50.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:50.638+0000 7fab59e33640 1 -- 172.21.15.16:0/3087988697 wait complete. 2024-09-15T22:45:50.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:50 smithi059 ceph-mon[34872]: pgmap v227: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:50.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:50 smithi130 ceph-mon[35742]: pgmap v227: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:52.445 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:45:52.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:45:52.696 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:52 smithi059 ceph-mon[34872]: pgmap v228: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:52.713 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:52.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:52 smithi130 ceph-mon[35742]: pgmap v228: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:52.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:52 smithi016 ceph-mon[31492]: pgmap v228: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 -- 172.21.15.16:0/445523807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4102ef0 msgr2=0x7f35e410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 --2- 172.21.15.16:0/445523807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4102ef0 0x7f35e410da40 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009900 tx=0x7f35dc02a480 comp rx=0 tx=0).stop 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 -- 172.21.15.16:0/445523807 shutdown_connections 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 --2- 172.21.15.16:0/445523807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4102ef0 0x7f35e410da40 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 --2- 172.21.15.16:0/445523807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35e4102570 0x7f35e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 --2- 172.21.15.16:0/445523807 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f35e4100e20 0x7f35e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 -- 172.21.15.16:0/445523807 >> 172.21.15.16:0/445523807 conn(0x7f35e40fc7d0 msgr2=0x7f35e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 -- 172.21.15.16:0/445523807 shutdown_connections 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.177+0000 7f35e883c640 1 -- 172.21.15.16:0/445523807 wait complete. 2024-09-15T22:45:53.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 Processor -- start 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 -- start start 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4100e20 0x7f35e4197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35e4102570 0x7f35e4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f35e4102ef0 0x7f35e419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4110f80 con 0x7f35e4100e20 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4110e00 con 0x7f35e4102570 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.178+0000 7f35e883c640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f35e4111100 con 0x7f35e4102ef0 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4100e20 0x7f35e4197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4100e20 0x7f35e4197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44742/0 (socket says 172.21.15.16:44742) 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 -- 172.21.15.16:0/4103677217 learned_addr learned my addr 172.21.15.16:0/4103677217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f35e4102ef0 0x7f35e419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35e4102570 0x7f35e4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 -- 172.21.15.16:0/4103677217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f35e4102ef0 msgr2=0x7f35e419bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f35e4102ef0 0x7f35e419bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 -- 172.21.15.16:0/4103677217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35e4102570 msgr2=0x7f35e4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35e4102570 0x7f35e4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 -- 172.21.15.16:0/4103677217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e419c300 con 0x7f35e4100e20 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e259c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4100e20 0x7f35e4197350 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f35d400ea80 tx=0x7f35d400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35d37fe640 1 -- 172.21.15.16:0/4103677217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d400ce40 con 0x7f35e4100e20 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e419c5f0 con 0x7f35e4100e20 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35d37fe640 1 -- 172.21.15.16:0/4103677217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d4005020 con 0x7f35e4100e20 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35d37fe640 1 -- 172.21.15.16:0/4103677217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4011760 con 0x7f35e4100e20 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.179+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e41a3ed0 con 0x7f35e4100e20 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.181+0000 7f35d37fe640 1 -- 172.21.15.16:0/4103677217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f35d40051d0 con 0x7f35e4100e20 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 Processor -- start 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 -- start start 2024-09-15T22:45:53.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b739dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b73a300 0x559e9b746a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b746f50 0x559e9b749340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e9b69f280 con 0x559e9b7399e0 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e9b646b10 con 0x559e9b73a300 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35d37fe640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f35cc0a33b0 0x7f35cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.182+0000 7f35eac15740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559e9b69b240 con 0x559e9b746f50 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b739dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b739dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44748/0 (socket says 172.21.15.16:44748) 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e2d9d640 1 -- 172.21.15.16:0/1747278902 learned_addr learned my addr 172.21.15.16:0/1747278902 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b746f50 0x559e9b749340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35d37fe640 1 -- 172.21.15.16:0/4103677217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f35d40d0760 con 0x7f35e4100e20 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f35cc0a33b0 0x7f35cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e259c640 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b73a300 0x559e9b746a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e1d9b640 1 -- 172.21.15.16:0/1747278902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b73a300 msgr2=0x559e9b746a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b73a300 0x559e9b746a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e1d9b640 1 -- 172.21.15.16:0/1747278902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 msgr2=0x559e9b739dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b739dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e1d9b640 1 -- 172.21.15.16:0/1747278902 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9b6531f0 con 0x559e9b746f50 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b746f50 0x559e9b749340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35d800bbb0 tx=0x7f35d800f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b577bcfd3a55211c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.183+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f35cc0a33b0 0x7f35cc0a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0096c0 tx=0x7f35dc009260 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35d0ff9640 1 -- 172.21.15.16:0/1747278902 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8015070 con 0x559e9b746f50 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35d0ff9640 1 -- 172.21.15.16:0/1747278902 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d800fd90 con 0x559e9b746f50 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35d0ff9640 1 -- 172.21.15.16:0/1747278902 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8011440 con 0x559e9b746f50 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35eac15740 1 -- 172.21.15.16:0/1747278902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b746f50 msgr2=0x559e9b749340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35eac15740 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b746f50 0x559e9b749340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f35d800bbb0 tx=0x7f35d800f360 comp rx=0 tx=0).stop 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35eac15740 1 -- 172.21.15.16:0/1747278902 shutdown_connections 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35eac15740 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b746f50 0x559e9b749340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35eac15740 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b73a300 0x559e9b746a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35eac15740 1 --2- 172.21.15.16:0/1747278902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b739dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.184+0000 7f35eac15740 1 -- 172.21.15.16:0/1747278902 >> 172.21.15.16:0/1747278902 conn(0x559e9b7360f0 msgr2=0x559e9b7364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 -- 172.21.15.16:0/1747278902 shutdown_connections 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 -- 172.21.15.16:0/1747278902 wait complete. 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 -- start start 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 Processor -- start 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b742560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b742aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b754cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e9b73caa0 con 0x559e9b746f50 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e9b659f10 con 0x559e9b7399e0 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559e9b73cc20 con 0x559e9b73a300 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35eac15740 10 client.0 _ll_register_callbacks cb 0x559e9b622a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:53.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b754cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b754cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44752/0 (socket says 172.21.15.16:44752) 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35e2d9d640 1 -- 172.21.15.16:0/3051695886 learned_addr learned my addr 172.21.15.16:0/3051695886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.185+0000 7f35e259c640 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b742560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b742aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35e2d9d640 1 -- 172.21.15.16:0/3051695886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 msgr2=0x559e9b742aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b742aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35e2d9d640 1 -- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 msgr2=0x559e9b742560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b742560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35e2d9d640 1 -- 172.21.15.16:0/3051695886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9b4106a0 con 0x559e9b746f50 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b754cd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02a9a0 tx=0x7f35dc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94b1bb17ee4a110 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:53.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35c9ffb640 1 -- 172.21.15.16:0/3051695886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc051070 con 0x559e9b746f50 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35c9ffb640 1 -- 172.21.15.16:0/3051695886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f35dc031750 con 0x559e9b746f50 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d800fd90 con 0x559e9b746f50 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35c9ffb640 1 -- 172.21.15.16:0/3051695886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35dc035040 con 0x559e9b746f50 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35eac15740 10 client.17076 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.186+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e9b7579c0 con 0x559e9b746f50 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35c9ffb640 1 -- 172.21.15.16:0/3051695886 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d800fd90 con 0x559e9b746f50 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35c9ffb640 1 -- 172.21.15.16:0/3051695886 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35dc04bb20 con 0x559e9b746f50 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35eac15740 10 client.17076 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35eac15740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35eac15740 20 client.17076 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:53.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35eac15740 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b758e20 0x559e9b75b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35eac15740 4 client.17076 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9b6ac7f0 con 0x559e9b758e20 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35e259c640 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b758e20 0x559e9b75b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35c9ffb640 10 client.17076 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.187+0000 7f35e259c640 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b758e20 0x559e9b75b250 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96a12379fff5bb1 server_cookie=625c13d70cc66e49 in_seq=0 out_seq=0 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35c9ffb640 1 -- 172.21.15.16:0/3051695886 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e9b6ac7f0 con 0x559e9b758e20 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35c9ffb640 10 client.17076 handle_command_reply: tid=0 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 client.17076 shutdown 2024-09-15T22:45:53.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 msgr2=0x559e9b754cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b754cd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02a9a0 tx=0x7f35dc02a480 comp rx=0 tx=0).stop 2024-09-15T22:45:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 shutdown_connections 2024-09-15T22:45:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b758e20 0x559e9b75b250 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b754cd0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b742aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 --2- 172.21.15.16:0/3051695886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b742560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 >> 172.21.15.16:0/3051695886 conn(0x559e9b7360f0 msgr2=0x559e9b73bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.188+0000 7f35c9ffb640 0 client.17076 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.189+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 shutdown_connections 2024-09-15T22:45:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.189+0000 7f35eac15740 1 -- 172.21.15.16:0/3051695886 wait complete. 2024-09-15T22:45:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.189+0000 7f35eac15740 20 client.17076 trim_cache size 0 max 16384 2024-09-15T22:45:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 Processor -- start 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 -- start start 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b741c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b754cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b760b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41af690 con 0x559e9b746f50 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41acf60 con 0x559e9b7399e0 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.206+0000 7f35eac15740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41aadd0 con 0x559e9b73a300 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b760b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b760b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44760/0 (socket says 172.21.15.16:44760) 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 -- 172.21.15.16:0/295160488 learned_addr learned my addr 172.21.15.16:0/295160488 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b754cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e259c640 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b741c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 -- 172.21.15.16:0/295160488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 msgr2=0x559e9b754cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b754cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 -- 172.21.15.16:0/295160488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 msgr2=0x559e9b741c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b741c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 -- 172.21.15.16:0/295160488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9b731830 con 0x559e9b746f50 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e259c640 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b741c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b760b20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f35dc04bec0 tx=0x7f35dc02ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1198f0fa59c0203 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35cbfff640 1 -- 172.21.15.16:0/295160488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc051040 con 0x559e9b746f50 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35cbfff640 1 -- 172.21.15.16:0/295160488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35dc004460 con 0x559e9b746f50 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.207+0000 7f35cbfff640 1 -- 172.21.15.16:0/295160488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0053a0 con 0x559e9b746f50 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 -- 172.21.15.16:0/295160488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 msgr2=0x559e9b760b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b760b20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f35dc04bec0 tx=0x7f35dc02ac00 comp rx=0 tx=0).stop 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 -- 172.21.15.16:0/295160488 shutdown_connections 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b746f50 0x559e9b760b20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b754cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 --2- 172.21.15.16:0/295160488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b7399e0 0x559e9b741c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 -- 172.21.15.16:0/295160488 >> 172.21.15.16:0/295160488 conn(0x559e9b7360f0 msgr2=0x559e9b746300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 -- 172.21.15.16:0/295160488 shutdown_connections 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 -- 172.21.15.16:0/295160488 wait complete. 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.208+0000 7f35eac15740 1 -- start start 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 1 Processor -- start 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b750750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b750c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b746f50 0x559e9b7c2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41ab420 con 0x559e9b7399e0 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41ac890 con 0x559e9b746f50 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f35e41abaf0 con 0x559e9b73a300 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35eac15740 10 client.0 _ll_register_callbacks cb 0x7f35dc005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b750750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b750750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44768/0 (socket says 172.21.15.16:44768) 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 -- 172.21.15.16:0/789559456 learned_addr learned my addr 172.21.15.16:0/789559456 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b750c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b746f50 0x559e9b7c2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 -- 172.21.15.16:0/789559456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 msgr2=0x559e9b750c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b750c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 -- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b746f50 msgr2=0x559e9b7c2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b746f50 0x559e9b7c2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e259c640 1 -- 172.21.15.16:0/789559456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9b7c3030 con 0x559e9b7399e0 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.209+0000 7f35e2d9d640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b746f50 0x559e9b7c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35e259c640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b750750 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f35d409c390 tx=0x7f35d4098930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76f0a3558551d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35c9ffb640 1 -- 172.21.15.16:0/789559456 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d40b9070 con 0x559e9b7399e0 2024-09-15T22:45:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35c9ffb640 1 -- 172.21.15.16:0/789559456 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f35d40cdd80 con 0x559e9b7399e0 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35dc004460 con 0x559e9b7399e0 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35c9ffb640 1 -- 172.21.15.16:0/789559456 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f35d401f040 con 0x559e9b7399e0 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 10 client.17082 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e9b7c42e0 con 0x559e9b7399e0 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35c9ffb640 1 -- 172.21.15.16:0/789559456 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d40a5300 con 0x559e9b7399e0 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35c9ffb640 1 -- 172.21.15.16:0/789559456 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35d400f040 con 0x559e9b7399e0 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 10 client.17082 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 20 client.17082 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b7c4a00 0x559e9b7c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 4 client.17082 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.210+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f35dc004460 con 0x559e9b7c4a00 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.211+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b7c4a00 0x559e9b7c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.211+0000 7f35e1d9b640 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b7c4a00 0x559e9b7c6df0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f09fa7794103d5e server_cookie=c82fa8b1c11cf5a3 in_seq=0 out_seq=0 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.211+0000 7f35c9ffb640 10 client.17082 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35c9ffb640 1 -- 172.21.15.16:0/789559456 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f35dc004460 con 0x559e9b7c4a00 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35c9ffb640 10 client.17082 handle_command_reply: tid=0 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 client.17082 shutdown 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 msgr2=0x559e9b750750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b750750 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f35d409c390 tx=0x7f35d4098930 comp rx=0 tx=0).stop 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 shutdown_connections 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x559e9b7c4a00 0x559e9b7c6df0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e9b746f50 0x559e9b7c2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e9b73a300 0x559e9b750c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 --2- 172.21.15.16:0/789559456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e9b7399e0 0x559e9b750750 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 >> 172.21.15.16:0/789559456 conn(0x559e9b7360f0 msgr2=0x559e9b734a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35c9ffb640 0 client.17082 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 shutdown_connections 2024-09-15T22:45:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.212+0000 7f35eac15740 1 -- 172.21.15.16:0/789559456 wait complete. 2024-09-15T22:45:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35eac15740 20 client.17082 trim_cache size 0 max 16384 2024-09-15T22:45:53.225 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:45:53.225 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:45:53.225 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "b0ce0867-76d7-40c7-97d4-114be1cb29d7", 2024-09-15T22:45:53.225 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:45:53.225 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f35cc0a33b0 msgr2=0x7f35cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f35cc0a33b0 0x7f35cc0a5870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0096c0 tx=0x7f35dc009260 comp rx=0 tx=0).stop 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4100e20 msgr2=0x7f35e4197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4100e20 0x7f35e4197350 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f35d400ea80 tx=0x7f35d400ef40 comp rx=0 tx=0).stop 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 shutdown_connections 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f35cc0a33b0 0x7f35cc0a5870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f35e4102ef0 0x7f35e419bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35e4102570 0x7f35e4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 --2- 172.21.15.16:0/4103677217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4100e20 0x7f35e4197350 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 >> 172.21.15.16:0/4103677217 conn(0x7f35e40fc7d0 msgr2=0x7f35e4105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 shutdown_connections 2024-09-15T22:45:53.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:53.213+0000 7f35e883c640 1 -- 172.21.15.16:0/4103677217 wait complete. 2024-09-15T22:45:54.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:45:54.296 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:54 smithi059 ceph-mon[34872]: pgmap v229: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:54 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:45:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:54 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:54 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:45:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:54 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:54.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:54 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:45:54.701 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:54 smithi016 ceph-mon[31492]: pgmap v229: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:54.701 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:54 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:45:54.701 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:54 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:54.701 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:54 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:45:54.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:54 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:54.702 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:54 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:45:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:54 smithi130 ceph-mon[35742]: pgmap v229: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:54 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:45:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:54 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:54 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:45:54.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:54 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:45:54.780 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:54 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:45:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 -- 172.21.15.16:0/892896231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c10c150 msgr2=0x7f321c10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 --2- 172.21.15.16:0/892896231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c10c150 0x7f321c10e5f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f3214009ca0 tx=0x7f321402a950 comp rx=0 tx=0).stop 2024-09-15T22:45:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 -- 172.21.15.16:0/892896231 shutdown_connections 2024-09-15T22:45:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 --2- 172.21.15.16:0/892896231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c10c150 0x7f321c10e5f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 --2- 172.21.15.16:0/892896231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321c103640 0x7f321c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 --2- 172.21.15.16:0/892896231 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f321c102d20 0x7f321c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 -- 172.21.15.16:0/892896231 >> 172.21.15.16:0/892896231 conn(0x7f321c0fc780 msgr2=0x7f321c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 -- 172.21.15.16:0/892896231 shutdown_connections 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 -- 172.21.15.16:0/892896231 wait complete. 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.783+0000 7f3222581640 1 Processor -- start 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3222581640 1 -- start start 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3222581640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c102d20 0x7f321c100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3222581640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321c103640 0x7f321c100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3222581640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f321c10c150 0x7f321c101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3222581640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f321c111f50 con 0x7f321c102d20 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3222581640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f321c111dd0 con 0x7f321c103640 2024-09-15T22:45:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3222581640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f321c1120d0 con 0x7f321c10c150 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c102d20 0x7f321c100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c102d20 0x7f321c100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44784/0 (socket says 172.21.15.16:44784) 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 -- 172.21.15.16:0/2098963370 learned_addr learned my addr 172.21.15.16:0/2098963370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321b7fe640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321c103640 0x7f321c100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f3220b1e640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f321c10c150 0x7f321c101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 -- 172.21.15.16:0/2098963370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f321c10c150 msgr2=0x7f321c101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f321c10c150 0x7f321c101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 -- 172.21.15.16:0/2098963370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321c103640 msgr2=0x7f321c100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321c103640 0x7f321c100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321bfff640 1 -- 172.21.15.16:0/2098963370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c1a68d0 con 0x7f321c102d20 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.784+0000 7f321b7fe640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321c103640 0x7f321c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.785+0000 7f321bfff640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c102d20 0x7f321c100980 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d950 tx=0x7f320c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.785+0000 7f32197fa640 1 -- 172.21.15.16:0/2098963370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c017070 con 0x7f321c102d20 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.785+0000 7f32197fa640 1 -- 172.21.15.16:0/2098963370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320c009c30 con 0x7f321c102d20 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.785+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c1a6bc0 con 0x7f321c102d20 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.785+0000 7f32197fa640 1 -- 172.21.15.16:0/2098963370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c00f6b0 con 0x7f321c102d20 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.785+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c1a7100 con 0x7f321c102d20 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.787+0000 7f32197fa640 1 -- 172.21.15.16:0/2098963370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f320c0109d0 con 0x7f321c102d20 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.787+0000 7f322495a740 1 Processor -- start 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.787+0000 7f322495a740 1 -- start start 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f322495a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e974a0 0x564183e97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f322495a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f322495a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183ea4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f322495a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564183dfd2e0 con 0x564183ea2680 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f322495a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564183da4ce0 con 0x564183e974a0 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f322495a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564183df92a0 con 0x564183e97dc0 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f32197fa640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f32040a33b0 0x7f32040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183ea4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f3220b1e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e974a0 0x564183e97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321bfff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f3220b1e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e974a0 0x564183e97880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:32794/0 (socket says 172.21.15.16:32794) 2024-09-15T22:45:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183ea4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44792/0 (socket says 172.21.15.16:44792) 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f3220b1e640 1 -- 172.21.15.16:0/2989294017 learned_addr learned my addr 172.21.15.16:0/2989294017 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f3220b1e640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f32040a33b0 0x7f32040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f32197fa640 1 -- 172.21.15.16:0/2098963370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f320c0cfe30 con 0x7f321c102d20 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321bfff640 1 -- 172.21.15.16:0/2989294017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e974a0 msgr2=0x564183e97880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321bfff640 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e974a0 0x564183e97880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321bfff640 1 -- 172.21.15.16:0/2989294017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 msgr2=0x564183ea4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321bfff640 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183ea4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321bfff640 1 -- 172.21.15.16:0/2989294017 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564183d89440 con 0x564183e97dc0 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.788+0000 7f321b7fe640 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183ea4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.789+0000 7f3220b1e640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f32040a33b0 0x7f32040a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3214009430 tx=0x7f32140095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.789+0000 7f321bfff640 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea2140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f320c0cd170 tx=0x7f320c06eb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a75eb7c5dbf151 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.789+0000 7f32027fc640 1 -- 172.21.15.16:0/2989294017 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c020070 con 0x564183e97dc0 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.789+0000 7f32027fc640 1 -- 172.21.15.16:0/2989294017 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320c009de0 con 0x564183e97dc0 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.789+0000 7f32027fc640 1 -- 172.21.15.16:0/2989294017 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c098370 con 0x564183e97dc0 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 -- 172.21.15.16:0/2989294017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 msgr2=0x564183ea2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea2140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f320c0cd170 tx=0x7f320c06eb30 comp rx=0 tx=0).stop 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 -- 172.21.15.16:0/2989294017 shutdown_connections 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183ea4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea2140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 --2- 172.21.15.16:0/2989294017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e974a0 0x564183e97880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 -- 172.21.15.16:0/2989294017 >> 172.21.15.16:0/2989294017 conn(0x564183e93a30 msgr2=0x564183e93e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 -- 172.21.15.16:0/2989294017 shutdown_connections 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 -- 172.21.15.16:0/2989294017 wait complete. 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 -- start start 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.790+0000 7f322495a740 1 Processor -- start 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183ea1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e97dc0 0x564183e9f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183e9f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564183ea51f0 con 0x564183e97dc0 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564183db8050 con 0x564183ea2680 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x564183ea5370 con 0x564183e974a0 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 10 client.0 _ll_register_callbacks cb 0x564183df8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:54.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e97dc0 0x564183e9f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e97dc0 0x564183e9f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44798/0 (socket says 172.21.15.16:44798) 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 -- 172.21.15.16:0/1376302347 learned_addr learned my addr 172.21.15.16:0/1376302347 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321bfff640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183ea1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f3220b1e640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183e9f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 -- 172.21.15.16:0/1376302347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 msgr2=0x564183ea1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183ea1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 -- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 msgr2=0x564183e9f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183e9f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 -- 172.21.15.16:0/1376302347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564183b6e3c0 con 0x564183e97dc0 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321bfff640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183ea1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f321b7fe640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e97dc0 0x564183e9f2a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3210009e70 tx=0x7f32100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bcbfb012aa38b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f31fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:54.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f31fb7fe640 1 -- 172.21.15.16:0/1376302347 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210004220 con 0x564183e97dc0 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f31fb7fe640 1 -- 172.21.15.16:0/1376302347 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f32100043c0 con 0x564183e97dc0 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.791+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f320c009de0 con 0x564183e97dc0 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f31fb7fe640 1 -- 172.21.15.16:0/1376302347 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3210012660 con 0x564183e97dc0 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 10 client.17094 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564183eb0c70 con 0x564183e97dc0 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f31fb7fe640 1 -- 172.21.15.16:0/1376302347 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320c009de0 con 0x564183e97dc0 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f31fb7fe640 1 -- 172.21.15.16:0/1376302347 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f321000f040 con 0x564183e97dc0 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 10 client.17094 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 10 client.17094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 20 client.17094 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183eb21d0 0x564183eb4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 4 client.17094 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:54.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564183e0a850 con 0x564183eb21d0 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.792+0000 7f321bfff640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183eb21d0 0x564183eb4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.793+0000 7f321bfff640 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183eb21d0 0x564183eb4610 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94e65b656b5dc01e server_cookie=177921b245426ed4 in_seq=0 out_seq=0 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.793+0000 7f31fb7fe640 10 client.17094 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.793+0000 7f31fb7fe640 1 -- 172.21.15.16:0/1376302347 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564183e0a850 con 0x564183eb21d0 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.793+0000 7f31fb7fe640 10 client.17094 handle_command_reply: tid=0 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.793+0000 7f322495a740 1 client.17094 shutdown 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e97dc0 msgr2=0x564183e9f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e97dc0 0x564183e9f2a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3210009e70 tx=0x7f32100117b0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 shutdown_connections 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183eb21d0 0x564183eb4610 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183e9f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e97dc0 0x564183e9f2a0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f31fb7fe640 0 client.17094 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 --2- 172.21.15.16:0/1376302347 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183ea1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 >> 172.21.15.16:0/1376302347 conn(0x564183e93a30 msgr2=0x564183e99710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 shutdown_connections 2024-09-15T22:45:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 1 -- 172.21.15.16:0/1376302347 wait complete. 2024-09-15T22:45:54.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.794+0000 7f322495a740 20 client.17094 trim_cache size 0 max 16384 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 Processor -- start 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 -- start start 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183eb9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e97dc0 0x564183eba420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183e9c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f321c1b3b70 con 0x564183ea2680 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f321c1b1440 con 0x564183e97dc0 2024-09-15T22:45:54.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f322495a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f321c1af2b0 con 0x564183e974a0 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.811+0000 7f3220b1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183e9c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183e9c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44806/0 (socket says 172.21.15.16:44806) 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 -- 172.21.15.16:0/3857596574 learned_addr learned my addr 172.21.15.16:0/3857596574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f321b7fe640 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e97dc0 0x564183eba420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f321bfff640 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183eb9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 -- 172.21.15.16:0/3857596574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 msgr2=0x564183eb9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183eb9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 -- 172.21.15.16:0/3857596574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e97dc0 msgr2=0x564183eba420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e97dc0 0x564183eba420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 -- 172.21.15.16:0/3857596574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564183e946c0 con 0x564183ea2680 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3220b1e640 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183e9c520 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3214009840 tx=0x7f3214041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9355211b27d5bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3201ffb640 1 -- 172.21.15.16:0/3857596574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214034040 con 0x564183ea2680 2024-09-15T22:45:54.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3201ffb640 1 -- 172.21.15.16:0/3857596574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3214003e50 con 0x564183ea2680 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.812+0000 7f3201ffb640 1 -- 172.21.15.16:0/3857596574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32140047b0 con 0x564183ea2680 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 -- 172.21.15.16:0/3857596574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 msgr2=0x564183e9c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183e9c520 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3214009840 tx=0x7f3214041000 comp rx=0 tx=0).stop 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 -- 172.21.15.16:0/3857596574 shutdown_connections 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183ea2680 0x564183e9c520 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183e97dc0 0x564183eba420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 --2- 172.21.15.16:0/3857596574 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e974a0 0x564183eb9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 -- 172.21.15.16:0/3857596574 >> 172.21.15.16:0/3857596574 conn(0x564183e93a30 msgr2=0x564183ea1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 -- 172.21.15.16:0/3857596574 shutdown_connections 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 -- 172.21.15.16:0/3857596574 wait complete. 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.813+0000 7f322495a740 1 -- start start 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 1 Processor -- start 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e974a0 0x564183ea8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183ea8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f321c1af900 con 0x564183e974a0 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f321c1b0d70 con 0x564183ea2680 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f321c1affd0 con 0x564183e97dc0 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f322495a740 10 client.0 _ll_register_callbacks cb 0x7f3210009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f321bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e974a0 0x564183ea8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f321bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e974a0 0x564183ea8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44810/0 (socket says 172.21.15.16:44810) 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f321bfff640 1 -- 172.21.15.16:0/2838731343 learned_addr learned my addr 172.21.15.16:0/2838731343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f321b7fe640 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f3220b1e640 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183ea8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.814+0000 7f321bfff640 1 -- 172.21.15.16:0/2838731343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 msgr2=0x564183ea88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f321bfff640 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f321bfff640 1 -- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 msgr2=0x564183ea8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f321bfff640 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183ea8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f321bfff640 1 -- 172.21.15.16:0/2838731343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564183f1bef0 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f321bfff640 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e974a0 0x564183ea8360 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f320c0986d0 tx=0x7f320c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c798524ea75910 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f31fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2838731343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c06c060 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3214003e50 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2838731343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f320c0667f0 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2838731343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f320c020040 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f322495a740 10 client.17100 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564183f1d1c0 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2838731343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3214003e50 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2838731343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f320c0cd3f0 con 0x564183e974a0 2024-09-15T22:45:54.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f322495a740 10 client.17100 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f322495a740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f322495a740 20 client.17100 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.815+0000 7f322495a740 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183f1d450 0x564183f1f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.816+0000 7f322495a740 4 client.17100 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.816+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564183e04d60 con 0x564183f1d450 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.816+0000 7f321b7fe640 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183f1d450 0x564183f1f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.816+0000 7f321b7fe640 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183f1d450 0x564183f1f840 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b092292bd5e2c2 server_cookie=d1035cdba38090d5 in_seq=0 out_seq=0 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.816+0000 7f31fb7fe640 10 client.17100 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f31fb7fe640 1 -- 172.21.15.16:0/2838731343 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564183e04d60 con 0x564183f1d450 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f31fb7fe640 10 client.17100 handle_command_reply: tid=0 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 client.17100 shutdown 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e974a0 msgr2=0x564183ea8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e974a0 0x564183ea8360 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f320c0986d0 tx=0x7f320c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 shutdown_connections 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564183f1d450 0x564183f1f840 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564183ea2680 0x564183ea8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x564183e97dc0 0x564183ea88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f31fb7fe640 0 client.17100 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 --2- 172.21.15.16:0/2838731343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564183e974a0 0x564183ea8360 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 >> 172.21.15.16:0/2838731343 conn(0x564183e93a30 msgr2=0x564183e92350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 shutdown_connections 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 1 -- 172.21.15.16:0/2838731343 wait complete. 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.817+0000 7f322495a740 20 client.17100 trim_cache size 0 max 16384 2024-09-15T22:45:54.825 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:45:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:45:54.826 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:45:54.826 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f32040a33b0 msgr2=0x7f32040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f32040a33b0 0x7f32040a5870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3214009430 tx=0x7f32140095c0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c102d20 msgr2=0x7f321c100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c102d20 0x7f321c100980 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d950 tx=0x7f320c00de10 comp rx=0 tx=0).stop 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 shutdown_connections 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f32040a33b0 0x7f32040a5870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f321c10c150 0x7f321c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f321c103640 0x7f321c100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 --2- 172.21.15.16:0/2098963370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321c102d20 0x7f321c100980 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 >> 172.21.15.16:0/2098963370 conn(0x7f321c0fc780 msgr2=0x7f321c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:54.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 shutdown_connections 2024-09-15T22:45:54.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:54.818+0000 7f3222581640 1 -- 172.21.15.16:0/2098963370 wait complete. 2024-09-15T22:45:55.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:45:55.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:45:55.903 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.376+0000 7f19f2deb640 1 -- 172.21.15.16:0/413360904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 msgr2=0x7f19ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.376+0000 7f19f2deb640 1 --2- 172.21.15.16:0/413360904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1021c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009900 tx=0x7f19dc02a410 comp rx=0 tx=0).stop 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 -- 172.21.15.16:0/413360904 shutdown_connections 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 --2- 172.21.15.16:0/413360904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19ec106f10 0x7f19ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 --2- 172.21.15.16:0/413360904 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19ec102790 0x7f19ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 --2- 172.21.15.16:0/413360904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1021c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 -- 172.21.15.16:0/413360904 >> 172.21.15.16:0/413360904 conn(0x7f19ec0fd8c0 msgr2=0x7f19ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 -- 172.21.15.16:0/413360904 shutdown_connections 2024-09-15T22:45:56.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 -- 172.21.15.16:0/413360904 wait complete. 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 Processor -- start 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 -- start start 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.377+0000 7f19f2deb640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19ec102790 0x7f19ec198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f2deb640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19ec106f10 0x7f19ec19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f2deb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec111f70 con 0x7f19ec101de0 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f2deb640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec111df0 con 0x7f19ec106f10 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f2deb640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec1120f0 con 0x7f19ec102790 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44822/0 (socket says 172.21.15.16:44822) 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 -- 172.21.15.16:0/741976343 learned_addr learned my addr 172.21.15.16:0/741976343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19ebfff640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19ec102790 0x7f19ec198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f1388640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19ec106f10 0x7f19ec19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 -- 172.21.15.16:0/741976343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19ec102790 msgr2=0x7f19ec198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19ec102790 0x7f19ec198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 -- 172.21.15.16:0/741976343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19ec106f10 msgr2=0x7f19ec19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19ec106f10 0x7f19ec19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 -- 172.21.15.16:0/741976343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec19d380 con 0x7f19ec101de0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f0b87640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1983d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f19dc005ab0 tx=0x7f19dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19e9ffb640 1 -- 172.21.15.16:0/741976343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc03f030 con 0x7f19ec101de0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19e9ffb640 1 -- 172.21.15.16:0/741976343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19dc003ce0 con 0x7f19ec101de0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec19d610 con 0x7f19ec101de0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.378+0000 7f19e9ffb640 1 -- 172.21.15.16:0/741976343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc035070 con 0x7f19ec101de0 2024-09-15T22:45:56.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.379+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec1a4f50 con 0x7f19ec101de0 2024-09-15T22:45:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.380+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec1032c0 con 0x7f19ec101de0 2024-09-15T22:45:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.381+0000 7f19e9ffb640 1 -- 172.21.15.16:0/741976343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f19dc042030 con 0x7f19ec101de0 2024-09-15T22:45:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.382+0000 7f19e9ffb640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19d40a33b0 0x7f19d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.382+0000 7f19ebfff640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19d40a33b0 0x7f19d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.382+0000 7f19e9ffb640 1 -- 172.21.15.16:0/741976343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f19dc0f0c40 con 0x7f19ec101de0 2024-09-15T22:45:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.382+0000 7f19ebfff640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19d40a33b0 0x7f19d40a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0fdce0 tx=0x7f19e000a250 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:56.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.385+0000 7f19e9ffb640 1 -- 172.21.15.16:0/741976343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19dc0bd210 con 0x7f19ec101de0 2024-09-15T22:45:56.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:56 smithi016 ceph-mon[31492]: pgmap v230: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.603+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19ec10a0f0 con 0x7f19ec101de0 2024-09-15T22:45:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.603+0000 7f19e9ffb640 1 -- 172.21.15.16:0/741976343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f19dc0bcb10 con 0x7f19ec101de0 2024-09-15T22:45:56.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:45:56.605 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:45:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:45:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19d40a33b0 msgr2=0x7f19d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19d40a33b0 0x7f19d40a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0fdce0 tx=0x7f19e000a250 comp rx=0 tx=0).stop 2024-09-15T22:45:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 msgr2=0x7f19ec1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1983d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f19dc005ab0 tx=0x7f19dc0042f0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 shutdown_connections 2024-09-15T22:45:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19ec106f10 0x7f19ec19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19d40a33b0 0x7f19d40a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f19ec102790 0x7f19ec198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 --2- 172.21.15.16:0/741976343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec101de0 0x7f19ec1983d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 >> 172.21.15.16:0/741976343 conn(0x7f19ec0fd8c0 msgr2=0x7f19ec0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 shutdown_connections 2024-09-15T22:45:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:56.606+0000 7f19f2deb640 1 -- 172.21.15.16:0/741976343 wait complete. 2024-09-15T22:45:56.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:56 smithi059 ceph-mon[34872]: pgmap v230: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:56.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:56 smithi130 ceph-mon[35742]: pgmap v230: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:57.394 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:45:57.655 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:57.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:57 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/741976343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:57.776 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:57 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/741976343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:57.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:57 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/741976343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 -- 172.21.15.16:0/3812800336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410100930 msgr2=0x7fe410100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3812800336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410100930 0x7fe410100d10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe400009900 tx=0x7fe40002a410 comp rx=0 tx=0).stop 2024-09-15T22:45:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 -- 172.21.15.16:0/3812800336 shutdown_connections 2024-09-15T22:45:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3812800336 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe41010e4e0 0x7fe4101108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3812800336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4101012e0 0x7fe41010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3812800336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe410100930 0x7fe410100d10 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 -- 172.21.15.16:0/3812800336 >> 172.21.15.16:0/3812800336 conn(0x7fe4100fc7a0 msgr2=0x7fe4100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 -- 172.21.15.16:0/3812800336 shutdown_connections 2024-09-15T22:45:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.145+0000 7fe416ca8640 1 -- 172.21.15.16:0/3812800336 wait complete. 2024-09-15T22:45:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 Processor -- start 2024-09-15T22:45:58.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 -- start start 2024-09-15T22:45:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe410100930 0x7fe41019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4101012e0 0x7fe41019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe41010e4e0 0x7fe41019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe410114170 con 0x7fe4101012e0 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe410113ff0 con 0x7fe41010e4e0 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe416ca8640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101142f0 con 0x7fe410100930 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4101012e0 0x7fe41019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4101012e0 0x7fe41019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44848/0 (socket says 172.21.15.16:44848) 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe40ffff640 1 -- 172.21.15.16:0/3945787105 learned_addr learned my addr 172.21.15.16:0/3945787105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe414a44640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe410100930 0x7fe41019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.146+0000 7fe415245640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe41010e4e0 0x7fe41019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40ffff640 1 -- 172.21.15.16:0/3945787105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe410100930 msgr2=0x7fe41019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40ffff640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe410100930 0x7fe41019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40ffff640 1 -- 172.21.15.16:0/3945787105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe41010e4e0 msgr2=0x7fe41019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40ffff640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe41010e4e0 0x7fe41019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40ffff640 1 -- 172.21.15.16:0/3945787105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41019f600 con 0x7fe4101012e0 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40ffff640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4101012e0 0x7fe41019ab90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800e9f0 tx=0x7fe3f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40dffb640 1 -- 172.21.15.16:0/3945787105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800cd80 con 0x7fe4101012e0 2024-09-15T22:45:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40dffb640 1 -- 172.21.15.16:0/3945787105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3f8010910 con 0x7fe4101012e0 2024-09-15T22:45:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41019f8f0 con 0x7fe4101012e0 2024-09-15T22:45:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe40dffb640 1 -- 172.21.15.16:0/3945787105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8018990 con 0x7fe4101012e0 2024-09-15T22:45:58.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.147+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4101a71d0 con 0x7fe4101012e0 2024-09-15T22:45:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.150+0000 7fe40dffb640 1 -- 172.21.15.16:0/3945787105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe3f8010ac0 con 0x7fe4101012e0 2024-09-15T22:45:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.150+0000 7fe419081740 1 Processor -- start 2024-09-15T22:45:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.150+0000 7fe419081740 1 -- start start 2024-09-15T22:45:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe419081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda5f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe419081740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe419081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda605760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe419081740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cfda55e2f0 con 0x55cfda603370 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe419081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cfda505cf0 con 0x55cfda5f9b40 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe419081740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cfda55a2b0 con 0x55cfda5f85d0 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda605760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe414a44640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda605760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44860/0 (socket says 172.21.15.16:44860) 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe414a44640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:43678/0 (socket says 172.21.15.16:43678) 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe40ffff640 1 -- 172.21.15.16:0/4034556368 learned_addr learned my addr 172.21.15.16:0/4034556368 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:58.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe415245640 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda5f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe414a44640 1 -- 172.21.15.16:0/4034556368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 msgr2=0x55cfda5f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe414a44640 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda5f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe414a44640 1 -- 172.21.15.16:0/4034556368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 msgr2=0x55cfda605760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe414a44640 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda605760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe414a44640 1 -- 172.21.15.16:0/4034556368 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfda4ea3a0 con 0x55cfda5f85d0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe415245640 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda5f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe40ffff640 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda605760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.151+0000 7fe40dffb640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3fc0a3430 0x7fe3fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.152+0000 7fe415245640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3fc0a3430 0x7fe3fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.152+0000 7fe40dffb640 1 -- 172.21.15.16:0/3945787105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fe3f80d0340 con 0x7fe4101012e0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.152+0000 7fe414a44640 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f8a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe400005940 tx=0x7fe400005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ebbf973b325af4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.152+0000 7fe406ffd640 1 -- 172.21.15.16:0/4034556368 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400040070 con 0x55cfda5f85d0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.152+0000 7fe406ffd640 1 -- 172.21.15.16:0/4034556368 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4000318f0 con 0x55cfda5f85d0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.152+0000 7fe406ffd640 1 -- 172.21.15.16:0/4034556368 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400035040 con 0x55cfda5f85d0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 -- 172.21.15.16:0/4034556368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 msgr2=0x55cfda5f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe415245640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3fc0a3430 0x7fe3fc0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe410102b10 tx=0x7fe408009600 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f8a10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe400005940 tx=0x7fe400005e00 comp rx=0 tx=0).stop 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 -- 172.21.15.16:0/4034556368 shutdown_connections 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda605760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f8a10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 --2- 172.21.15.16:0/4034556368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda5f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 -- 172.21.15.16:0/4034556368 >> 172.21.15.16:0/4034556368 conn(0x55cfda5c1310 msgr2=0x55cfda5f4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 -- 172.21.15.16:0/4034556368 shutdown_connections 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 -- 172.21.15.16:0/4034556368 wait complete. 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 -- start start 2024-09-15T22:45:58.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.153+0000 7fe419081740 1 Processor -- start 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe419081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f85d0 0x55cfda602cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe419081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f9b40 0x55cfda5fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe419081740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda603370 0x55cfda600520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe419081740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cfda606010 con 0x55cfda5f9b40 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe419081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cfda519060 con 0x55cfda5f85d0 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe419081740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55cfda606190 con 0x55cfda603370 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe419081740 10 client.0 _ll_register_callbacks cb 0x55cfda26c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f9b40 0x55cfda5fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f9b40 0x55cfda5fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44868/0 (socket says 172.21.15.16:44868) 2024-09-15T22:45:58.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 -- 172.21.15.16:0/96646438 learned_addr learned my addr 172.21.15.16:0/96646438 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe414a44640 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f85d0 0x55cfda602cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe415245640 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda603370 0x55cfda600520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 -- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda603370 msgr2=0x55cfda600520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda603370 0x55cfda600520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 -- 172.21.15.16:0/96646438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f85d0 msgr2=0x55cfda602cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f85d0 0x55cfda602cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.154+0000 7fe40ffff640 1 -- 172.21.15.16:0/96646438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfda2cf3c0 con 0x55cfda5f9b40 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe40ffff640 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f9b40 0x55cfda5fffa0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8098780 tx=0x7fe3f8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafbc96be4de50cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe3effff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe3effff640 1 -- 172.21.15.16:0/96646438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f809be60 con 0x55cfda5f9b40 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe3effff640 1 -- 172.21.15.16:0/96646438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fe3f80bc0f0 con 0x55cfda5f9b40 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4000318f0 con 0x55cfda5f9b40 2024-09-15T22:45:58.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe3effff640 1 -- 172.21.15.16:0/96646438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3f809bc80 con 0x55cfda5f9b40 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 10 client.17118 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfda611a90 con 0x55cfda5f9b40 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe3effff640 1 -- 172.21.15.16:0/96646438 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4000318f0 con 0x55cfda5f9b40 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe3effff640 1 -- 172.21.15.16:0/96646438 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3f80a0240 con 0x55cfda5f9b40 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 10 client.17118 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 10 client.17118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 20 client.17118 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda612ef0 0x55cfda615320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 4 client.17118 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.155+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfda56b860 con 0x55cfda612ef0 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.156+0000 7fe414a44640 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda612ef0 0x55cfda615320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.156+0000 7fe3effff640 10 client.17118 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.156+0000 7fe414a44640 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda612ef0 0x55cfda615320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95e579fec1c52603 server_cookie=1b3d2e0aae3f4dc6 in_seq=0 out_seq=0 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe3effff640 1 -- 172.21.15.16:0/96646438 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cfda56b860 con 0x55cfda612ef0 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe3effff640 10 client.17118 handle_command_reply: tid=0 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 client.17118 shutdown 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f9b40 msgr2=0x55cfda5fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f9b40 0x55cfda5fffa0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8098780 tx=0x7fe3f8009310 comp rx=0 tx=0).stop 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 shutdown_connections 2024-09-15T22:45:58.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda612ef0 0x55cfda615320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda603370 0x55cfda600520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f9b40 0x55cfda5fffa0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 --2- 172.21.15.16:0/96646438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f85d0 0x55cfda602cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 >> 172.21.15.16:0/96646438 conn(0x55cfda5c1310 msgr2=0x55cfda5fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe3effff640 0 client.17118 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 shutdown_connections 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 1 -- 172.21.15.16:0/96646438 wait complete. 2024-09-15T22:45:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.157+0000 7fe419081740 20 client.17118 trim_cache size 0 max 16384 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.170+0000 7fe419081740 1 Processor -- start 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.170+0000 7fe419081740 1 -- start start 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe419081740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe419081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda60f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe419081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda5fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe419081740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b3ae0 con 0x55cfda603370 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe419081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b13b0 con 0x55cfda5f9b40 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe419081740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101af220 con 0x55cfda5f85d0 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda5fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda5fb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44876/0 (socket says 172.21.15.16:44876) 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 -- 172.21.15.16:0/2412793247 learned_addr learned my addr 172.21.15.16:0/2412793247 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:58.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe414a44640 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe40ffff640 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda60f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 -- 172.21.15.16:0/2412793247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 msgr2=0x55cfda5f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 -- 172.21.15.16:0/2412793247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 msgr2=0x55cfda60f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda60f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 -- 172.21.15.16:0/2412793247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfda602910 con 0x55cfda603370 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe40ffff640 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda60f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe415245640 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda5fb120 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe40801c6a0 tx=0x7fe408019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e43f705e733ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.171+0000 7fe4067fc640 1 -- 172.21.15.16:0/2412793247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4080108b0 con 0x55cfda603370 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe4067fc640 1 -- 172.21.15.16:0/2412793247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4080283e0 con 0x55cfda603370 2024-09-15T22:45:58.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe4067fc640 1 -- 172.21.15.16:0/2412793247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4080275d0 con 0x55cfda603370 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 -- 172.21.15.16:0/2412793247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 msgr2=0x55cfda5fb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda5fb120 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe40801c6a0 tx=0x7fe408019000 comp rx=0 tx=0).stop 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 -- 172.21.15.16:0/2412793247 shutdown_connections 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda603370 0x55cfda5fb120 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda5f9b40 0x55cfda60f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 --2- 172.21.15.16:0/2412793247 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f85d0 0x55cfda5f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 -- 172.21.15.16:0/2412793247 >> 172.21.15.16:0/2412793247 conn(0x55cfda5c1310 msgr2=0x55cfda5f4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 -- 172.21.15.16:0/2412793247 shutdown_connections 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 -- 172.21.15.16:0/2412793247 wait complete. 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.172+0000 7fe419081740 1 -- start start 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 1 Processor -- start 2024-09-15T22:45:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f85d0 0x55cfda60b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f9b40 0x55cfda60bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda603370 0x55cfda5fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101af870 con 0x55cfda5f85d0 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101b0ce0 con 0x55cfda603370 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101aff40 con 0x55cfda5f9b40 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f85d0 0x55cfda60b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe419081740 10 client.0 _ll_register_callbacks cb 0x7fe3f800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f85d0 0x55cfda60b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44878/0 (socket says 172.21.15.16:44878) 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 -- 172.21.15.16:0/3125352264 learned_addr learned my addr 172.21.15.16:0/3125352264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe40ffff640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f9b40 0x55cfda60bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe415245640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda603370 0x55cfda5fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 -- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f9b40 msgr2=0x55cfda60bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f9b40 0x55cfda60bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 -- 172.21.15.16:0/3125352264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda603370 msgr2=0x55cfda5fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda603370 0x55cfda5fc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe414a44640 1 -- 172.21.15.16:0/3125352264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfda5fcf30 con 0x55cfda5f85d0 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.173+0000 7fe415245640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda603370 0x55cfda5fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe414a44640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f85d0 0x55cfda60b600 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe4000096c0 tx=0x7fe400005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb232737353ddbc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe3effff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe3effff640 1 -- 172.21.15.16:0/3125352264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400040040 con 0x55cfda5f85d0 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe3effff640 1 -- 172.21.15.16:0/3125352264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fe400004030 con 0x55cfda5f85d0 2024-09-15T22:45:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4080283e0 con 0x55cfda5f85d0 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe3effff640 1 -- 172.21.15.16:0/3125352264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe400063050 con 0x55cfda5f85d0 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe419081740 10 client.17124 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cfda67d590 con 0x55cfda5f85d0 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe3effff640 1 -- 172.21.15.16:0/3125352264 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4080283e0 con 0x55cfda5f85d0 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe3effff640 1 -- 172.21.15.16:0/3125352264 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe400003c80 con 0x55cfda5f85d0 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe419081740 10 client.17124 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe419081740 10 client.17124 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.174+0000 7fe419081740 20 client.17124 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe419081740 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda67d8d0 0x55cfda67fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe419081740 4 client.17124 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfda565d70 con 0x55cfda67d8d0 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe40ffff640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda67d8d0 0x55cfda67fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe40ffff640 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda67d8d0 0x55cfda67fcc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7dd8c80c4f40e34 server_cookie=3db22f4483b279e2 in_seq=0 out_seq=0 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe3effff640 10 client.17124 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe3effff640 1 -- 172.21.15.16:0/3125352264 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfda565d70 con 0x55cfda67d8d0 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe3effff640 10 client.17124 handle_command_reply: tid=0 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe419081740 1 client.17124 shutdown 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f85d0 msgr2=0x55cfda60b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.175+0000 7fe419081740 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f85d0 0x55cfda60b600 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe4000096c0 tx=0x7fe400005500 comp rx=0 tx=0).stop 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 shutdown_connections 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55cfda67d8d0 0x55cfda67fcc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cfda603370 0x55cfda5fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55cfda5f9b40 0x55cfda60bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 --2- 172.21.15.16:0/3125352264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfda5f85d0 0x55cfda60b600 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 >> 172.21.15.16:0/3125352264 conn(0x55cfda5c1310 msgr2=0x55cfda5fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe3effff640 0 client.17124 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:45:58.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 shutdown_connections 2024-09-15T22:45:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 1 -- 172.21.15.16:0/3125352264 wait complete. 2024-09-15T22:45:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe419081740 20 client.17124 trim_cache size 0 max 16384 2024-09-15T22:45:58.183 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3fc0a3430 msgr2=0x7fe3fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3fc0a3430 0x7fe3fc0a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe410102b10 tx=0x7fe408009600 comp rx=0 tx=0).stop 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4101012e0 msgr2=0x7fe41019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.176+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4101012e0 0x7fe41019ab90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800e9f0 tx=0x7fe3f800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 shutdown_connections 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe41010e4e0 0x7fe41019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4101012e0 0x7fe41019ab90 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fe3fc0a3430 0x7fe3fc0a58f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 --2- 172.21.15.16:0/3945787105 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fe410100930 0x7fe41019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 >> 172.21.15.16:0/3945787105 conn(0x7fe4100fc7a0 msgr2=0x7fe4100feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 shutdown_connections 2024-09-15T22:45:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:58.177+0000 7fe416ca8640 1 -- 172.21.15.16:0/3945787105 wait complete. 2024-09-15T22:45:58.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:45:58 smithi059 ceph-mon[34872]: pgmap v231: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:45:58.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:45:58 smithi130 ceph-mon[35742]: pgmap v231: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:45:58.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:45:58 smithi016 ceph-mon[31492]: pgmap v231: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:45:59.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:45:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.778+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3235995707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec100a80 msgr2=0x7f8aec100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.778+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3235995707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec100a80 0x7f8aec100ee0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009930 tx=0x7f8ae002a420 comp rx=0 tx=0).stop 2024-09-15T22:45:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3235995707 shutdown_connections 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3235995707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aec101420 0x7f8aec10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3235995707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec100a80 0x7f8aec100ee0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3235995707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aec106a80 0x7f8aec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3235995707 >> 172.21.15.16:0/3235995707 conn(0x7f8aec0fc7a0 msgr2=0x7f8aec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3235995707 shutdown_connections 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3235995707 wait complete. 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 Processor -- start 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.779+0000 7f8af2bf7640 1 -- start start 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af2bf7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aec100a80 0x7f8aec199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af2bf7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec101420 0x7f8aec199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af2bf7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aec106a80 0x7f8aec1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af2bf7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec110e80 con 0x7f8aec101420 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af2bf7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec110d00 con 0x7f8aec100a80 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af2bf7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec111000 con 0x7f8aec106a80 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec101420 0x7f8aec199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec101420 0x7f8aec199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44902/0 (socket says 172.21.15.16:44902) 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 -- 172.21.15.16:0/3168396316 learned_addr learned my addr 172.21.15.16:0/3168396316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af0993640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aec100a80 0x7f8aec199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af1194640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aec106a80 0x7f8aec1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 -- 172.21.15.16:0/3168396316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aec106a80 msgr2=0x7f8aec1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aec106a80 0x7f8aec1a43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 -- 172.21.15.16:0/3168396316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aec100a80 msgr2=0x7f8aec199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aec100a80 0x7f8aec199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 -- 172.21.15.16:0/3168396316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec1a48f0 con 0x7f8aec101420 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8af0993640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aec100a80 0x7f8aec199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8aebfff640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec101420 0x7f8aec199870 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009230 tx=0x7f8ae0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.780+0000 7f8ae9ffb640 1 -- 172.21.15.16:0/3168396316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae003f030 con 0x7f8aec101420 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.781+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec1a4ac0 con 0x7f8aec101420 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.781+0000 7f8ae9ffb640 1 -- 172.21.15.16:0/3168396316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae0003ad0 con 0x7f8aec101420 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.781+0000 7f8ae9ffb640 1 -- 172.21.15.16:0/3168396316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0035070 con 0x7f8aec101420 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.781+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec1a4f10 con 0x7f8aec101420 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.783+0000 7f8ae9ffb640 1 -- 172.21.15.16:0/3168396316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8ae0042030 con 0x7f8aec101420 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.783+0000 7f8af4fd0740 1 Processor -- start 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.783+0000 7f8af4fd0740 1 -- start start 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.783+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc7eb40 0x55abafc7d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.783+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7d5d0 0x55abafc7da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc88370 0x55abafc8a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55abafbe32f0 con 0x55abafc88370 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55abafb8acf0 con 0x55abafc7d5d0 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55abafbdf2b0 con 0x55abafc7eb40 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc88370 0x55abafc8a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8ae9ffb640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8ad40a33b0 0x7f8ad40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc88370 0x55abafc8a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44904/0 (socket says 172.21.15.16:44904) 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 -- 172.21.15.16:0/1762139229 learned_addr learned my addr 172.21.15.16:0/1762139229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af0993640 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7d5d0 0x55abafc7da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af1194640 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc7eb40 0x55abafc7d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af1194640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8ad40a33b0 0x7f8ad40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8ae9ffb640 1 -- 172.21.15.16:0/3168396316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f8ae00f06d0 con 0x7f8aec101420 2024-09-15T22:45:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 -- 172.21.15.16:0/1762139229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc7eb40 msgr2=0x55abafc7d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc7eb40 0x55abafc7d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 -- 172.21.15.16:0/1762139229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7d5d0 msgr2=0x55abafc7da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7d5d0 0x55abafc7da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 -- 172.21.15.16:0/1762139229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abafb6f3a0 con 0x55abafc88370 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8af0993640 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7d5d0 0x55abafc7da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.784+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc88370 0x55abafc8a760 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00c4ce0 tx=0x7f8ae00bc210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20ad0097f994d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af1194640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8ad40a33b0 0x7f8ad40a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400afd0 tx=0x7f8ae4005db0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8ad2ffd640 1 -- 172.21.15.16:0/1762139229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0049070 con 0x55abafc88370 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8ad2ffd640 1 -- 172.21.15.16:0/1762139229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae00c0950 con 0x55abafc88370 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8ad2ffd640 1 -- 172.21.15.16:0/1762139229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae00edc90 con 0x55abafc88370 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1762139229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc88370 msgr2=0x55abafc8a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc88370 0x55abafc8a760 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00c4ce0 tx=0x7f8ae00bc210 comp rx=0 tx=0).stop 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1762139229 shutdown_connections 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc88370 0x55abafc8a760 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7d5d0 0x55abafc7da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1762139229 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc7eb40 0x55abafc7d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1762139229 >> 172.21.15.16:0/1762139229 conn(0x55abafc46310 msgr2=0x55abafc79c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.785+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1762139229 shutdown_connections 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1762139229 wait complete. 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 -- start start 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 Processor -- start 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc87dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc85810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55abafc8ae30 con 0x55abafc7d5d0 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55abafb9e060 con 0x55abafc7eb40 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55abafc8afb0 con 0x55abafc88370 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af4fd0740 10 client.0 _ll_register_callbacks cb 0x55abaf8f18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af0993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc87dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af0993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc87dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44918/0 (socket says 172.21.15.16:44918) 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.786+0000 7f8af0993640 1 -- 172.21.15.16:0/1174614737 learned_addr learned my addr 172.21.15.16:0/1174614737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af1194640 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc85810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af0993640 1 -- 172.21.15.16:0/1174614737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 msgr2=0x55abafc85810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af0993640 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc85810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af0993640 1 -- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 msgr2=0x55abafc85080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af0993640 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc85080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af0993640 1 -- 172.21.15.16:0/1174614737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abaf9543c0 con 0x55abafc7d5d0 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af0993640 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc87dd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009ea0 tx=0x7f8adc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1930d8d34715506 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8acbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8acbfff640 1 -- 172.21.15.16:0/1174614737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc012860 con 0x55abafc7d5d0 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8acbfff640 1 -- 172.21.15.16:0/1174614737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f8adc012a00 con 0x55abafc7d5d0 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ae00c0950 con 0x55abafc7d5d0 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8acbfff640 1 -- 172.21.15.16:0/1174614737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8adc025050 con 0x55abafc7d5d0 2024-09-15T22:45:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af4fd0740 10 client.17136 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.787+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abafc968f0 con 0x55abafc7d5d0 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8acbfff640 1 -- 172.21.15.16:0/1174614737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ae00c0950 con 0x55abafc7d5d0 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8acbfff640 1 -- 172.21.15.16:0/1174614737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8adc00f040 con 0x55abafc7d5d0 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8af4fd0740 10 client.17136 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8af4fd0740 10 client.17136 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8af4fd0740 20 client.17136 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafc97db0 0x55abafc9a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8af4fd0740 4 client.17136 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abafbf0860 con 0x55abafc97db0 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafc97db0 0x55abafc9a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafc97db0 0x55abafc9a270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e1488bbb3e92172 server_cookie=216a2e8ffe85cb33 in_seq=0 out_seq=0 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.788+0000 7f8acbfff640 10 client.17136 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8acbfff640 1 -- 172.21.15.16:0/1174614737 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8ae00a1e80 con 0x55abafc97db0 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8acbfff640 10 client.17136 handle_command_reply: tid=0 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 client.17136 shutdown 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 msgr2=0x55abafc87dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc87dd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009ea0 tx=0x7f8adc009f90 comp rx=0 tx=0).stop 2024-09-15T22:45:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 shutdown_connections 2024-09-15T22:45:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafc97db0 0x55abafc9a270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc85810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc85080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8acbfff640 0 client.17136 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1174614737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc87dd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 >> 172.21.15.16:0/1174614737 conn(0x55abafc46310 msgr2=0x55abafc7f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.789+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 shutdown_connections 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.790+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1174614737 wait complete. 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.790+0000 7f8af4fd0740 20 client.17136 trim_cache size 0 max 16384 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.802+0000 7f8af4fd0740 1 Processor -- start 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af4fd0740 1 -- start start 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc9fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafca00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1b1940 con 0x55abafc7d5d0 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1af210 con 0x55abafc7eb40 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1ad080 con 0x55abafc88370 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc78bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44924/0 (socket says 172.21.15.16:44924) 2024-09-15T22:45:59.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 -- 172.21.15.16:0/1061416713 learned_addr learned my addr 172.21.15.16:0/1061416713 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af1194640 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafca00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc9fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 -- 172.21.15.16:0/1061416713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 msgr2=0x55abafca00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafca00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 -- 172.21.15.16:0/1061416713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 msgr2=0x55abafc9fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc9fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 -- 172.21.15.16:0/1061416713 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abafc94000 con 0x55abafc7d5d0 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc9fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.803+0000 7f8af0993640 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc78bf0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8adc000c00 tx=0x7f8adc005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c10049c8eff323d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8ad27fc640 1 -- 172.21.15.16:0/1061416713 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc012860 con 0x55abafc7d5d0 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8ad27fc640 1 -- 172.21.15.16:0/1061416713 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8adc009720 con 0x55abafc7d5d0 2024-09-15T22:45:59.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8ad27fc640 1 -- 172.21.15.16:0/1061416713 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0194c0 con 0x55abafc7d5d0 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1061416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 msgr2=0x55abafc78bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc78bf0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f8adc000c00 tx=0x7f8adc005c30 comp rx=0 tx=0).stop 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1061416713 shutdown_connections 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafca00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc9fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1061416713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc78bf0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.804+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1061416713 >> 172.21.15.16:0/1061416713 conn(0x55abafc46310 msgr2=0x55abafc784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1061416713 shutdown_connections 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1061416713 wait complete. 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 -- start start 2024-09-15T22:45:59.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 Processor -- start 2024-09-15T22:45:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc90250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc90790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc8ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1ad6d0 con 0x55abafc7d5d0 2024-09-15T22:45:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1aeb40 con 0x55abafc7eb40 2024-09-15T22:45:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec1adda0 con 0x55abafc88370 2024-09-15T22:45:59.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af4fd0740 10 client.0 _ll_register_callbacks cb 0x7f8adc009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.805+0000 7f8af0993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc90250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc90250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44940/0 (socket says 172.21.15.16:44940) 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 -- 172.21.15.16:0/1404352939 learned_addr learned my addr 172.21.15.16:0/1404352939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af1194640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc8ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc90790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 -- 172.21.15.16:0/1404352939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 msgr2=0x55abafc8ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc8ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 -- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 msgr2=0x55abafc90790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc90790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 -- 172.21.15.16:0/1404352939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abafc8d6a0 con 0x55abafc7d5d0 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af1194640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc8ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:59.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af0993640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc90250 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009ea0 tx=0x7f8adc01ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b57280520add6cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8acbfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8acbfff640 1 -- 172.21.15.16:0/1404352939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc012660 con 0x55abafc7d5d0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8adc009720 con 0x55abafc7d5d0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8acbfff640 1 -- 172.21.15.16:0/1404352939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f8adc012800 con 0x55abafc7d5d0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8acbfff640 1 -- 172.21.15.16:0/1404352939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8adc045050 con 0x55abafc7d5d0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af4fd0740 10 client.17142 fetch_fsmap learned FSMap version 26 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.806+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55abafd035e0 con 0x55abafc7d5d0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8acbfff640 1 -- 172.21.15.16:0/1404352939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8adc009720 con 0x55abafc7d5d0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8acbfff640 1 -- 172.21.15.16:0/1404352939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8adc03cba0 con 0x55abafc7d5d0 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8af4fd0740 10 client.17142 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8af4fd0740 10 client.17142 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:45:59.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8af4fd0740 20 client.17142 populate_metadata read hostname 'smithi016' 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafd03ca0 0x55abafd06090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8af4fd0740 4 client.17142 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abafbead70 con 0x55abafd03ca0 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.807+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafd03ca0 0x55abafd06090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8aebfff640 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafd03ca0 0x55abafd06090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaab5ddf06978c4 server_cookie=2d317ac2621fb561 in_seq=0 out_seq=0 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8acbfff640 10 client.17142 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8acbfff640 1 -- 172.21.15.16:0/1404352939 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abafbead70 con 0x55abafd03ca0 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8acbfff640 10 client.17142 handle_command_reply: tid=0 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 client.17142 shutdown 2024-09-15T22:45:59.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 msgr2=0x55abafc90250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc90250 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009ea0 tx=0x7f8adc01ab40 comp rx=0 tx=0).stop 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 shutdown_connections 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55abafd03ca0 0x55abafd06090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55abafc88370 0x55abafc8ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55abafc7eb40 0x55abafc90790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 --2- 172.21.15.16:0/1404352939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55abafc7d5d0 0x55abafc90250 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 >> 172.21.15.16:0/1404352939 conn(0x55abafc46310 msgr2=0x55abafc78d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:59.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8acbfff640 0 client.17142 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:45:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 shutdown_connections 2024-09-15T22:45:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 1 -- 172.21.15.16:0/1404352939 wait complete. 2024-09-15T22:45:59.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.808+0000 7f8af4fd0740 20 client.17142 trim_cache size 0 max 16384 2024-09-15T22:45:59.830 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:45:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8ad40a33b0 msgr2=0x7f8ad40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8ad40a33b0 0x7f8ad40a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400afd0 tx=0x7f8ae4005db0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec101420 msgr2=0x7f8aec199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec101420 0x7f8aec199870 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009230 tx=0x7f8ae0004270 comp rx=0 tx=0).stop 2024-09-15T22:45:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 shutdown_connections 2024-09-15T22:45:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8ad40a33b0 0x7f8ad40a5870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8aec106a80 0x7f8aec1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aec101420 0x7f8aec199870 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 --2- 172.21.15.16:0/3168396316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aec100a80 0x7f8aec199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 >> 172.21.15.16:0/3168396316 conn(0x7f8aec0fc7a0 msgr2=0x7f8aec0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 shutdown_connections 2024-09-15T22:45:59.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:45:59.809+0000 7f8af2bf7640 1 -- 172.21.15.16:0/3168396316 wait complete. 2024-09-15T22:46:00.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:00 smithi016 ceph-mon[31492]: pgmap v232: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:00.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:46:00.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:00 smithi059 ceph-mon[34872]: pgmap v232: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:00.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:00 smithi130 ceph-mon[35742]: pgmap v232: 129 pgs: 129 active+clean; 21 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:00.785 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 -- 172.21.15.16:0/1708987714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d8103660 msgr2=0x7fa1d810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1708987714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d8103660 0x7fa1d810bb50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009900 tx=0x7fa1cc02a480 comp rx=0 tx=0).stop 2024-09-15T22:46:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 -- 172.21.15.16:0/1708987714 shutdown_connections 2024-09-15T22:46:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1708987714 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1d810c170 0x7fa1d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1708987714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d8103660 0x7fa1d810bb50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1708987714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d8102d40 0x7fa1d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 -- 172.21.15.16:0/1708987714 >> 172.21.15.16:0/1708987714 conn(0x7fa1d80fc7a0 msgr2=0x7fa1d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 -- 172.21.15.16:0/1708987714 shutdown_connections 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.246+0000 7fa1ddead640 1 -- 172.21.15.16:0/1708987714 wait complete. 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.247+0000 7fa1ddead640 1 Processor -- start 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.247+0000 7fa1ddead640 1 -- start start 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1ddead640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d8102d40 0x7fa1d81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1ddead640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1d8103660 0x7fa1d8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1ddead640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d810c170 0x7fa1d819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1ddead640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8111f70 con 0x7fa1d810c170 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1ddead640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d8111df0 con 0x7fa1d8102d40 2024-09-15T22:46:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1ddead640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81120f0 con 0x7fa1d8103660 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d810c170 0x7fa1d819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1d8103660 0x7fa1d8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d810c170 0x7fa1d819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44960/0 (socket says 172.21.15.16:44960) 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 -- 172.21.15.16:0/1472849055 learned_addr learned my addr 172.21.15.16:0/1472849055 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d8102d40 0x7fa1d81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 -- 172.21.15.16:0/1472849055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1d8103660 msgr2=0x7fa1d8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1d8103660 0x7fa1d8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 -- 172.21.15.16:0/1472849055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d8102d40 msgr2=0x7fa1d81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d8102d40 0x7fa1d81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 -- 172.21.15.16:0/1472849055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d819d390 con 0x7fa1d810c170 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d7fff640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d810c170 0x7fa1d819ccb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000cce0 tx=0x7fa1d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.248+0000 7fa1d4ff9640 1 -- 172.21.15.16:0/1472849055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000f450 con 0x7fa1d810c170 2024-09-15T22:46:01.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.249+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d819d680 con 0x7fa1d810c170 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.249+0000 7fa1d4ff9640 1 -- 172.21.15.16:0/1472849055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d000fad0 con 0x7fa1d810c170 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.249+0000 7fa1d4ff9640 1 -- 172.21.15.16:0/1472849055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0018c10 con 0x7fa1d810c170 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.249+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d81a4f60 con 0x7fa1d810c170 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1d4ff9640 1 -- 172.21.15.16:0/1472849055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa1d000f5f0 con 0x7fa1d810c170 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 Processor -- start 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 -- start start 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x56084730c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x56084730ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x560847319760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5608472722f0 con 0x560847317370 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x560847219cf0 con 0x56084730db40 2024-09-15T22:46:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1e0286740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56084726e2b0 con 0x56084730c5d0 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x560847319760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x560847319760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44968/0 (socket says 172.21.15.16:44968) 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.251+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/623198428 learned_addr learned my addr 172.21.15.16:0/623198428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d4ff9640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1c00a33b0 0x7fa1c00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x56084730c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x56084730ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d7fff640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1c00a33b0 0x7fa1c00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d4ff9640 1 -- 172.21.15.16:0/1472849055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fa1d00d0750 con 0x7fa1d810c170 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/623198428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 msgr2=0x56084730ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x56084730ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/623198428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 msgr2=0x56084730c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x56084730c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/623198428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608471fe3a0 con 0x560847317370 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x560847319760 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009900 tx=0x7fa1cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5169574d70213f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1bdffb640 1 -- 172.21.15.16:0/623198428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc035070 con 0x560847317370 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1bdffb640 1 -- 172.21.15.16:0/623198428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc005020 con 0x560847317370 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.252+0000 7fa1bdffb640 1 -- 172.21.15.16:0/623198428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc004a90 con 0x560847317370 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 -- 172.21.15.16:0/623198428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 msgr2=0x560847319760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x560847319760 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009900 tx=0x7fa1cc000bc0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1d7fff640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1c00a33b0 0x7fa1c00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00cdba0 tx=0x7fa1d0091b10 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 -- 172.21.15.16:0/623198428 shutdown_connections 2024-09-15T22:46:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x560847319760 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x56084730ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 --2- 172.21.15.16:0/623198428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x56084730c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 -- 172.21.15.16:0/623198428 >> 172.21.15.16:0/623198428 conn(0x5608472d5310 msgr2=0x560847308c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 -- 172.21.15.16:0/623198428 shutdown_connections 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 -- 172.21.15.16:0/623198428 wait complete. 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.253+0000 7fa1e0286740 1 -- start start 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 1 Processor -- start 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730c5d0 0x560847316d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730db40 0x560847313fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 0x5608473145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56084731a010 con 0x56084730db40 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56084722d060 con 0x56084730c5d0 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56084731a190 con 0x560847317370 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1e0286740 10 client.0 _ll_register_callbacks cb 0x560846f808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730db40 0x560847313fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730db40 0x560847313fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44974/0 (socket says 172.21.15.16:44974) 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 -- 172.21.15.16:0/386397428 learned_addr learned my addr 172.21.15.16:0/386397428 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 0x5608473145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 -- 172.21.15.16:0/386397428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 msgr2=0x5608473145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730c5d0 0x560847316d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 0x5608473145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 -- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730c5d0 msgr2=0x560847316d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730c5d0 0x560847316d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d77fe640 1 -- 172.21.15.16:0/386397428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560846fe33c0 con 0x56084730db40 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.254+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730c5d0 0x560847316d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730db40 0x560847313fe0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800bee0 tx=0x7fa1c8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a20ee56c77d81bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/386397428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8004230 con 0x56084730db40 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/386397428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fa1c80043d0 con 0x56084730db40 2024-09-15T22:46:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1cc005020 con 0x56084730db40 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/386397428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1c8024050 con 0x56084730db40 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 10 client.17154 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560847325b00 con 0x56084730db40 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/386397428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc005020 con 0x56084730db40 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/386397428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1c8003ab0 con 0x56084730db40 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 10 client.17154 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 20 client.17154 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847327010 0x560847329420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 4 client.17154 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.255+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084727f860 con 0x560847327010 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.256+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847327010 0x560847329420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.256+0000 7fa1b6ffd640 10 client.17154 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.256+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847327010 0x560847329420 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e565047700a3920 server_cookie=60aa8300b254cb0b in_seq=0 out_seq=0 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/386397428 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56084727f860 con 0x560847327010 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1b6ffd640 10 client.17154 handle_command_reply: tid=0 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 client.17154 shutdown 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730db40 msgr2=0x560847313fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730db40 0x560847313fe0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800bee0 tx=0x7fa1c8012820 comp rx=0 tx=0).stop 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 shutdown_connections 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847327010 0x560847329420 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 0x5608473145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730db40 0x560847313fe0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 --2- 172.21.15.16:0/386397428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730c5d0 0x560847316d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1b6ffd640 0 client.17154 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 >> 172.21.15.16:0/386397428 conn(0x5608472d5310 msgr2=0x56084730e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 shutdown_connections 2024-09-15T22:46:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 1 -- 172.21.15.16:0/386397428 wait complete. 2024-09-15T22:46:01.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.257+0000 7fa1e0286740 20 client.17154 trim_cache size 0 max 16384 2024-09-15T22:46:01.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 Processor -- start 2024-09-15T22:46:01.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 -- start start 2024-09-15T22:46:01.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x560847308e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x5608473138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81b1880 con 0x560847317370 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81af150 con 0x56084730db40 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.270+0000 7fa1e0286740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81acfc0 con 0x56084730c5d0 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x5608473138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x5608473138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44988/0 (socket says 172.21.15.16:44988) 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/1600288324 learned_addr learned my addr 172.21.15.16:0/1600288324 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x560847308e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/1600288324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 msgr2=0x560847308e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x560847308e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/1600288324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 msgr2=0x5608473232e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473232e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/1600288324 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560847309960 con 0x560847317370 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x5608473138a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006010 tx=0x7fa1cc04e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811732d92fbc0daf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1bd7fa640 1 -- 172.21.15.16:0/1600288324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc035070 con 0x560847317370 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1bd7fa640 1 -- 172.21.15.16:0/1600288324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc03ccb0 con 0x560847317370 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.271+0000 7fa1bd7fa640 1 -- 172.21.15.16:0/1600288324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc03dbd0 con 0x560847317370 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 -- 172.21.15.16:0/1600288324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 msgr2=0x5608473138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x5608473138a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006010 tx=0x7fa1cc04e040 comp rx=0 tx=0).stop 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 -- 172.21.15.16:0/1600288324 shutdown_connections 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560847317370 0x5608473138a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473232e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 --2- 172.21.15.16:0/1600288324 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56084730c5d0 0x560847308e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 -- 172.21.15.16:0/1600288324 >> 172.21.15.16:0/1600288324 conn(0x5608472d5310 msgr2=0x560847308700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 -- 172.21.15.16:0/1600288324 shutdown_connections 2024-09-15T22:46:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 -- 172.21.15.16:0/1600288324 wait complete. 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.272+0000 7fa1e0286740 1 -- start start 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 1 Processor -- start 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730c5d0 0x560847310f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473114d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 0x560847311a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81ad610 con 0x56084730c5d0 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81aea80 con 0x56084730db40 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d81adce0 con 0x560847317370 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1e0286740 10 client.0 _ll_register_callbacks cb 0x7fa1c800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730c5d0 0x560847310f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730c5d0 0x560847310f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44992/0 (socket says 172.21.15.16:44992) 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/321605180 learned_addr learned my addr 172.21.15.16:0/321605180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/321605180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 msgr2=0x560847311a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 0x560847311a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 msgr2=0x5608473114d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473114d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.273+0000 7fa1d6ffd640 1 -- 172.21.15.16:0/321605180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560847390d00 con 0x56084730c5d0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1d6ffd640 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730c5d0 0x560847310f90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc02a4c0 tx=0x7fa1cc005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7ad2bc2b5a7f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/321605180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc035040 con 0x56084730c5d0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/321605180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7fa1cc03dad0 con 0x56084730c5d0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1cc03ccb0 con 0x56084730c5d0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/321605180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1cc03ce60 con 0x56084730c5d0 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 10 client.17160 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560847391f40 con 0x56084730c5d0 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/321605180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1cc03ccb0 con 0x56084730c5d0 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/321605180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1cc004750 con 0x56084730c5d0 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 10 client.17160 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 20 client.17160 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847392160 0x560847394550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 4 client.17160 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.274+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560847279d70 con 0x560847392160 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847392160 0x560847394550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1b6ffd640 10 client.17160 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1d77fe640 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847392160 0x560847394550 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8554f11f81f325ea server_cookie=90beaa19682a97e3 in_seq=0 out_seq=0 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1b6ffd640 1 -- 172.21.15.16:0/321605180 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560847279d70 con 0x560847392160 2024-09-15T22:46:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1b6ffd640 10 client.17160 handle_command_reply: tid=0 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1e0286740 1 client.17160 shutdown 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730c5d0 msgr2=0x560847310f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1e0286740 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730c5d0 0x560847310f90 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc02a4c0 tx=0x7fa1cc005dd0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.275+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 shutdown_connections 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x560847392160 0x560847394550 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560847317370 0x560847311a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56084730db40 0x5608473114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1b6ffd640 0 client.17160 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 1 --2- 172.21.15.16:0/321605180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56084730c5d0 0x560847310f90 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 >> 172.21.15.16:0/321605180 conn(0x5608472d5310 msgr2=0x560847315840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 shutdown_connections 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 1 -- 172.21.15.16:0/321605180 wait complete. 2024-09-15T22:46:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1e0286740 20 client.17160 trim_cache size 0 max 16384 2024-09-15T22:46:01.292 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1c00a33b0 msgr2=0x7fa1c00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1c00a33b0 0x7fa1c00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00cdba0 tx=0x7fa1d0091b10 comp rx=0 tx=0).stop 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d810c170 msgr2=0x7fa1d819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d810c170 0x7fa1d819ccb0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000cce0 tx=0x7fa1d000a620 comp rx=0 tx=0).stop 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 shutdown_connections 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1d810c170 0x7fa1d819ccb0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1c00a33b0 0x7fa1c00a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa1d8103660 0x7fa1d8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 --2- 172.21.15.16:0/1472849055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1d8102d40 0x7fa1d81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.276+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 >> 172.21.15.16:0/1472849055 conn(0x7fa1d80fc7a0 msgr2=0x7fa1d80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:01.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.277+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 shutdown_connections 2024-09-15T22:46:01.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:01.277+0000 7fa1ddead640 1 -- 172.21.15.16:0/1472849055 wait complete. 2024-09-15T22:46:02.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:02 smithi059 ceph-mon[34872]: pgmap v233: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:46:02.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:02 smithi130 ceph-mon[35742]: pgmap v233: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:46:02.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:02 smithi016 ceph-mon[31492]: pgmap v233: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:46:02.990 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:46:02.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:46:03.262 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:03.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.724+0000 7f23600d9640 1 -- 172.21.15.16:0/951964726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358102dd0 msgr2=0x7f23581031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.724+0000 7f23600d9640 1 --2- 172.21.15.16:0/951964726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358102dd0 0x7f23581031b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2348009cd0 tx=0x7f234802a8f0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- 172.21.15.16:0/951964726 shutdown_connections 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 --2- 172.21.15.16:0/951964726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2358108300 0x7f235810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 --2- 172.21.15.16:0/951964726 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2358103780 0x7f2358107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 --2- 172.21.15.16:0/951964726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358102dd0 0x7f23581031b0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- 172.21.15.16:0/951964726 >> 172.21.15.16:0/951964726 conn(0x7f23580fc7d0 msgr2=0x7f23580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- 172.21.15.16:0/951964726 shutdown_connections 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- 172.21.15.16:0/951964726 wait complete. 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 Processor -- start 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- start start 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2358102dd0 0x7f2358100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2358103780 0x7f2358100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358108300 0x7f23581014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2358112170 con 0x7f2358108300 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2358111ff0 con 0x7f2358103780 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.725+0000 7f23600d9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23581122f0 con 0x7f2358102dd0 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358108300 0x7f23581014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235d674640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2358103780 0x7f2358100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235de75640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2358102dd0 0x7f2358100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358108300 0x7f23581014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54370/0 (socket says 172.21.15.16:54370) 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 -- 172.21.15.16:0/4287077193 learned_addr learned my addr 172.21.15.16:0/4287077193 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 -- 172.21.15.16:0/4287077193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2358102dd0 msgr2=0x7f2358100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2358102dd0 0x7f2358100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 -- 172.21.15.16:0/4287077193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2358103780 msgr2=0x7f2358100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2358103780 0x7f2358100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 -- 172.21.15.16:0/4287077193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23581a6c60 con 0x7f2358108300 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f235e676640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358108300 0x7f23581014b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2350009fd0 tx=0x7f235000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f2356ffd640 1 -- 172.21.15.16:0/4287077193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350015020 con 0x7f2358108300 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f2356ffd640 1 -- 172.21.15.16:0/4287077193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2350011d10 con 0x7f2358108300 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f2356ffd640 1 -- 172.21.15.16:0/4287077193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235000f2e0 con 0x7f2358108300 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23581a6f50 con 0x7f2358108300 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.726+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23581a7410 con 0x7f2358108300 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f2356ffd640 1 -- 172.21.15.16:0/4287077193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2350018020 con 0x7f2358108300 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 Processor -- start 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 -- start start 2024-09-15T22:46:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a635f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a640800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a643130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c0a59b280 con 0x562c0a635b60 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562c0a542b10 con 0x562c0a640d40 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f23624b2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562c0a597240 con 0x562c0a636480 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.729+0000 7f2356ffd640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23400a33b0 0x7f23400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a635f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a635f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54374/0 (socket says 172.21.15.16:54374) 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 -- 172.21.15.16:0/1250346418 learned_addr learned my addr 172.21.15.16:0/1250346418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235e676640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23400a33b0 0x7f23400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235de75640 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a640800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a643130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f2356ffd640 1 -- 172.21.15.16:0/4287077193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f23500cf800 con 0x7f2358108300 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 -- 172.21.15.16:0/1250346418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 msgr2=0x562c0a640800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a640800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 -- 172.21.15.16:0/1250346418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 msgr2=0x562c0a643130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a643130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 -- 172.21.15.16:0/1250346418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c0a54f1f0 con 0x562c0a635b60 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235d674640 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a635f40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f234c009330 tx=0x7f234c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc66f90bc74869e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f235e676640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23400a33b0 0x7f23400a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f23500ccd60 tx=0x7f2350090cd0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f233ffff640 1 -- 172.21.15.16:0/1250346418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c017410 con 0x562c0a635b60 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f233ffff640 1 -- 172.21.15.16:0/1250346418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234c01c070 con 0x562c0a635b60 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.730+0000 7f233ffff640 1 -- 172.21.15.16:0/1250346418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c0157d0 con 0x562c0a635b60 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 -- 172.21.15.16:0/1250346418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 msgr2=0x562c0a635f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a635f40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f234c009330 tx=0x7f234c009570 comp rx=0 tx=0).stop 2024-09-15T22:46:03.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 -- 172.21.15.16:0/1250346418 shutdown_connections 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a643130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a640800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 --2- 172.21.15.16:0/1250346418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a635f40 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 -- 172.21.15.16:0/1250346418 >> 172.21.15.16:0/1250346418 conn(0x562c0a6320f0 msgr2=0x562c0a6324d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 -- 172.21.15.16:0/1250346418 shutdown_connections 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.731+0000 7f23624b2740 1 -- 172.21.15.16:0/1250346418 wait complete. 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 -- start start 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 Processor -- start 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a63e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a63e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562c0a643910 con 0x562c0a635b60 2024-09-15T22:46:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562c0a555f10 con 0x562c0a640d40 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562c0a643a90 con 0x562c0a636480 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f23624b2740 10 client.0 _ll_register_callbacks cb 0x562c0a51ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54384/0 (socket says 172.21.15.16:54384) 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f235d674640 1 -- 172.21.15.16:0/3113467291 learned_addr learned my addr 172.21.15.16:0/3113467291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.732+0000 7f235de75640 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a63e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f235d674640 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a63e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f235d674640 1 -- 172.21.15.16:0/3113467291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 msgr2=0x562c0a63e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f235d674640 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a63e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f235d674640 1 -- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 msgr2=0x562c0a63e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f235d674640 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a63e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f235d674640 1 -- 172.21.15.16:0/3113467291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c0a30c6a0 con 0x562c0a635b60 2024-09-15T22:46:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f235d674640 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63db60 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f234c017d30 tx=0x7f234c017c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4870624a7350ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f233d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f233d7fa640 1 -- 172.21.15.16:0/3113467291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c0176a0 con 0x562c0a635b60 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f233d7fa640 1 -- 172.21.15.16:0/3113467291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f234c017840 con 0x562c0a635b60 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234c01c070 con 0x562c0a635b60 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f233d7fa640 1 -- 172.21.15.16:0/3113467291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f234c017a40 con 0x562c0a635b60 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f23624b2740 10 client.17172 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c0a64f490 con 0x562c0a635b60 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f233d7fa640 1 -- 172.21.15.16:0/3113467291 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234c01c070 con 0x562c0a635b60 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.733+0000 7f233d7fa640 1 -- 172.21.15.16:0/3113467291 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f234c0205b0 con 0x562c0a635b60 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f23624b2740 10 client.17172 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f23624b2740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f23624b2740 20 client.17172 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f23624b2740 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a650840 0x562c0a652c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f23624b2740 4 client.17172 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c0a5a87f0 con 0x562c0a650840 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f235de75640 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a650840 0x562c0a652c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f235de75640 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a650840 0x562c0a652c30 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b632f36abe1362 server_cookie=74036234c43d4191 in_seq=0 out_seq=0 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.734+0000 7f233d7fa640 10 client.17172 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f233d7fa640 1 -- 172.21.15.16:0/3113467291 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c0a5a87f0 con 0x562c0a650840 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f233d7fa640 10 client.17172 handle_command_reply: tid=0 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 client.17172 shutdown 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 msgr2=0x562c0a63db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63db60 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f234c017d30 tx=0x7f234c017c80 comp rx=0 tx=0).stop 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 shutdown_connections 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a650840 0x562c0a652c30 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a63e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a63e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 --2- 172.21.15.16:0/3113467291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63db60 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f233d7fa640 0 client.17172 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:03.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 >> 172.21.15.16:0/3113467291 conn(0x562c0a6320f0 msgr2=0x562c0a637dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:03.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 shutdown_connections 2024-09-15T22:46:03.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.735+0000 7f23624b2740 1 -- 172.21.15.16:0/3113467291 wait complete. 2024-09-15T22:46:03.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.736+0000 7f23624b2740 20 client.17172 trim_cache size 0 max 16384 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 Processor -- start 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 -- start start 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a636480 0x562c0a63ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a640d40 0x562c0a63b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b3c10 con 0x562c0a635b60 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b14e0 con 0x562c0a636480 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f23624b2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23581af350 con 0x562c0a640d40 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54394/0 (socket says 172.21.15.16:54394) 2024-09-15T22:46:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.753+0000 7f235d674640 1 -- 172.21.15.16:0/532777238 learned_addr learned my addr 172.21.15.16:0/532777238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f235de75640 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a636480 0x562c0a63ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f235d674640 1 -- 172.21.15.16:0/532777238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a640d40 msgr2=0x562c0a63b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f235d674640 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a640d40 0x562c0a63b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f235d674640 1 -- 172.21.15.16:0/532777238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a636480 msgr2=0x562c0a63ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f235d674640 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a636480 0x562c0a63ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f235d674640 1 -- 172.21.15.16:0/532777238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c0a640160 con 0x562c0a635b60 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f235d674640 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63d0c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f234c004810 tx=0x7f234c020f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e03790e343134f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f233f7fe640 1 -- 172.21.15.16:0/532777238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c0176a0 con 0x562c0a635b60 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f233f7fe640 1 -- 172.21.15.16:0/532777238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234c01c070 con 0x562c0a635b60 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.754+0000 7f233f7fe640 1 -- 172.21.15.16:0/532777238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c02e6e0 con 0x562c0a635b60 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 -- 172.21.15.16:0/532777238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 msgr2=0x562c0a63d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63d0c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f234c004810 tx=0x7f234c020f10 comp rx=0 tx=0).stop 2024-09-15T22:46:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 -- 172.21.15.16:0/532777238 shutdown_connections 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a640d40 0x562c0a63b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a636480 0x562c0a63ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 --2- 172.21.15.16:0/532777238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a63d0c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 -- 172.21.15.16:0/532777238 >> 172.21.15.16:0/532777238 conn(0x562c0a6320f0 msgr2=0x562c0a64c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 -- 172.21.15.16:0/532777238 shutdown_connections 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 -- 172.21.15.16:0/532777238 wait complete. 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 -- start start 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.755+0000 7f23624b2740 1 Processor -- start 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f23624b2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a651c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f23624b2740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a652160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f23624b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a6526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f23624b2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23581af9a0 con 0x562c0a635b60 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f23624b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b0e10 con 0x562c0a640d40 2024-09-15T22:46:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f23624b2740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23581b0070 con 0x562c0a636480 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f23624b2740 10 client.0 _ll_register_callbacks cb 0x7f234c009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a651c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a651c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54406/0 (socket says 172.21.15.16:54406) 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 -- 172.21.15.16:0/2765814296 learned_addr learned my addr 172.21.15.16:0/2765814296 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235de75640 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a652160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a6526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 -- 172.21.15.16:0/2765814296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 msgr2=0x562c0a652160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a652160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 -- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 msgr2=0x562c0a6526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a6526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 -- 172.21.15.16:0/2765814296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c0a6ba3c0 con 0x562c0a635b60 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f235d674640 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a651c20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f234c0091f0 tx=0x7f234c009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104ee8ec9800ac3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f233d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.756+0000 7f233d7fa640 1 -- 172.21.15.16:0/2765814296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234c040020 con 0x562c0a635b60 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f233d7fa640 1 -- 172.21.15.16:0/2765814296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f234c020630 con 0x562c0a635b60 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234c01c070 con 0x562c0a635b60 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f233d7fa640 1 -- 172.21.15.16:0/2765814296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f234c04b050 con 0x562c0a635b60 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 10 client.17178 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c0a6bb700 con 0x562c0a635b60 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f233d7fa640 1 -- 172.21.15.16:0/2765814296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f234c01c070 con 0x562c0a635b60 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f233d7fa640 1 -- 172.21.15.16:0/2765814296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f234c039530 con 0x562c0a635b60 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 10 client.17178 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 20 client.17178 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a6bbd10 0x562c0a6be100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 4 client.17178 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c0a5a2d00 con 0x562c0a6bbd10 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.757+0000 7f235de75640 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a6bbd10 0x562c0a6be100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.758+0000 7f235de75640 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a6bbd10 0x562c0a6be100 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba19f65314907913 server_cookie=ba310e2fec73760d in_seq=0 out_seq=0 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.758+0000 7f233d7fa640 10 client.17178 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f233d7fa640 1 -- 172.21.15.16:0/2765814296 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c0a5a2d00 con 0x562c0a6bbd10 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f233d7fa640 10 client.17178 handle_command_reply: tid=0 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 client.17178 shutdown 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 msgr2=0x562c0a651c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a651c20 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f234c0091f0 tx=0x7f234c009d70 comp rx=0 tx=0).stop 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 shutdown_connections 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x562c0a6bbd10 0x562c0a6be100 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562c0a640d40 0x562c0a6526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562c0a636480 0x562c0a652160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 --2- 172.21.15.16:0/2765814296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562c0a635b60 0x562c0a651c20 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f233d7fa640 0 client.17178 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 >> 172.21.15.16:0/2765814296 conn(0x562c0a6320f0 msgr2=0x562c0a63db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 shutdown_connections 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 1 -- 172.21.15.16:0/2765814296 wait complete. 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.759+0000 7f23624b2740 20 client.17178 trim_cache size 0 max 16384 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:46:03.771 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "6da5061d-cb63-43e0-b6d9-2109b67def8d", 2024-09-15T22:46:03.772 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:46:03.772 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:46:03.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23400a33b0 msgr2=0x7f23400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23400a33b0 0x7f23400a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f23500ccd60 tx=0x7f2350090cd0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358108300 msgr2=0x7f23581014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358108300 0x7f23581014b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2350009fd0 tx=0x7f235000b820 comp rx=0 tx=0).stop 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 shutdown_connections 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2358108300 0x7f23581014b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2358103780 0x7f2358100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23400a33b0 0x7f23400a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 --2- 172.21.15.16:0/4287077193 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2358102dd0 0x7f2358100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 >> 172.21.15.16:0/4287077193 conn(0x7f23580fc7d0 msgr2=0x7f2358105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 shutdown_connections 2024-09-15T22:46:03.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:03.760+0000 7f23600d9640 1 -- 172.21.15.16:0/4287077193 wait complete. 2024-09-15T22:46:04.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:46:04.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:04 smithi059 ceph-mon[34872]: pgmap v234: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:46:04.757 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:04.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:04 smithi130 ceph-mon[35742]: pgmap v234: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:46:04.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:04 smithi016 ceph-mon[31492]: pgmap v234: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:46:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 -- 172.21.15.16:0/1156136450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc100e20 msgr2=0x7f9abc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/1156136450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc100e20 0x7f9abc101200 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009900 tx=0x7f9aac02a480 comp rx=0 tx=0).stop 2024-09-15T22:46:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 -- 172.21.15.16:0/1156136450 shutdown_connections 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/1156136450 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9abc102ef0 0x7f9abc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/1156136450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc102570 0x7f9abc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/1156136450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc100e20 0x7f9abc101200 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 -- 172.21.15.16:0/1156136450 >> 172.21.15.16:0/1156136450 conn(0x7f9abc0fc7d0 msgr2=0x7f9abc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 -- 172.21.15.16:0/1156136450 shutdown_connections 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.333+0000 7f9ac43a6640 1 -- 172.21.15.16:0/1156136450 wait complete. 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 Processor -- start 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 -- start start 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9abc100e20 0x7f9abc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc102570 0x7f9abc19ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc102ef0 0x7f9abc19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc110f80 con 0x7f9abc102ef0 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc110e00 con 0x7f9abc102570 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac43a6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc111100 con 0x7f9abc100e20 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc102ef0 0x7f9abc19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc102ef0 0x7f9abc19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54428/0 (socket says 172.21.15.16:54428) 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 -- 172.21.15.16:0/2679240773 learned_addr learned my addr 172.21.15.16:0/2679240773 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2142640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9abc100e20 0x7f9abc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac1941640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc102570 0x7f9abc19ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 -- 172.21.15.16:0/2679240773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9abc100e20 msgr2=0x7f9abc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9abc100e20 0x7f9abc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 -- 172.21.15.16:0/2679240773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc102570 msgr2=0x7f9abc19ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc102570 0x7f9abc19ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 -- 172.21.15.16:0/2679240773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc19e710 con 0x7f9abc102ef0 2024-09-15T22:46:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac1941640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc102570 0x7f9abc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.334+0000 7f9ac2943640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc102ef0 0x7f9abc19dfe0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400d950 tx=0x7f9ab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.335+0000 7f9abb7fe640 1 -- 172.21.15.16:0/2679240773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4017030 con 0x7f9abc102ef0 2024-09-15T22:46:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.335+0000 7f9abb7fe640 1 -- 172.21.15.16:0/2679240773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab4004020 con 0x7f9abc102ef0 2024-09-15T22:46:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.335+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc19ea00 con 0x7f9abc102ef0 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.335+0000 7f9abb7fe640 1 -- 172.21.15.16:0/2679240773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4003c00 con 0x7f9abc102ef0 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.335+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc1a7370 con 0x7f9abc102ef0 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.337+0000 7f9abb7fe640 1 -- 172.21.15.16:0/2679240773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9ab401a070 con 0x7f9abc102ef0 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.337+0000 7f9ac677f740 1 Processor -- start 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.337+0000 7f9ac677f740 1 -- start start 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c3dc0 0x5624af8ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac677f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624af8292e0 con 0x5624af8ce680 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac677f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5624af7d0ce0 con 0x5624af8c34a0 2024-09-15T22:46:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac677f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5624af8252a0 con 0x5624af8c3dc0 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9abb7fe640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9aa40a3430 0x7f9aa40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54434/0 (socket says 172.21.15.16:54434) 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 -- 172.21.15.16:0/2837314392 learned_addr learned my addr 172.21.15.16:0/2837314392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac2943640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9aa40a3430 0x7f9aa40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac2142640 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c3dc0 0x5624af8ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9abb7fe640 1 -- 172.21.15.16:0/2679240773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f9ab40cfc20 con 0x7f9abc102ef0 2024-09-15T22:46:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 -- 172.21.15.16:0/2837314392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c3dc0 msgr2=0x5624af8ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c3dc0 0x5624af8ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 -- 172.21.15.16:0/2837314392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 msgr2=0x5624af8c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 -- 172.21.15.16:0/2837314392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624af7b5440 con 0x5624af8ce680 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac2142640 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c3dc0 0x5624af8ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.338+0000 7f9ac1941640 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8d0a70 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00137d0 tx=0x7f9ab00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fcee3fb708a61d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac2943640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9aa40a3430 0x7f9aa40a58f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40cd070 tx=0x7f9ab4086040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ab8dea640 1 -- 172.21.15.16:0/2837314392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0019030 con 0x5624af8ce680 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ab8dea640 1 -- 172.21.15.16:0/2837314392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0013e10 con 0x5624af8ce680 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ab8dea640 1 -- 172.21.15.16:0/2837314392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000fbb0 con 0x5624af8ce680 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac677f740 1 -- 172.21.15.16:0/2837314392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 msgr2=0x5624af8d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac677f740 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8d0a70 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00137d0 tx=0x7f9ab00098d0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac677f740 1 -- 172.21.15.16:0/2837314392 shutdown_connections 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac677f740 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8d0a70 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac677f740 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c3dc0 0x5624af8ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac677f740 1 --2- 172.21.15.16:0/2837314392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.339+0000 7f9ac677f740 1 -- 172.21.15.16:0/2837314392 >> 172.21.15.16:0/2837314392 conn(0x5624af8bfa30 msgr2=0x5624af8bfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 -- 172.21.15.16:0/2837314392 shutdown_connections 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 -- 172.21.15.16:0/2837314392 wait complete. 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 -- start start 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 Processor -- start 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8ce000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5624af8d1250 con 0x5624af8c3dc0 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5624af7e4050 con 0x5624af8c34a0 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5624af8d13d0 con 0x5624af8ce680 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac677f740 10 client.0 _ll_register_callbacks cb 0x5624af824160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac2142640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac2142640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8cb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54442/0 (socket says 172.21.15.16:54442) 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac1941640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.340+0000 7f9ac2142640 1 -- 172.21.15.16:0/494123311 learned_addr learned my addr 172.21.15.16:0/494123311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac1941640 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac2142640 1 -- 172.21.15.16:0/494123311 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 msgr2=0x5624af8cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac2142640 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac2142640 1 -- 172.21.15.16:0/494123311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 msgr2=0x5624af8ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac2142640 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac2142640 1 -- 172.21.15.16:0/494123311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624af59a3c0 con 0x5624af8c3dc0 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac2142640 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8cb2b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0314e0 tx=0x7f9aac0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e90623c783b0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/494123311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac035030 con 0x5624af8c3dc0 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/494123311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f9aac0048c0 con 0x5624af8c3dc0 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab0013e10 con 0x5624af8c3dc0 2024-09-15T22:46:05.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/494123311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9aac005d50 con 0x5624af8c3dc0 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac677f740 10 client.17190 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624af8dcc70 con 0x5624af8c3dc0 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.341+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/494123311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0013e10 con 0x5624af8c3dc0 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/494123311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9aac031540 con 0x5624af8c3dc0 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac677f740 10 client.17190 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac677f740 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac677f740 20 client.17190 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac677f740 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af8de130 0x5624af8e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac677f740 4 client.17190 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624af836850 con 0x5624af8de130 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac1941640 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af8de130 0x5624af8e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9ac1941640 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af8de130 0x5624af8e05b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74249dd84f90f68 server_cookie=bcfbf2460dd0eaec in_seq=0 out_seq=0 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.342+0000 7f9aa1ffb640 10 client.17190 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/494123311 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624af836850 con 0x5624af8de130 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9aa1ffb640 10 client.17190 handle_command_reply: tid=0 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 client.17190 shutdown 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 msgr2=0x5624af8cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8cb2b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0314e0 tx=0x7f9aac0052d0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 shutdown_connections 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af8de130 0x5624af8e05b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8cb2b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9aa1ffb640 0 client.17190 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:05.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 --2- 172.21.15.16:0/494123311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 >> 172.21.15.16:0/494123311 conn(0x5624af8bfa30 msgr2=0x5624af8c5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:05.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.343+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 shutdown_connections 2024-09-15T22:46:05.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.344+0000 7f9ac677f740 1 -- 172.21.15.16:0/494123311 wait complete. 2024-09-15T22:46:05.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.344+0000 7f9ac677f740 20 client.17190 trim_cache size 0 max 16384 2024-09-15T22:46:05.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 Processor -- start 2024-09-15T22:46:05.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 -- start start 2024-09-15T22:46:05.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c34a0 0x5624af8da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c3dc0 0x5624af8e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b3b10 con 0x5624af8ce680 2024-09-15T22:46:05.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b13e0 con 0x5624af8c3dc0 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac677f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1af250 con 0x5624af8c34a0 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac1941640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac1941640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8e63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54456/0 (socket says 172.21.15.16:54456) 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac1941640 1 -- 172.21.15.16:0/72492812 learned_addr learned my addr 172.21.15.16:0/72492812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac2142640 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c3dc0 0x5624af8e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.361+0000 7f9ac1941640 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c34a0 0x5624af8da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9ac1941640 1 -- 172.21.15.16:0/72492812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c34a0 msgr2=0x5624af8da4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9ac1941640 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c34a0 0x5624af8da4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9ac1941640 1 -- 172.21.15.16:0/72492812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c3dc0 msgr2=0x5624af8e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9ac1941640 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c3dc0 0x5624af8e5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9ac1941640 1 -- 172.21.15.16:0/72492812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624af8c32d0 con 0x5624af8ce680 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9ac2142640 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c3dc0 0x5624af8e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9ac1941640 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8e63c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0002230 tx=0x7f9ab000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4c5cd2ed961da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9aa3fff640 1 -- 172.21.15.16:0/72492812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0019050 con 0x5624af8ce680 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9aa3fff640 1 -- 172.21.15.16:0/72492812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab001a040 con 0x5624af8ce680 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.362+0000 7f9aa3fff640 1 -- 172.21.15.16:0/72492812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0011800 con 0x5624af8ce680 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- 172.21.15.16:0/72492812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 msgr2=0x5624af8e63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8e63c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0002230 tx=0x7f9ab000f890 comp rx=0 tx=0).stop 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- 172.21.15.16:0/72492812 shutdown_connections 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8ce680 0x5624af8e63c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c3dc0 0x5624af8e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 --2- 172.21.15.16:0/72492812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8c34a0 0x5624af8da4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- 172.21.15.16:0/72492812 >> 172.21.15.16:0/72492812 conn(0x5624af8bfa30 msgr2=0x5624af8c68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- 172.21.15.16:0/72492812 shutdown_connections 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- 172.21.15.16:0/72492812 wait complete. 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- start start 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 Processor -- start 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8d1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1af8a0 con 0x5624af8c3dc0 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1b0d10 con 0x5624af8c34a0 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1aff70 con 0x5624af8ce680 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.363+0000 7f9ac677f740 10 client.0 _ll_register_callbacks cb 0x7f9aac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:05.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8d1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8d1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54460/0 (socket says 172.21.15.16:54460) 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 -- 172.21.15.16:0/122134552 learned_addr learned my addr 172.21.15.16:0/122134552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac1941640 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac1941640 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 -- 172.21.15.16:0/122134552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 msgr2=0x5624af8d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8d2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 -- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 msgr2=0x5624af8d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 -- 172.21.15.16:0/122134552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624af8d2a80 con 0x5624af8c3dc0 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac2142640 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8d1f00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009850 tx=0x7f9aac004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1091eab575762dc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:05.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9aa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/122134552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac035040 con 0x5624af8c3dc0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/122134552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9467+0+0 (secure 0 0 0) 0x7f9aac0048c0 con 0x5624af8c3dc0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab001a040 con 0x5624af8c3dc0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/122134552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9aac05e050 con 0x5624af8c3dc0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.364+0000 7f9ac677f740 10 client.17196 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624af948c90 con 0x5624af8c3dc0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/122134552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab001a040 con 0x5624af8c3dc0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/122134552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9aac059030 con 0x5624af8c3dc0 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac677f740 10 client.17196 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac677f740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac677f740 20 client.17196 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:05.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac677f740 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af949490 0x5624af94b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac677f740 4 client.17196 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5624af830d60 con 0x5624af949490 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.365+0000 7f9ac1941640 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af949490 0x5624af94b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac1941640 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af949490 0x5624af94b8b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e797eb35b1d4f6 server_cookie=2a68144584761b63 in_seq=0 out_seq=0 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9aa1ffb640 10 client.17196 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9aa1ffb640 1 -- 172.21.15.16:0/122134552 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5624af830d60 con 0x5624af949490 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9aa1ffb640 10 client.17196 handle_command_reply: tid=0 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 client.17196 shutdown 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 msgr2=0x5624af8d1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8d1f00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009850 tx=0x7f9aac004610 comp rx=0 tx=0).stop 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 shutdown_connections 2024-09-15T22:46:05.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5624af949490 0x5624af94b8b0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5624af8ce680 0x5624af8d2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624af8c3dc0 0x5624af8d1f00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 --2- 172.21.15.16:0/122134552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5624af8c34a0 0x5624af8d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9aa1ffb640 0 client.17196 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 >> 172.21.15.16:0/122134552 conn(0x5624af8bfa30 msgr2=0x5624af8be350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 shutdown_connections 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 1 -- 172.21.15.16:0/122134552 wait complete. 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.366+0000 7f9ac677f740 20 client.17196 trim_cache size 0 max 16384 2024-09-15T22:46:05.379 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "6da5061d-cb63-43e0-b6d9-2109b67def8d": { 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "6da5061d-cb63-43e0-b6d9-2109b67def8d", 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:46:05.380 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:46:05.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9aa40a3430 msgr2=0x7f9aa40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9aa40a3430 0x7f9aa40a58f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40cd070 tx=0x7f9ab4086040 comp rx=0 tx=0).stop 2024-09-15T22:46:05.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc102ef0 msgr2=0x7f9abc19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:05.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc102ef0 0x7f9abc19dfe0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400d950 tx=0x7f9ab400de10 comp rx=0 tx=0).stop 2024-09-15T22:46:05.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 shutdown_connections 2024-09-15T22:46:05.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9abc102ef0 0x7f9abc19dfe0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9abc102570 0x7f9abc19ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9aa40a3430 0x7f9aa40a58f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 --2- 172.21.15.16:0/2679240773 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9abc100e20 0x7f9abc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:05.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 >> 172.21.15.16:0/2679240773 conn(0x7f9abc0fc7d0 msgr2=0x7f9abc06a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:05.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 shutdown_connections 2024-09-15T22:46:05.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:05.367+0000 7f9ac43a6640 1 -- 172.21.15.16:0/2679240773 wait complete. 2024-09-15T22:46:05.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:05 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:46:05.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:05 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:05.557 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:05 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:46:05.558 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:05 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:05 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:46:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:05 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:05 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:46:05.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:05 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:05.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:05 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:46:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:05 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:05 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:46:05.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:05 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:06.070 INFO:tasks.cephfs.filesystem:scrub status for tag:6da5061d-cb63-43e0-b6d9-2109b67def8d - {'path': '/', 'tag': '6da5061d-cb63-43e0-b6d9-2109b67def8d', 'options': 'recursive,force'} 2024-09-15T22:46:06.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:06 smithi059 ceph-mon[34872]: pgmap v235: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:46:06.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:06 smithi130 ceph-mon[35742]: pgmap v235: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:46:06.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:06 smithi016 ceph-mon[31492]: pgmap v235: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:46:08.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:08 smithi059 ceph-mon[34872]: pgmap v236: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-15T22:46:08.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:08 smithi130 ceph-mon[35742]: pgmap v236: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-15T22:46:08.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:08 smithi016 ceph-mon[31492]: pgmap v236: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-09-15T22:46:09.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:09 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:46:09.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:09 smithi059 ceph-mon[34872]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:46:09.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:09 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:46:09.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:09 smithi130 ceph-mon[35742]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:46:09.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:09 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:46:09.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:09 smithi016 ceph-mon[31492]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:46:10.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:10 smithi059 ceph-mon[34872]: pgmap v238: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:46:10.708 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:10 smithi016 ceph-mon[31492]: pgmap v238: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:46:10.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:10 smithi130 ceph-mon[35742]: pgmap v238: 129 pgs: 129 active+clean; 21 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:46:12.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:12 smithi130 ceph-mon[35742]: pgmap v239: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:12 smithi016 ceph-mon[31492]: pgmap v239: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:12.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:12 smithi059 ceph-mon[34872]: pgmap v239: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:14.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:14 smithi130 ceph-mon[35742]: pgmap v240: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:14.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:14 smithi130 ceph-mon[35742]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:46:14.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:14 smithi016 ceph-mon[31492]: pgmap v240: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:14.808 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:14 smithi016 ceph-mon[31492]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:46:14.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:14 smithi059 ceph-mon[34872]: pgmap v240: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:14.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:14 smithi059 ceph-mon[34872]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:46:16.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:16 smithi130 ceph-mon[35742]: pgmap v242: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:46:16.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:16 smithi016 ceph-mon[31492]: pgmap v242: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:46:16.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:16 smithi059 ceph-mon[34872]: pgmap v242: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:46:18.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:18 smithi130 ceph-mon[35742]: pgmap v243: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 234 B/s wr, 7 op/s 2024-09-15T22:46:18.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:18 smithi016 ceph-mon[31492]: pgmap v243: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 234 B/s wr, 7 op/s 2024-09-15T22:46:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:18 smithi059 ceph-mon[34872]: pgmap v243: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 234 B/s wr, 7 op/s 2024-09-15T22:46:20.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:20 smithi130 ceph-mon[35742]: pgmap v244: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:20.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:20 smithi016 ceph-mon[31492]: pgmap v244: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:20.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:20 smithi059 ceph-mon[34872]: pgmap v244: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:46:22.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:22 smithi130 ceph-mon[35742]: pgmap v245: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:46:22.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:22 smithi016 ceph-mon[31492]: pgmap v245: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:46:22.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:22 smithi059 ceph-mon[34872]: pgmap v245: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:46:24.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:24 smithi130 ceph-mon[35742]: pgmap v246: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:46:24.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:24 smithi016 ceph-mon[31492]: pgmap v246: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:46:24.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:24 smithi059 ceph-mon[34872]: pgmap v246: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:46:26.745 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:26 smithi059 ceph-mon[34872]: pgmap v247: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:46:26.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:26 smithi130 ceph-mon[35742]: pgmap v247: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:46:26.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:26 smithi016 ceph-mon[31492]: pgmap v247: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:46:28.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:28 smithi130 ceph-mon[35742]: pgmap v248: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:46:28.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:28 smithi016 ceph-mon[31492]: pgmap v248: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:46:28.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:28 smithi059 ceph-mon[34872]: pgmap v248: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:46:30.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:30 smithi130 ceph-mon[35742]: pgmap v249: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:30.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:30 smithi016 ceph-mon[31492]: pgmap v249: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:30.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:30 smithi059 ceph-mon[34872]: pgmap v249: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:32.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:32 smithi130 ceph-mon[35742]: pgmap v250: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:32.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:32 smithi016 ceph-mon[31492]: pgmap v250: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:32.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:32 smithi059 ceph-mon[34872]: pgmap v250: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:34.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:34 smithi130 ceph-mon[35742]: pgmap v251: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:34.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:34 smithi016 ceph-mon[31492]: pgmap v251: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:34.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:34 smithi059 ceph-mon[34872]: pgmap v251: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:36.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:46:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:36.733 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:36 smithi016 ceph-mon[31492]: pgmap v252: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:36.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:36 smithi130 ceph-mon[35742]: pgmap v252: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:36.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:36 smithi059 ceph-mon[34872]: pgmap v252: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 -- 172.21.15.16:0/3528600822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c106f10 msgr2=0x7f448c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 --2- 172.21.15.16:0/3528600822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c106f10 0x7f448c10ea90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4484009ca0 tx=0x7f448402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 -- 172.21.15.16:0/3528600822 shutdown_connections 2024-09-15T22:46:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 --2- 172.21.15.16:0/3528600822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c106f10 0x7f448c10ea90 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 --2- 172.21.15.16:0/3528600822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f448c102790 0x7f448c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 --2- 172.21.15.16:0/3528600822 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f448c101de0 0x7f448c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 -- 172.21.15.16:0/3528600822 >> 172.21.15.16:0/3528600822 conn(0x7f448c0fd8c0 msgr2=0x7f448c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 -- 172.21.15.16:0/3528600822 shutdown_connections 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.940+0000 7f44911b9640 1 -- 172.21.15.16:0/3528600822 wait complete. 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.941+0000 7f44911b9640 1 Processor -- start 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.941+0000 7f44911b9640 1 -- start start 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.941+0000 7f44911b9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f448c101de0 0x7f448c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.941+0000 7f44911b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c102790 0x7f448c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f44911b9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f448c106f10 0x7f448c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f44911b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f448c111f70 con 0x7f448c102790 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f44911b9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f448c111df0 con 0x7f448c106f10 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f44911b9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1120f0 con 0x7f448c101de0 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c102790 0x7f448c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f448c101de0 0x7f448c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c102790 0x7f448c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38596/0 (socket says 172.21.15.16:38596) 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f448c101de0 0x7f448c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:33014/0 (socket says 172.21.15.16:33014) 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 -- 172.21.15.16:0/2725044557 learned_addr learned my addr 172.21.15.16:0/2725044557 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448b59e640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f448c106f10 0x7f448c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 -- 172.21.15.16:0/2725044557 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f448c101de0 msgr2=0x7f448c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f448c101de0 0x7f448c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 -- 172.21.15.16:0/2725044557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f448c106f10 msgr2=0x7f448c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f448c106f10 0x7f448c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 -- 172.21.15.16:0/2725044557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448c19d380 con 0x7f448c102790 2024-09-15T22:46:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f448a59c640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c102790 0x7f448c198910 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f448000d950 tx=0x7f448000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.942+0000 7f4473fff640 1 -- 172.21.15.16:0/2725044557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4480017070 con 0x7f448c102790 2024-09-15T22:46:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.943+0000 7f4473fff640 1 -- 172.21.15.16:0/2725044557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4480009c30 con 0x7f448c102790 2024-09-15T22:46:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.943+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448c19d670 con 0x7f448c102790 2024-09-15T22:46:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.943+0000 7f4473fff640 1 -- 172.21.15.16:0/2725044557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448000f6b0 con 0x7f448c102790 2024-09-15T22:46:36.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.943+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448c1a4f50 con 0x7f448c102790 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4473fff640 1 -- 172.21.15.16:0/2725044557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f44800109d0 con 0x7f448c102790 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 Processor -- start 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 -- start start 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c40880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c4b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c4b680 0x556ba2c4da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ba2ba62e0 con 0x556ba2c4b680 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ba2b4dce0 con 0x556ba2c404a0 2024-09-15T22:46:36.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4493592740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556ba2ba22a0 con 0x556ba2c40dc0 2024-09-15T22:46:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.945+0000 7f4473fff640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f44740a33b0 0x7f44740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c4b680 0x556ba2c4da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c4b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448b59e640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f44740a33b0 0x7f44740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c4b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:33016/0 (socket says 172.21.15.16:33016) 2024-09-15T22:46:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 -- 172.21.15.16:0/1075863273 learned_addr learned my addr 172.21.15.16:0/1075863273 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f4473fff640 1 -- 172.21.15.16:0/2725044557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f44800cfbb0 con 0x7f448c102790 2024-09-15T22:46:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448b59e640 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c40880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 -- 172.21.15.16:0/1075863273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 msgr2=0x556ba2c40880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c40880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 -- 172.21.15.16:0/1075863273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c4b680 msgr2=0x556ba2c4da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c4b680 0x556ba2c4da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 -- 172.21.15.16:0/1075863273 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba2b32440 con 0x556ba2c40dc0 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448b59e640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f44740a33b0 0x7f44740a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4484004af0 tx=0x7f448402f310 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f448ad9d640 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c4b140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b760 tx=0x7f447c00bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad3beae1bcbdae5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f44717fa640 1 -- 172.21.15.16:0/1075863273 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c003dd0 con 0x556ba2c40dc0 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f44717fa640 1 -- 172.21.15.16:0/1075863273 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c015070 con 0x556ba2c40dc0 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.946+0000 7f44717fa640 1 -- 172.21.15.16:0/1075863273 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c004e60 con 0x556ba2c40dc0 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 -- 172.21.15.16:0/1075863273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 msgr2=0x556ba2c4b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c4b140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b760 tx=0x7f447c00bc20 comp rx=0 tx=0).stop 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 -- 172.21.15.16:0/1075863273 shutdown_connections 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c4b680 0x556ba2c4da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c4b140 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 --2- 172.21.15.16:0/1075863273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c40880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 -- 172.21.15.16:0/1075863273 >> 172.21.15.16:0/1075863273 conn(0x556ba2c3ca30 msgr2=0x556ba2c3ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 -- 172.21.15.16:0/1075863273 shutdown_connections 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.947+0000 7f4493592740 1 -- 172.21.15.16:0/1075863273 wait complete. 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 -- start start 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 Processor -- start 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c4afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c48250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c4b680 0x556ba2c48790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556ba2c4e250 con 0x556ba2c404a0 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ba2b61050 con 0x556ba2c4b680 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x556ba2c4e3d0 con 0x556ba2c40dc0 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f4493592740 10 client.0 _ll_register_callbacks cb 0x556ba2ba1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c4afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c4afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38618/0 (socket says 172.21.15.16:38618) 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 -- 172.21.15.16:0/3267949034 learned_addr learned my addr 172.21.15.16:0/3267949034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448b59e640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c4b680 0x556ba2c48790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448a59c640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c48250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 -- 172.21.15.16:0/3267949034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 msgr2=0x556ba2c48250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c48250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 -- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c4b680 msgr2=0x556ba2c48790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c4b680 0x556ba2c48790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448ad9d640 1 -- 172.21.15.16:0/3267949034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba29173c0 con 0x556ba2c404a0 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448a59c640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c48250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.948+0000 7f448b59e640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c4b680 0x556ba2c48790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f448ad9d640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c4afa0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f447c0039c0 tx=0x7f447c00bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b26bf050cf4df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:36.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f446eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f446eff5640 1 -- 172.21.15.16:0/3267949034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c028060 con 0x556ba2c404a0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f446eff5640 1 -- 172.21.15.16:0/3267949034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f447c011640 con 0x556ba2c404a0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f447c015070 con 0x556ba2c404a0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f446eff5640 1 -- 172.21.15.16:0/3267949034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f447c02f050 con 0x556ba2c404a0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 10 client.17208 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ba2c59bd0 con 0x556ba2c404a0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f446eff5640 1 -- 172.21.15.16:0/3267949034 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c015070 con 0x556ba2c404a0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f446eff5640 1 -- 172.21.15.16:0/3267949034 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f447c022070 con 0x556ba2c404a0 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 10 client.17208 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 10 client.17208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 20 client.17208 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2c5b090 0x556ba2c5d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 4 client.17208 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.949+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba2bb3850 con 0x556ba2c5b090 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.950+0000 7f448a59c640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2c5b090 0x556ba2c5d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.950+0000 7f448a59c640 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2c5b090 0x556ba2c5d480 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c0956c7a21d354a server_cookie=ae48f7033780afd4 in_seq=0 out_seq=0 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.950+0000 7f446eff5640 10 client.17208 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f446eff5640 1 -- 172.21.15.16:0/3267949034 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ba2bb3850 con 0x556ba2c5b090 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f446eff5640 10 client.17208 handle_command_reply: tid=0 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 client.17208 shutdown 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 msgr2=0x556ba2c4afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c4afa0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f447c0039c0 tx=0x7f447c00bf00 comp rx=0 tx=0).stop 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 shutdown_connections 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2c5b090 0x556ba2c5d480 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c4b680 0x556ba2c48790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c40dc0 0x556ba2c48250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f446eff5640 0 client.17208 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 --2- 172.21.15.16:0/3267949034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c4afa0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 >> 172.21.15.16:0/3267949034 conn(0x556ba2c3ca30 msgr2=0x556ba2c42710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 shutdown_connections 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 1 -- 172.21.15.16:0/3267949034 wait complete. 2024-09-15T22:46:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.951+0000 7f4493592740 20 client.17208 trim_cache size 0 max 16384 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 Processor -- start 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 -- start start 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c3b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c40dc0 0x556ba2c3be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c62a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1b1970 con 0x556ba2c40dc0 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1af240 con 0x556ba2c404a0 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.968+0000 7f4493592740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1ad0b0 con 0x556ba2c4b680 2024-09-15T22:46:36.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c40dc0 0x556ba2c3be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c40dc0 0x556ba2c3be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38622/0 (socket says 172.21.15.16:38622) 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 -- 172.21.15.16:0/3046071911 learned_addr learned my addr 172.21.15.16:0/3046071911 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448ad9d640 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c3b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448b59e640 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c62a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 -- 172.21.15.16:0/3046071911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 msgr2=0x556ba2c62a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c62a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 -- 172.21.15.16:0/3046071911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 msgr2=0x556ba2c3b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c3b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 -- 172.21.15.16:0/3046071911 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba2c3c860 con 0x556ba2c40dc0 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448ad9d640 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c3b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:36.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f448a59c640 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c40dc0 0x556ba2c3be70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f448009bb20 tx=0x7f448006b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651bee9f053006eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f4470ff9640 1 -- 172.21.15.16:0/3046071911 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44800c0070 con 0x556ba2c40dc0 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f4470ff9640 1 -- 172.21.15.16:0/3046071911 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f448009bb90 con 0x556ba2c40dc0 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.969+0000 7f4470ff9640 1 -- 172.21.15.16:0/3046071911 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44800c0aa0 con 0x556ba2c40dc0 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 -- 172.21.15.16:0/3046071911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c40dc0 msgr2=0x556ba2c3be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c40dc0 0x556ba2c3be70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f448009bb20 tx=0x7f448006b7b0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 -- 172.21.15.16:0/3046071911 shutdown_connections 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c62a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c40dc0 0x556ba2c3be70 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 --2- 172.21.15.16:0/3046071911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c404a0 0x556ba2c3b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 -- 172.21.15.16:0/3046071911 >> 172.21.15.16:0/3046071911 conn(0x556ba2c3ca30 msgr2=0x556ba2c3b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 -- 172.21.15.16:0/3046071911 shutdown_connections 2024-09-15T22:46:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 -- 172.21.15.16:0/3046071911 wait complete. 2024-09-15T22:46:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.970+0000 7f4493592740 1 -- start start 2024-09-15T22:46:36.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 1 Processor -- start 2024-09-15T22:46:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c40dc0 0x556ba2c51800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c51d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1ad700 con 0x556ba2c404a0 2024-09-15T22:46:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1aeb70 con 0x556ba2c40dc0 2024-09-15T22:46:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f448c1addd0 con 0x556ba2c4b680 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f4493592740 10 client.0 _ll_register_callbacks cb 0x7f448001ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38630/0 (socket says 172.21.15.16:38630) 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 -- 172.21.15.16:0/347722853 learned_addr learned my addr 172.21.15.16:0/347722853 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448a59c640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c40dc0 0x556ba2c51800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 -- 172.21.15.16:0/347722853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 msgr2=0x556ba2c51d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448b59e640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c51d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c51d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 -- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c40dc0 msgr2=0x556ba2c51800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c40dc0 0x556ba2c51800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 -- 172.21.15.16:0/347722853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba2cc4a00 con 0x556ba2c404a0 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448a59c640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c40dc0 0x556ba2c51800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448b59e640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c51d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.971+0000 7f448ad9d640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c512c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b6b0 tx=0x7f447c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d99fba4fa4f066 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f446eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f446eff5640 1 -- 172.21.15.16:0/347722853 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c019df0 con 0x556ba2c404a0 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f446eff5640 1 -- 172.21.15.16:0/347722853 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f447c019420 con 0x556ba2c404a0 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f448009bb90 con 0x556ba2c404a0 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f446eff5640 1 -- 172.21.15.16:0/347722853 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f447c03f050 con 0x556ba2c404a0 2024-09-15T22:46:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 10 client.17214 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556ba2cc5c10 con 0x556ba2c404a0 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f446eff5640 1 -- 172.21.15.16:0/347722853 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f448009bb90 con 0x556ba2c404a0 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f446eff5640 1 -- 172.21.15.16:0/347722853 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f447c022050 con 0x556ba2c404a0 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 10 client.17214 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 20 client.17214 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2cc6270 0x556ba2cc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 4 client.17214 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.972+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ba2c493a0 con 0x556ba2cc6270 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f448a59c640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2cc6270 0x556ba2cc8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f446eff5640 10 client.17214 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f448a59c640 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2cc6270 0x556ba2cc8660 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aada52521b3e7e25 server_cookie=bb83e9da34cf3632 in_seq=0 out_seq=0 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f446eff5640 1 -- 172.21.15.16:0/347722853 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556ba2c493a0 con 0x556ba2cc6270 2024-09-15T22:46:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f446eff5640 10 client.17214 handle_command_reply: tid=0 2024-09-15T22:46:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 client.17214 shutdown 2024-09-15T22:46:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 msgr2=0x556ba2c512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c512c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f447c00b6b0 tx=0x7f447c004220 comp rx=0 tx=0).stop 2024-09-15T22:46:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 shutdown_connections 2024-09-15T22:46:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x556ba2cc6270 0x556ba2cc8660 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x556ba2c4b680 0x556ba2c51d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ba2c40dc0 0x556ba2c51800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 --2- 172.21.15.16:0/347722853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556ba2c404a0 0x556ba2c512c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.973+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 >> 172.21.15.16:0/347722853 conn(0x556ba2c3ca30 msgr2=0x556ba2c3ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f446eff5640 0 client.17214 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 shutdown_connections 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f4493592740 1 -- 172.21.15.16:0/347722853 wait complete. 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f4493592740 20 client.17214 trim_cache size 0 max 16384 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:46:36.985 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:46:36.986 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:46:36.986 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:46:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f44740a33b0 msgr2=0x7f44740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f44740a33b0 0x7f44740a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4484004af0 tx=0x7f448402f310 comp rx=0 tx=0).stop 2024-09-15T22:46:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c102790 msgr2=0x7f448c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c102790 0x7f448c198910 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f448000d950 tx=0x7f448000de10 comp rx=0 tx=0).stop 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 shutdown_connections 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f448c106f10 0x7f448c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448c102790 0x7f448c198910 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f44740a33b0 0x7f44740a5870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 --2- 172.21.15.16:0/2725044557 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f448c101de0 0x7f448c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 >> 172.21.15.16:0/2725044557 conn(0x7f448c0fd8c0 msgr2=0x7f448c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 shutdown_connections 2024-09-15T22:46:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:36.974+0000 7f44911b9640 1 -- 172.21.15.16:0/2725044557 wait complete. 2024-09-15T22:46:37.825 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:46:37.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:46:38.105 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:38.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 -- 172.21.15.16:0/3894643488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 msgr2=0x7f0b68100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/3894643488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b68100ee0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009930 tx=0x7f0b5002a420 comp rx=0 tx=0).stop 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 -- 172.21.15.16:0/3894643488 shutdown_connections 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/3894643488 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b68101420 0x7f0b6810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/3894643488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b68100ee0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/3894643488 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b68106a80 0x7f0b68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 -- 172.21.15.16:0/3894643488 >> 172.21.15.16:0/3894643488 conn(0x7f0b680fc7a0 msgr2=0x7f0b680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 -- 172.21.15.16:0/3894643488 shutdown_connections 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.601+0000 7f0b6d04a640 1 -- 172.21.15.16:0/3894643488 wait complete. 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 Processor -- start 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 -- start start 2024-09-15T22:46:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b6819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b68101420 0x7f0b6819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b68106a80 0x7f0b6819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68110e80 con 0x7f0b68100a80 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68110d00 con 0x7f0b68101420 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6d04a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68111000 con 0x7f0b68106a80 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b6819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b6819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38642/0 (socket says 172.21.15.16:38642) 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 -- 172.21.15.16:0/1208188774 learned_addr learned my addr 172.21.15.16:0/1208188774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b5ffff640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b68101420 0x7f0b6819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b6759e640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b68106a80 0x7f0b6819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 -- 172.21.15.16:0/1208188774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b68106a80 msgr2=0x7f0b6819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b68106a80 0x7f0b6819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 -- 172.21.15.16:0/1208188774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b68101420 msgr2=0x7f0b6819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b68101420 0x7f0b6819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.602+0000 7f0b66d9d640 1 -- 172.21.15.16:0/1208188774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6819f5a0 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.603+0000 7f0b66d9d640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b6819a5f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800c920 tx=0x7f0b5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.603+0000 7f0b64d99640 1 -- 172.21.15.16:0/1208188774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58007c70 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.603+0000 7f0b64d99640 1 -- 172.21.15.16:0/1208188774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b58007e10 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.603+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6819f890 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.603+0000 7f0b64d99640 1 -- 172.21.15.16:0/1208188774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58018890 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.603+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681a7170 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.604+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b68100ee0 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.605+0000 7f0b64d99640 1 -- 172.21.15.16:0/1208188774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0b58018a50 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.606+0000 7f0b64d99640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b540a33b0 0x7f0b540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.606+0000 7f0b64d99640 1 -- 172.21.15.16:0/1208188774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f0b580d0390 con 0x7f0b68100a80 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.606+0000 7f0b5ffff640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b540a33b0 0x7f0b540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:38.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.608+0000 7f0b5ffff640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b540a33b0 0x7f0b540a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009900 tx=0x7f0b500058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:38.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.610+0000 7f0b64d99640 1 -- 172.21.15.16:0/1208188774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b58014030 con 0x7f0b68100a80 2024-09-15T22:46:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:38 smithi130 ceph-mon[35742]: pgmap v253: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:46:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:46:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:46:38.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:46:38.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:38 smithi016 ceph-mon[31492]: pgmap v253: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:46:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:46:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:46:38.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:46:38.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.822+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b6819b950 con 0x7f0b68100a80 2024-09-15T22:46:38.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.823+0000 7f0b64d99640 1 -- 172.21.15.16:0/1208188774 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f0b5809ca50 con 0x7f0b68100a80 2024-09-15T22:46:38.824 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:46:38.824 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:46:38.825 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:46:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b540a33b0 msgr2=0x7f0b540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b540a33b0 0x7f0b540a5870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009900 tx=0x7f0b500058d0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 msgr2=0x7f0b6819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:38.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b6819a5f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f0b5800c920 tx=0x7f0b5800cde0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 shutdown_connections 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0b540a33b0 0x7f0b540a5870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0b68106a80 0x7f0b6819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b68101420 0x7f0b6819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 --2- 172.21.15.16:0/1208188774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0b68100a80 0x7f0b6819a5f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 >> 172.21.15.16:0/1208188774 conn(0x7f0b680fc7a0 msgr2=0x7f0b6806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 shutdown_connections 2024-09-15T22:46:38.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:38.825+0000 7f0b6d04a640 1 -- 172.21.15.16:0/1208188774 wait complete. 2024-09-15T22:46:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:38 smithi059 ceph-mon[34872]: pgmap v253: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:46:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:46:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:46:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:46:39.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:39 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1208188774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:39.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:46:39.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:39 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1208188774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:39.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:39 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1208188774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.425+0000 7f460cd26640 1 -- 172.21.15.16:0/1209081948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 msgr2=0x7f46081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.425+0000 7f460cd26640 1 --2- 172.21.15.16:0/1209081948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f46081021c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009900 tx=0x7f45f802a410 comp rx=0 tx=0).stop 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 -- 172.21.15.16:0/1209081948 shutdown_connections 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 --2- 172.21.15.16:0/1209081948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4608106f10 0x7f460810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 --2- 172.21.15.16:0/1209081948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4608102790 0x7f46081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 --2- 172.21.15.16:0/1209081948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f46081021c0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 -- 172.21.15.16:0/1209081948 >> 172.21.15.16:0/1209081948 conn(0x7f46080fd8c0 msgr2=0x7f46080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 -- 172.21.15.16:0/1209081948 shutdown_connections 2024-09-15T22:46:40.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 -- 172.21.15.16:0/1209081948 wait complete. 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 Processor -- start 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.426+0000 7f460cd26640 1 -- start start 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460cd26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f4608198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460cd26640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4608102790 0x7f4608198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460cd26640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4608106f10 0x7f460819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460cd26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4608111f30 con 0x7f4608101de0 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460cd26640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4608111db0 con 0x7f4608106f10 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460cd26640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46081120b0 con 0x7f4608102790 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f4608198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f4608198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38672/0 (socket says 172.21.15.16:38672) 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 -- 172.21.15.16:0/119978249 learned_addr learned my addr 172.21.15.16:0/119978249 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f4606d9d640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4608106f10 0x7f460819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f4605d9b640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4608102790 0x7f4608198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 -- 172.21.15.16:0/119978249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4608102790 msgr2=0x7f4608198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4608102790 0x7f4608198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 -- 172.21.15.16:0/119978249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4608106f10 msgr2=0x7f460819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4608106f10 0x7f460819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 -- 172.21.15.16:0/119978249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460819d3b0 con 0x7f4608101de0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.427+0000 7f460659c640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f4608198400 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005ab0 tx=0x7f45f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.428+0000 7f45f77fe640 1 -- 172.21.15.16:0/119978249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f803f030 con 0x7f4608101de0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.428+0000 7f45f77fe640 1 -- 172.21.15.16:0/119978249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8003ce0 con 0x7f4608101de0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.428+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460819d640 con 0x7f4608101de0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.428+0000 7f45f77fe640 1 -- 172.21.15.16:0/119978249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8035070 con 0x7f4608101de0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.428+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46081a4f80 con 0x7f4608101de0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.430+0000 7f45f77fe640 1 -- 172.21.15.16:0/119978249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f45f8042030 con 0x7f4608101de0 2024-09-15T22:46:40.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.430+0000 7f460f0ff740 1 Processor -- start 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.430+0000 7f460f0ff740 1 -- start start 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7d81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d815d0 0x5570f7d81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d90780 0x5570f7d92b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f460f0ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f7ce72f0 con 0x5570f7d82b40 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f460f0ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5570f7c8ecf0 con 0x5570f7d90780 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f460f0ff740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5570f7ce32b0 con 0x5570f7d815d0 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7d81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7d81090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38676/0 (socket says 172.21.15.16:38676) 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 -- 172.21.15.16:0/372493976 learned_addr learned my addr 172.21.15.16:0/372493976 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f45f77fe640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f45f00a33b0 0x7f45f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f460659c640 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d815d0 0x5570f7d81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4605d9b640 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d90780 0x5570f7d92b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f45f00a33b0 0x7f45f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 -- 172.21.15.16:0/372493976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d815d0 msgr2=0x5570f7d81a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d815d0 0x5570f7d81a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 -- 172.21.15.16:0/372493976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d90780 msgr2=0x5570f7d92b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d90780 0x5570f7d92b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 -- 172.21.15.16:0/372493976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f7c733a0 con 0x5570f7d82b40 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f45f77fe640 1 -- 172.21.15.16:0/119978249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f45f80f0930 con 0x7f4608101de0 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.431+0000 7f4606d9d640 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7d81090 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4600013940 tx=0x7f4600013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8757346abf4160c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f45f4ff9640 1 -- 172.21.15.16:0/372493976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600021030 con 0x5570f7d82b40 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f45f4ff9640 1 -- 172.21.15.16:0/372493976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46000043a0 con 0x5570f7d82b40 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f45f4ff9640 1 -- 172.21.15.16:0/372493976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460001d930 con 0x5570f7d82b40 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f460f0ff740 1 -- 172.21.15.16:0/372493976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 msgr2=0x5570f7d81090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f4606d9d640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f45f00a33b0 0x7f45f00a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4600021000 tx=0x7f4600020000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f460f0ff740 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7d81090 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4600013940 tx=0x7f4600013e00 comp rx=0 tx=0).stop 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f460f0ff740 1 -- 172.21.15.16:0/372493976 shutdown_connections 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.432+0000 7f460f0ff740 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d90780 0x5570f7d92b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d815d0 0x5570f7d81a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 --2- 172.21.15.16:0/372493976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7d81090 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 -- 172.21.15.16:0/372493976 >> 172.21.15.16:0/372493976 conn(0x5570f7d4a310 msgr2=0x5570f7d7dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 -- 172.21.15.16:0/372493976 shutdown_connections 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 -- 172.21.15.16:0/372493976 wait complete. 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 -- start start 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 Processor -- start 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d90160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d8d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d90780 0x5570f7d8d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5570f7d93480 con 0x5570f7d90780 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5570f7ca2060 con 0x5570f7d815d0 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5570f7d93600 con 0x5570f7d82b40 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.433+0000 7f460f0ff740 10 client.0 _ll_register_callbacks cb 0x5570f79f58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d90780 0x5570f7d8d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d90780 0x5570f7d8d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38684/0 (socket says 172.21.15.16:38684) 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 -- 172.21.15.16:0/4189184573 learned_addr learned my addr 172.21.15.16:0/4189184573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f460659c640 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d90160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4605d9b640 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d8d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 -- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 msgr2=0x5570f7d8d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d8d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 -- 172.21.15.16:0/4189184573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 msgr2=0x5570f7d90160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d90160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 -- 172.21.15.16:0/4189184573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f7a583c0 con 0x5570f7d90780 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d90780 0x5570f7d8d9e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f460001da90 tx=0x7f460001d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531e1411eaf85f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f45edffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f45edffb640 1 -- 172.21.15.16:0/4189184573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460001dda0 con 0x5570f7d90780 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f45edffb640 1 -- 172.21.15.16:0/4189184573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f4600004550 con 0x5570f7d90780 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46000043a0 con 0x5570f7d90780 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f45edffb640 1 -- 172.21.15.16:0/4189184573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f460003b050 con 0x5570f7d90780 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f460f0ff740 10 client.17232 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.434+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570f7d9ef00 con 0x5570f7d90780 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f45edffb640 1 -- 172.21.15.16:0/4189184573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46000043a0 con 0x5570f7d90780 2024-09-15T22:46:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f45edffb640 1 -- 172.21.15.16:0/4189184573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4600005050 con 0x5570f7d90780 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460f0ff740 10 client.17232 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460f0ff740 10 client.17232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460f0ff740 20 client.17232 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7da0300 0x5570f7da26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460f0ff740 4 client.17232 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570f7cf4860 con 0x5570f7da0300 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460659c640 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7da0300 0x5570f7da26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f460659c640 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7da0300 0x5570f7da26f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f89c2bcb47919c79 server_cookie=4d196f6acd5f4b9d in_seq=0 out_seq=0 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.435+0000 7f45edffb640 10 client.17232 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:46:40.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.436+0000 7f45edffb640 1 -- 172.21.15.16:0/4189184573 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570f7cf4860 con 0x5570f7da0300 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.436+0000 7f45edffb640 10 client.17232 handle_command_reply: tid=0 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.436+0000 7f460f0ff740 1 client.17232 shutdown 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d90780 msgr2=0x5570f7d8d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d90780 0x5570f7d8d9e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f460001da90 tx=0x7f460001d5b0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 shutdown_connections 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7da0300 0x5570f7da26f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d90780 0x5570f7d8d9e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d8d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f45edffb640 0 client.17232 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:46:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4189184573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d90160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 >> 172.21.15.16:0/4189184573 conn(0x5570f7d4a310 msgr2=0x5570f7d83960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 shutdown_connections 2024-09-15T22:46:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 1 -- 172.21.15.16:0/4189184573 wait complete. 2024-09-15T22:46:40.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.437+0000 7f460f0ff740 20 client.17232 trim_cache size 0 max 16384 2024-09-15T22:46:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 Processor -- start 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 -- start start 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d4a310 0x5570f7d80bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d9c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7da7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46081af6e0 con 0x5570f7d82b40 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46081acfb0 con 0x5570f7d815d0 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f460f0ff740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46081aae20 con 0x5570f7d4a310 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7da7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f4606d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7da7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38700/0 (socket says 172.21.15.16:38700) 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f4606d9d640 1 -- 172.21.15.16:0/4137003854 learned_addr learned my addr 172.21.15.16:0/4137003854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.450+0000 7f4605d9b640 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d9c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f460659c640 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d4a310 0x5570f7d80bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f4606d9d640 1 -- 172.21.15.16:0/4137003854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d4a310 msgr2=0x5570f7d80bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d4a310 0x5570f7d80bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f4606d9d640 1 -- 172.21.15.16:0/4137003854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 msgr2=0x5570f7d9c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d9c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f4606d9d640 1 -- 172.21.15.16:0/4137003854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f7d8b960 con 0x5570f7d82b40 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f4605d9b640 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d9c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7da7e60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4600013e40 tx=0x7f4600005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743bf7e44931d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f45effff640 1 -- 172.21.15.16:0/4137003854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600043070 con 0x5570f7d82b40 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f45effff640 1 -- 172.21.15.16:0/4137003854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600029070 con 0x5570f7d82b40 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.451+0000 7f45effff640 1 -- 172.21.15.16:0/4137003854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4600004fd0 con 0x5570f7d82b40 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- 172.21.15.16:0/4137003854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 msgr2=0x5570f7da7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7da7e60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4600013e40 tx=0x7f4600005f60 comp rx=0 tx=0).stop 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- 172.21.15.16:0/4137003854 shutdown_connections 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d82b40 0x5570f7da7e60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d815d0 0x5570f7d9c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4137003854 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d4a310 0x5570f7d80bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- 172.21.15.16:0/4137003854 >> 172.21.15.16:0/4137003854 conn(0x5570f7d88e00 msgr2=0x5570f7d8bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- 172.21.15.16:0/4137003854 shutdown_connections 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- 172.21.15.16:0/4137003854 wait complete. 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- start start 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 Processor -- start 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d4a310 0x5570f7d945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d815d0 0x5570f7d94b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d95040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ab470 con 0x5570f7d815d0 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46081ac8e0 con 0x5570f7d4a310 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46081abb40 con 0x5570f7d82b40 2024-09-15T22:46:40.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f460f0ff740 10 client.0 _ll_register_callbacks cb 0x7f45fc00a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.452+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d815d0 0x5570f7d94b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d815d0 0x5570f7d94b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38716/0 (socket says 172.21.15.16:38716) 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f460659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d4a310 0x5570f7d945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 -- 172.21.15.16:0/4249701840 learned_addr learned my addr 172.21.15.16:0/4249701840 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d95040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 -- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 msgr2=0x5570f7d95040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d95040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 -- 172.21.15.16:0/4249701840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d4a310 msgr2=0x5570f7d945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d4a310 0x5570f7d945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 -- 172.21.15.16:0/4249701840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f7e09e30 con 0x5570f7d815d0 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4606d9d640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d95040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f460659c640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d4a310 0x5570f7d945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f4605d9b640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d815d0 0x5570f7d94b00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f46080fdce0 tx=0x7f45fc010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac583d8c79036e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f45edffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f45edffb640 1 -- 172.21.15.16:0/4249701840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45fc004020 con 0x5570f7d815d0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f45edffb640 1 -- 172.21.15.16:0/4249701840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f45fc0041c0 con 0x5570f7d815d0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4600029070 con 0x5570f7d815d0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f45edffb640 1 -- 172.21.15.16:0/4249701840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45fc028050 con 0x5570f7d815d0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f460f0ff740 10 client.17238 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.453+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5570f7e0b170 con 0x5570f7d815d0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f45edffb640 1 -- 172.21.15.16:0/4249701840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4600029070 con 0x5570f7d815d0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f45edffb640 1 -- 172.21.15.16:0/4249701840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f45fc003b50 con 0x5570f7d815d0 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f460f0ff740 10 client.17238 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f460f0ff740 10 client.17238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f460f0ff740 20 client.17238 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7d90780 0x5570f7e0d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f460f0ff740 4 client.17238 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570f7ceed70 con 0x5570f7d90780 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.454+0000 7f460659c640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7d90780 0x5570f7e0d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460659c640 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7d90780 0x5570f7e0d9e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9389a97017bc66c server_cookie=7299656e1f405997 in_seq=0 out_seq=0 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f45edffb640 10 client.17238 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f45edffb640 1 -- 172.21.15.16:0/4249701840 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570f7ceed70 con 0x5570f7d90780 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f45edffb640 10 client.17238 handle_command_reply: tid=0 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 client.17238 shutdown 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d815d0 msgr2=0x5570f7d94b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d815d0 0x5570f7d94b00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f46080fdce0 tx=0x7f45fc010af0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 shutdown_connections 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x5570f7d90780 0x5570f7e0d9e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5570f7d82b40 0x5570f7d95040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5570f7d815d0 0x5570f7d94b00 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 --2- 172.21.15.16:0/4249701840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5570f7d4a310 0x5570f7d945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 >> 172.21.15.16:0/4249701840 conn(0x5570f7d88e00 msgr2=0x5570f7d8ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f45edffb640 0 client.17238 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 shutdown_connections 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 1 -- 172.21.15.16:0/4249701840 wait complete. 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.455+0000 7f460f0ff740 20 client.17238 trim_cache size 0 max 16384 2024-09-15T22:46:40.465 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:46:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f45f00a33b0 msgr2=0x7f45f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f45f00a33b0 0x7f45f00a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4600021000 tx=0x7f4600020000 comp rx=0 tx=0).stop 2024-09-15T22:46:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 msgr2=0x7f4608198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f4608198400 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005ab0 tx=0x7f45f80042f0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 shutdown_connections 2024-09-15T22:46:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4608106f10 0x7f460819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f45f00a33b0 0x7f45f00a5870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4608102790 0x7f4608198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 --2- 172.21.15.16:0/119978249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608101de0 0x7f4608198400 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 >> 172.21.15.16:0/119978249 conn(0x7f46080fd8c0 msgr2=0x7f46080fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 shutdown_connections 2024-09-15T22:46:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:40.456+0000 7f460cd26640 1 -- 172.21.15.16:0/119978249 wait complete. 2024-09-15T22:46:40.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:40 smithi130 ceph-mon[35742]: pgmap v254: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:40.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:40 smithi016 ceph-mon[31492]: pgmap v254: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:40.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:40 smithi059 ceph-mon[34872]: pgmap v254: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:41.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:46:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 -- 172.21.15.16:0/1683648348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764101420 msgr2=0x7fa76410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 --2- 172.21.15.16:0/1683648348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764101420 0x7fa76410d950 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009cd0 tx=0x7fa75c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 -- 172.21.15.16:0/1683648348 shutdown_connections 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 --2- 172.21.15.16:0/1683648348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764101420 0x7fa76410d950 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 --2- 172.21.15.16:0/1683648348 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa764100a80 0x7fa764100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 --2- 172.21.15.16:0/1683648348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa764106a80 0x7fa764106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 -- 172.21.15.16:0/1683648348 >> 172.21.15.16:0/1683648348 conn(0x7fa7640fc7a0 msgr2=0x7fa7640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 -- 172.21.15.16:0/1683648348 shutdown_connections 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.960+0000 7fa768aba640 1 -- 172.21.15.16:0/1683648348 wait complete. 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 Processor -- start 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 -- start start 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa764100a80 0x7fa7641983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa764101420 0x7fa7641988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764106a80 0x7fa76419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa764110e80 con 0x7fa764106a80 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa764110d00 con 0x7fa764101420 2024-09-15T22:46:41.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.961+0000 7fa768aba640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa764111000 con 0x7fa764100a80 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764106a80 0x7fa76419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764106a80 0x7fa76419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59196/0 (socket says 172.21.15.16:59196) 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 -- 172.21.15.16:0/2500611937 learned_addr learned my addr 172.21.15.16:0/2500611937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa76259c640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa764100a80 0x7fa7641983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa75bfff640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa764101420 0x7fa7641988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 -- 172.21.15.16:0/2500611937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa764100a80 msgr2=0x7fa7641983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa764100a80 0x7fa7641983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 -- 172.21.15.16:0/2500611937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa764101420 msgr2=0x7fa7641988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa764101420 0x7fa7641988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 -- 172.21.15.16:0/2500611937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76419d360 con 0x7fa764106a80 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa76259c640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa764100a80 0x7fa7641983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa762d9d640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764106a80 0x7fa76419cc80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa75c006010 tx=0x7fa75c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa75b7fe640 1 -- 172.21.15.16:0/2500611937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c004450 con 0x7fa764106a80 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa75b7fe640 1 -- 172.21.15.16:0/2500611937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa75c003e10 con 0x7fa764106a80 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76419d5f0 con 0x7fa764106a80 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa75b7fe640 1 -- 172.21.15.16:0/2500611937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0466e0 con 0x7fa764106a80 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.962+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7641a4f30 con 0x7fa764106a80 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa75b7fe640 1 -- 172.21.15.16:0/2500611937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa75c0305a0 con 0x7fa764106a80 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 Processor -- start 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 -- start start 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b69090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b69a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562bb8acf2f0 con 0x562bb8b6ab40 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562bb8a76cf0 con 0x562bb8b695d0 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.965+0000 7fa76ae93740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562bb8acb2b0 con 0x562bb8b74370 2024-09-15T22:46:41.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b69090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b69090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59208/0 (socket says 172.21.15.16:59208) 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 -- 172.21.15.16:0/250094085 learned_addr learned my addr 172.21.15.16:0/250094085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75b7fe640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa7500a33b0 0x7fa7500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa76259c640 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b69a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa762d9d640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa7500a33b0 0x7fa7500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 -- 172.21.15.16:0/250094085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 msgr2=0x562bb8b76760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b76760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 -- 172.21.15.16:0/250094085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 msgr2=0x562bb8b69a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b69a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 -- 172.21.15.16:0/250094085 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb8a5b3a0 con 0x562bb8b6ab40 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75b7fe640 1 -- 172.21.15.16:0/2500611937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7fa75c02e030 con 0x7fa764106a80 2024-09-15T22:46:41.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa75bfff640 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b69090 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00cf70 tx=0x7fa74c00c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab3fe0e130a6027 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa758ff9640 1 -- 172.21.15.16:0/250094085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c018030 con 0x562bb8b6ab40 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa758ff9640 1 -- 172.21.15.16:0/250094085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c005500 con 0x562bb8b6ab40 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa758ff9640 1 -- 172.21.15.16:0/250094085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0108d0 con 0x562bb8b6ab40 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.966+0000 7fa762d9d640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa7500a33b0 0x7fa7500a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0bd4e0 tx=0x7fa75c08f560 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 -- 172.21.15.16:0/250094085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 msgr2=0x562bb8b69090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b69090 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00cf70 tx=0x7fa74c00c620 comp rx=0 tx=0).stop 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 -- 172.21.15.16:0/250094085 shutdown_connections 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b76760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b69a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 --2- 172.21.15.16:0/250094085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b69090 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 -- 172.21.15.16:0/250094085 >> 172.21.15.16:0/250094085 conn(0x562bb8b32310 msgr2=0x562bb8b65c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 -- 172.21.15.16:0/250094085 shutdown_connections 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.967+0000 7fa76ae93740 1 -- 172.21.15.16:0/250094085 wait complete. 2024-09-15T22:46:41.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 -- start start 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 Processor -- start 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b70fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b71530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562bb8b77070 con 0x562bb8b6ab40 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562bb8a8a060 con 0x562bb8b695d0 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x562bb8b771f0 con 0x562bb8b74370 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76ae93740 10 client.0 _ll_register_callbacks cb 0x562bb87dd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b70fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b70fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59210/0 (socket says 172.21.15.16:59210) 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa76259c640 1 -- 172.21.15.16:0/4074786696 learned_addr learned my addr 172.21.15.16:0/4074786696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.968+0000 7fa75bfff640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa75bfff640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b71530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76259c640 1 -- 172.21.15.16:0/4074786696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 msgr2=0x562bb8b71530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76259c640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b71530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76259c640 1 -- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 msgr2=0x562bb8b73d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76259c640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b73d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76259c640 1 -- 172.21.15.16:0/4074786696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb88403c0 con 0x562bb8b6ab40 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa75bfff640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b71530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76259c640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b70fb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa754011850 tx=0x7fa754011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cee8caeeef296ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa749ffb640 1 -- 172.21.15.16:0/4074786696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75400fdb0 con 0x562bb8b6ab40 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa749ffb640 1 -- 172.21.15.16:0/4074786696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7fa754004020 con 0x562bb8b6ab40 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74c005500 con 0x562bb8b6ab40 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa749ffb640 1 -- 172.21.15.16:0/4074786696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa754024050 con 0x562bb8b6ab40 2024-09-15T22:46:41.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76ae93740 10 client.17250 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562bb8b82b50 con 0x562bb8b6ab40 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.969+0000 7fa749ffb640 1 -- 172.21.15.16:0/4074786696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c005500 con 0x562bb8b6ab40 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa749ffb640 1 -- 172.21.15.16:0/4074786696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa754018030 con 0x562bb8b6ab40 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa76ae93740 10 client.17250 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa76ae93740 10 client.17250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa76ae93740 20 client.17250 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa76ae93740 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8b840b0 0x562bb8b864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa76ae93740 4 client.17250 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bb8adc860 con 0x562bb8b840b0 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa75bfff640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8b840b0 0x562bb8b864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa75bfff640 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8b840b0 0x562bb8b864d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bc8038fe1a415c6 server_cookie=8152df8aca5acdbb in_seq=0 out_seq=0 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.970+0000 7fa749ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa749ffb640 1 -- 172.21.15.16:0/4074786696 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562bb8adc860 con 0x562bb8b840b0 2024-09-15T22:46:41.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa749ffb640 10 client.17250 handle_command_reply: tid=0 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 client.17250 shutdown 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 msgr2=0x562bb8b70fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b70fb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa754011850 tx=0x7fa754011d10 comp rx=0 tx=0).stop 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 shutdown_connections 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8b840b0 0x562bb8b864d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b74370 0x562bb8b71530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b6ab40 0x562bb8b70fb0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa749ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 --2- 172.21.15.16:0/4074786696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b695d0 0x562bb8b73d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 >> 172.21.15.16:0/4074786696 conn(0x562bb8b32310 msgr2=0x562bb8b6b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 shutdown_connections 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.971+0000 7fa76ae93740 1 -- 172.21.15.16:0/4074786696 wait complete. 2024-09-15T22:46:41.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.972+0000 7fa76ae93740 20 client.17250 trim_cache size 0 max 16384 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 Processor -- start 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 -- start start 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b695d0 0x562bb8b80310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b6ab40 0x562bb8b8bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b74370 0x562bb8b8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641b17c0 con 0x562bb8b695d0 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641af090 con 0x562bb8b74370 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76ae93740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641acf00 con 0x562bb8b6ab40 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b695d0 0x562bb8b80310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b695d0 0x562bb8b80310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59224/0 (socket says 172.21.15.16:59224) 2024-09-15T22:46:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 -- 172.21.15.16:0/2710709210 learned_addr learned my addr 172.21.15.16:0/2710709210 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76259c640 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b6ab40 0x562bb8b8bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 -- 172.21.15.16:0/2710709210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b6ab40 msgr2=0x562bb8b8bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b6ab40 0x562bb8b8bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 -- 172.21.15.16:0/2710709210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b74370 msgr2=0x562bb8b8c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b74370 0x562bb8b8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 -- 172.21.15.16:0/2710709210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bb8b64310 con 0x562bb8b695d0 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa76259c640 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b6ab40 0x562bb8b8bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.985+0000 7fa75bfff640 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b695d0 0x562bb8b80310 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa74c012e30 tx=0x7fa74c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ab25abe3c04df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa74bfff640 1 -- 172.21.15.16:0/2710709210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c018040 con 0x562bb8b695d0 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa74bfff640 1 -- 172.21.15.16:0/2710709210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c019040 con 0x562bb8b695d0 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa74bfff640 1 -- 172.21.15.16:0/2710709210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c005240 con 0x562bb8b695d0 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 -- 172.21.15.16:0/2710709210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b695d0 msgr2=0x562bb8b80310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b695d0 0x562bb8b80310 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa74c012e30 tx=0x7fa74c00cee0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 -- 172.21.15.16:0/2710709210 shutdown_connections 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b74370 0x562bb8b8c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b6ab40 0x562bb8b8bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 --2- 172.21.15.16:0/2710709210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b695d0 0x562bb8b80310 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 -- 172.21.15.16:0/2710709210 >> 172.21.15.16:0/2710709210 conn(0x562bb8b32310 msgr2=0x562bb8b700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 -- 172.21.15.16:0/2710709210 shutdown_connections 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.986+0000 7fa76ae93740 1 -- 172.21.15.16:0/2710709210 wait complete. 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 -- start start 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 Processor -- start 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b695d0 0x562bb8b7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b6ab40 0x562bb8b77e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b74370 0x562bb8b78340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641ad550 con 0x562bb8b74370 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641ae9c0 con 0x562bb8b6ab40 2024-09-15T22:46:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa7641adc20 con 0x562bb8b695d0 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76ae93740 10 client.0 _ll_register_callbacks cb 0x7fa75400c7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b74370 0x562bb8b78340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa75bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b74370 0x562bb8b78340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59230/0 (socket says 172.21.15.16:59230) 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa75bfff640 1 -- 172.21.15.16:0/3575590668 learned_addr learned my addr 172.21.15.16:0/3575590668 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa76259c640 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b6ab40 0x562bb8b77e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.987+0000 7fa75bfff640 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b695d0 0x562bb8b7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa75bfff640 1 -- 172.21.15.16:0/3575590668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b695d0 msgr2=0x562bb8b7ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa75bfff640 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b695d0 0x562bb8b7ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa75bfff640 1 -- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b6ab40 msgr2=0x562bb8b77e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa75bfff640 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b6ab40 0x562bb8b77e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa75bfff640 1 -- 172.21.15.16:0/3575590668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bb8b6c120 con 0x562bb8b74370 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa75bfff640 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b74370 0x562bb8b78340 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00c520 tx=0x7fa74c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a4acfb922061a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa749ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa749ffb640 1 -- 172.21.15.16:0/3575590668 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c018070 con 0x562bb8b74370 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa749ffb640 1 -- 172.21.15.16:0/3575590668 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7fa74c004840 con 0x562bb8b74370 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74c019040 con 0x562bb8b74370 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa749ffb640 1 -- 172.21.15.16:0/3575590668 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa74c004a40 con 0x562bb8b74370 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa76ae93740 10 client.17256 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562bb8b66960 con 0x562bb8b74370 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa749ffb640 1 -- 172.21.15.16:0/3575590668 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c019040 con 0x562bb8b74370 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa749ffb640 1 -- 172.21.15.16:0/3575590668 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa74c010ba0 con 0x562bb8b74370 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa76ae93740 10 client.17256 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa76ae93740 10 client.17256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.988+0000 7fa76ae93740 20 client.17256 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa76ae93740 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8beec50 0x562bb8bf1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa76ae93740 4 client.17256 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:46:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bb8ad6d70 con 0x562bb8beec50 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa76259c640 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8beec50 0x562bb8bf1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa76259c640 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8beec50 0x562bb8bf1040 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=429e178227bb9c82 server_cookie=7daca223aae11390 in_seq=0 out_seq=0 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa749ffb640 10 client.17256 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa749ffb640 1 -- 172.21.15.16:0/3575590668 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bb8ad6d70 con 0x562bb8beec50 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa749ffb640 10 client.17256 handle_command_reply: tid=0 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.989+0000 7fa76ae93740 1 client.17256 shutdown 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b74370 msgr2=0x562bb8b78340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b74370 0x562bb8b78340 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00c520 tx=0x7fa74c0044a0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 shutdown_connections 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x562bb8beec50 0x562bb8bf1040 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562bb8b74370 0x562bb8b78340 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562bb8b6ab40 0x562bb8b77e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 --2- 172.21.15.16:0/3575590668 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x562bb8b695d0 0x562bb8b7ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 >> 172.21.15.16:0/3575590668 conn(0x562bb8b32310 msgr2=0x562bb8b72810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa749ffb640 0 client.17256 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 shutdown_connections 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 1 -- 172.21.15.16:0/3575590668 wait complete. 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa76ae93740 20 client.17256 trim_cache size 0 max 16384 2024-09-15T22:46:42.005 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa7500a33b0 msgr2=0x7fa7500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa7500a33b0 0x7fa7500a5870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0bd4e0 tx=0x7fa75c08f560 comp rx=0 tx=0).stop 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764106a80 msgr2=0x7fa76419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764106a80 0x7fa76419cc80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa75c006010 tx=0x7fa75c003c20 comp rx=0 tx=0).stop 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 shutdown_connections 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa764106a80 0x7fa76419cc80 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa764101420 0x7fa7641988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa7500a33b0 0x7fa7500a5870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 --2- 172.21.15.16:0/2500611937 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa764100a80 0x7fa7641983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 >> 172.21.15.16:0/2500611937 conn(0x7fa7640fc7a0 msgr2=0x7fa76406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 shutdown_connections 2024-09-15T22:46:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:41.990+0000 7fa768aba640 1 -- 172.21.15.16:0/2500611937 wait complete. 2024-09-15T22:46:42.648 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:42 smithi016 ceph-mon[31492]: pgmap v255: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:42.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:46:42.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:42 smithi130 ceph-mon[35742]: pgmap v255: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:42.934 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:42.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:42 smithi059 ceph-mon[34872]: pgmap v255: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:46:43.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 -- 172.21.15.16:0/1221871524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 msgr2=0x7f2de4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 --2- 172.21.15.16:0/1221871524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de4101200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009900 tx=0x7f2dcc02a480 comp rx=0 tx=0).stop 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 -- 172.21.15.16:0/1221871524 shutdown_connections 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 --2- 172.21.15.16:0/1221871524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4102ef0 0x7f2de410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 --2- 172.21.15.16:0/1221871524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2de4102570 0x7f2de41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 --2- 172.21.15.16:0/1221871524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de4101200 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 -- 172.21.15.16:0/1221871524 >> 172.21.15.16:0/1221871524 conn(0x7f2de40fc7d0 msgr2=0x7f2de40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 -- 172.21.15.16:0/1221871524 shutdown_connections 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.427+0000 7f2deae0f640 1 -- 172.21.15.16:0/1221871524 wait complete. 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 Processor -- start 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 -- start start 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2de4102570 0x7f2de41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4102ef0 0x7f2de419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4110f80 con 0x7f2de4100e20 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4110e00 con 0x7f2de4102ef0 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2deae0f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4111100 con 0x7f2de4102570 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59250/0 (socket says 172.21.15.16:59250) 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 -- 172.21.15.16:0/2183782678 learned_addr learned my addr 172.21.15.16:0/2183782678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de3fff640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2de4102570 0x7f2de41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de93ac640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4102ef0 0x7f2de419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 -- 172.21.15.16:0/2183782678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2de4102570 msgr2=0x7f2de41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2de4102570 0x7f2de41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 -- 172.21.15.16:0/2183782678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4102ef0 msgr2=0x7f2de419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4102ef0 0x7f2de419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 -- 172.21.15.16:0/2183782678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de419d460 con 0x7f2de4100e20 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.428+0000 7f2de8bab640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de41984b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc006010 tx=0x7f2dcc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.429+0000 7f2de1ffb640 1 -- 172.21.15.16:0/2183782678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc03f070 con 0x7f2de4100e20 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.429+0000 7f2de1ffb640 1 -- 172.21.15.16:0/2183782678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dcc031c30 con 0x7f2de4100e20 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.429+0000 7f2de1ffb640 1 -- 172.21.15.16:0/2183782678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc035030 con 0x7f2de4100e20 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.429+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de419d6f0 con 0x7f2de4100e20 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.429+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de41a5030 con 0x7f2de4100e20 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2de1ffb640 1 -- 172.21.15.16:0/2183782678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2dcc042070 con 0x7f2de4100e20 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 Processor -- start 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 -- start start 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf251e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf251ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ecf24842f0 con 0x55ecf2529370 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ecf242bcf0 con 0x55ecf251e5d0 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ecf24802b0 con 0x55ecf251fb40 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2de3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2de3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59260/0 (socket says 172.21.15.16:59260) 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.432+0000 7f2de3fff640 1 -- 172.21.15.16:0/2250124268 learned_addr learned my addr 172.21.15.16:0/2250124268 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de8bab640 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf251ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de93ac640 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf251e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de3fff640 1 -- 172.21.15.16:0/2250124268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 msgr2=0x55ecf251e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de3fff640 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf251e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de3fff640 1 -- 172.21.15.16:0/2250124268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 msgr2=0x55ecf251ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de3fff640 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf251ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de3fff640 1 -- 172.21.15.16:0/2250124268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf24103a0 con 0x55ecf2529370 2024-09-15T22:46:43.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de1ffb640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2dd00a3430 0x7f2dd00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de3fff640 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252b760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4010420 tx=0x7f2dd4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1ababf4b4dc141 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de93ac640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2dd00a3430 0x7f2dd00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2de1ffb640 1 -- 172.21.15.16:0/2183782678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f2dcc0f08b0 con 0x7f2de4100e20 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2ddaffd640 1 -- 172.21.15.16:0/2250124268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4016070 con 0x55ecf2529370 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2ddaffd640 1 -- 172.21.15.16:0/2250124268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd4004d20 con 0x55ecf2529370 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.433+0000 7f2ddaffd640 1 -- 172.21.15.16:0/2250124268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4005540 con 0x55ecf2529370 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2250124268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 msgr2=0x55ecf252b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2de93ac640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2dd00a3430 0x7f2dd00a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00cc20 tx=0x7f2ddc005ec0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252b760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4010420 tx=0x7f2dd4009f90 comp rx=0 tx=0).stop 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2250124268 shutdown_connections 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252b760 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf251ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2250124268 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf251e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2250124268 >> 172.21.15.16:0/2250124268 conn(0x55ecf24e7310 msgr2=0x55ecf251ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2250124268 shutdown_connections 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2250124268 wait complete. 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 -- start start 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.434+0000 7f2ded1e8740 1 Processor -- start 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf2528d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2526010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf2529370 0x55ecf2526600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ecf252c070 con 0x55ecf251fb40 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ecf243f060 con 0x55ecf251e5d0 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55ecf252c1f0 con 0x55ecf2529370 2024-09-15T22:46:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2ded1e8740 10 client.0 _ll_register_callbacks cb 0x55ecf21928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2526010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2526010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59276/0 (socket says 172.21.15.16:59276) 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 -- 172.21.15.16:0/2125563542 learned_addr learned my addr 172.21.15.16:0/2125563542 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de8bab640 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf2528d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de93ac640 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf2529370 0x55ecf2526600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 -- 172.21.15.16:0/2125563542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf2529370 msgr2=0x55ecf2526600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf2529370 0x55ecf2526600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 -- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 msgr2=0x55ecf2528d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf2528d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.435+0000 7f2de3fff640 1 -- 172.21.15.16:0/2125563542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf21f53c0 con 0x55ecf251fb40 2024-09-15T22:46:43.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2de3fff640 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2526010 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4006010 tx=0x7f2dd4004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa10a6b6ce46472 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2dc3fff640 1 -- 172.21.15.16:0/2125563542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4016070 con 0x55ecf251fb40 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2dc3fff640 1 -- 172.21.15.16:0/2125563542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f2dd4003b80 con 0x55ecf251fb40 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd4004d20 con 0x55ecf251fb40 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2dc3fff640 1 -- 172.21.15.16:0/2125563542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2dd4003d80 con 0x55ecf251fb40 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 10 client.17268 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecf2537b10 con 0x55ecf251fb40 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2dc3fff640 1 -- 172.21.15.16:0/2125563542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd4004d20 con 0x55ecf251fb40 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2dc3fff640 1 -- 172.21.15.16:0/2125563542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2dd400f040 con 0x55ecf251fb40 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 10 client.17268 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 20 client.17268 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:43.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf2538f70 0x55ecf253b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 4 client.17268 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.436+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecf2491860 con 0x55ecf2538f70 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.437+0000 7f2de8bab640 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf2538f70 0x55ecf253b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.437+0000 7f2de8bab640 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf2538f70 0x55ecf253b360 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bae73cc25b137ca server_cookie=ffb03eca19179e72 in_seq=0 out_seq=0 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.437+0000 7f2dc3fff640 10 client.17268 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2dc3fff640 1 -- 172.21.15.16:0/2125563542 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ecf2491860 con 0x55ecf2538f70 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2dc3fff640 10 client.17268 handle_command_reply: tid=0 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 client.17268 shutdown 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 msgr2=0x55ecf2526010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2526010 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4006010 tx=0x7f2dd4004c70 comp rx=0 tx=0).stop 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 shutdown_connections 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf2538f70 0x55ecf253b360 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf2529370 0x55ecf2526600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2526010 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2dc3fff640 0 client.17268 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:43.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/2125563542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf2528d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 >> 172.21.15.16:0/2125563542 conn(0x55ecf24e7310 msgr2=0x55ecf2520960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:43.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 shutdown_connections 2024-09-15T22:46:43.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 1 -- 172.21.15.16:0/2125563542 wait complete. 2024-09-15T22:46:43.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.438+0000 7f2ded1e8740 20 client.17268 trim_cache size 0 max 16384 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.451+0000 7f2ded1e8740 1 Processor -- start 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.451+0000 7f2ded1e8740 1 -- start start 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251e5d0 0x55ecf2521120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2521660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf2529370 0x55ecf2540b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41b1760 con 0x55ecf251fb40 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41af030 con 0x55ecf2529370 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41acea0 con 0x55ecf251e5d0 2024-09-15T22:46:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2521660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2521660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59282/0 (socket says 172.21.15.16:59282) 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 -- 172.21.15.16:0/3772321322 learned_addr learned my addr 172.21.15.16:0/3772321322 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de93ac640 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf2529370 0x55ecf2540b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de8bab640 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251e5d0 0x55ecf2521120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 -- 172.21.15.16:0/3772321322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251e5d0 msgr2=0x55ecf2521120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251e5d0 0x55ecf2521120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 -- 172.21.15.16:0/3772321322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf2529370 msgr2=0x55ecf2540b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf2529370 0x55ecf2540b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 -- 172.21.15.16:0/3772321322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf25290d0 con 0x55ecf251fb40 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de93ac640 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf2529370 0x55ecf2540b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2de3fff640 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2521660 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009930 tx=0x7f2dd4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abf9808c62ff149 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2dda7fc640 1 -- 172.21.15.16:0/3772321322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4016070 con 0x55ecf251fb40 2024-09-15T22:46:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2dda7fc640 1 -- 172.21.15.16:0/3772321322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd4009bf0 con 0x55ecf251fb40 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.452+0000 7f2dda7fc640 1 -- 172.21.15.16:0/3772321322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4012510 con 0x55ecf251fb40 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 -- 172.21.15.16:0/3772321322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 msgr2=0x55ecf2521660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2521660 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009930 tx=0x7f2dd4005040 comp rx=0 tx=0).stop 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 -- 172.21.15.16:0/3772321322 shutdown_connections 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf2529370 0x55ecf2540b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf251fb40 0x55ecf2521660 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/3772321322 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251e5d0 0x55ecf2521120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 -- 172.21.15.16:0/3772321322 >> 172.21.15.16:0/3772321322 conn(0x55ecf24e7310 msgr2=0x55ecf2528090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 -- 172.21.15.16:0/3772321322 shutdown_connections 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 -- 172.21.15.16:0/3772321322 wait complete. 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.453+0000 7f2ded1e8740 1 -- start start 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 1 Processor -- start 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf252d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf252d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41ad4f0 con 0x55ecf2529370 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41ae960 con 0x55ecf251e5d0 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41adbc0 con 0x55ecf251fb40 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2ded1e8740 10 client.0 _ll_register_callbacks cb 0x7f2dcc0f6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de93ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de93ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59292/0 (socket says 172.21.15.16:59292) 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de93ac640 1 -- 172.21.15.16:0/1334328545 learned_addr learned my addr 172.21.15.16:0/1334328545 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de8bab640 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf252d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de3fff640 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf252d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de93ac640 1 -- 172.21.15.16:0/1334328545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 msgr2=0x55ecf252d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de93ac640 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf252d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de93ac640 1 -- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 msgr2=0x55ecf252d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.454+0000 7f2de93ac640 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf252d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2de93ac640 1 -- 172.21.15.16:0/1334328545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf25a2cb0 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2de93ac640 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252dde0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc011e40 tx=0x7f2ddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5b7e4624110308 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2dc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2dc3fff640 1 -- 172.21.15.16:0/1334328545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc020440 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2dc3fff640 1 -- 172.21.15.16:0/1334328545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f2ddc0205e0 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd4009bf0 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2dc3fff640 1 -- 172.21.15.16:0/1334328545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2ddc033050 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2ded1e8740 10 client.17274 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ecf25a3f40 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.455+0000 7f2dc3fff640 1 -- 172.21.15.16:0/1334328545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd4009bf0 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2dc3fff640 1 -- 172.21.15.16:0/1334328545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2ddc028980 con 0x55ecf2529370 2024-09-15T22:46:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 10 client.17274 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 20 client.17274 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf25a4710 0x55ecf25a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 4 client.17274 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecf248bd70 con 0x55ecf25a4710 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2de8bab640 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf25a4710 0x55ecf25a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2dc3fff640 10 client.17274 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2de8bab640 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf25a4710 0x55ecf25a6b30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2599da32992193f7 server_cookie=e7c14ad2e72d184 in_seq=0 out_seq=0 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2dc3fff640 1 -- 172.21.15.16:0/1334328545 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecf248bd70 con 0x55ecf25a4710 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2dc3fff640 10 client.17274 handle_command_reply: tid=0 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 1 client.17274 shutdown 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.456+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 msgr2=0x55ecf252dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252dde0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc011e40 tx=0x7f2ddc0046c0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 shutdown_connections 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55ecf25a4710 0x55ecf25a6b30 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ecf2529370 0x55ecf252dde0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55ecf251fb40 0x55ecf252d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 --2- 172.21.15.16:0/1334328545 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ecf251e5d0 0x55ecf252d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 >> 172.21.15.16:0/1334328545 conn(0x55ecf24e7310 msgr2=0x55ecf25194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2dc3fff640 0 client.17274 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 shutdown_connections 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 1 -- 172.21.15.16:0/1334328545 wait complete. 2024-09-15T22:46:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.457+0000 7f2ded1e8740 20 client.17274 trim_cache size 0 max 16384 2024-09-15T22:46:43.466 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:46:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2dd00a3430 msgr2=0x7f2dd00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2dd00a3430 0x7f2dd00a58f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00cc20 tx=0x7f2ddc005ec0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 msgr2=0x7f2de41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de41984b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc006010 tx=0x7f2dcc003c10 comp rx=0 tx=0).stop 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 shutdown_connections 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2de4102ef0 0x7f2de419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2dd00a3430 0x7f2dd00a58f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2de4102570 0x7f2de41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 --2- 172.21.15.16:0/2183782678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2de4100e20 0x7f2de41984b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 >> 172.21.15.16:0/2183782678 conn(0x7f2de40fc7d0 msgr2=0x7f2de406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 shutdown_connections 2024-09-15T22:46:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:43.458+0000 7f2deae0f640 1 -- 172.21.15.16:0/2183782678 wait complete. 2024-09-15T22:46:44.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:44 smithi130 ceph-mon[35742]: pgmap v256: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:44.993 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:44 smithi059 ceph-mon[34872]: pgmap v256: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:44.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:44 smithi016 ceph-mon[31492]: pgmap v256: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:45.162 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:46:45.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:46:45.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 -- 172.21.15.16:0/4102176611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04102790 msgr2=0x7f4f041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/4102176611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04102790 0x7f4f041067e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009900 tx=0x7f4ef802a410 comp rx=0 tx=0).stop 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 -- 172.21.15.16:0/4102176611 shutdown_connections 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/4102176611 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f04106f10 0x7f4f0410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/4102176611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04102790 0x7f4f041067e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/4102176611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f04101de0 0x7f4f041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 -- 172.21.15.16:0/4102176611 >> 172.21.15.16:0/4102176611 conn(0x7f4f040fd8c0 msgr2=0x7f4f040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 -- 172.21.15.16:0/4102176611 shutdown_connections 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.937+0000 7f4f0a03d640 1 -- 172.21.15.16:0/4102176611 wait complete. 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 Processor -- start 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 -- start start 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f04101de0 0x7f4f040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f04102790 0x7f4f04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04106f10 0x7f4f04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04111f70 con 0x7f4f04106f10 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04111df0 con 0x7f4f04101de0 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f0a03d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041120f0 con 0x7f4f04102790 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.938+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04106f10 0x7f4f04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04106f10 0x7f4f04074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59312/0 (socket says 172.21.15.16:59312) 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 -- 172.21.15.16:0/3315555721 learned_addr learned my addr 172.21.15.16:0/3315555721 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f037fe640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f04101de0 0x7f4f040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f04102790 0x7f4f04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 -- 172.21.15.16:0/3315555721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f04102790 msgr2=0x7f4f04074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f04102790 0x7f4f04074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 -- 172.21.15.16:0/3315555721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f04101de0 msgr2=0x7f4f040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f04101de0 0x7f4f040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 -- 172.21.15.16:0/3315555721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f041a6900 con 0x7f4f04106f10 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f037fe640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f04101de0 0x7f4f040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f03fff640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04106f10 0x7f4f04074f70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00eaa0 tx=0x7f4efc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f00ff9640 1 -- 172.21.15.16:0/3315555721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00cdb0 con 0x7f4f04106f10 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f00ff9640 1 -- 172.21.15.16:0/3315555721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4efc005020 con 0x7f4f04106f10 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f041a6bf0 con 0x7f4f04106f10 2024-09-15T22:46:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f00ff9640 1 -- 172.21.15.16:0/3315555721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc0116b0 con 0x7f4f04106f10 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.939+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041a7180 con 0x7f4f04106f10 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f00ff9640 1 -- 172.21.15.16:0/3315555721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4efc005560 con 0x7f4f04106f10 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 Processor -- start 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 -- start start 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6ab60 0x56515de6af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6b480 0x56515de75800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de78130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56515ddd0280 con 0x56515de6b480 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56515dd77b10 con 0x56515de75d40 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f0c416740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56515ddcc240 con 0x56515de6ab60 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6b480 0x56515de75800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f037fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6b480 0x56515de75800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59328/0 (socket says 172.21.15.16:59328) 2024-09-15T22:46:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f037fe640 1 -- 172.21.15.16:0/4218908950 learned_addr learned my addr 172.21.15.16:0/4218908950 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f00ff9640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4eec0a3430 0x7f4eec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.942+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6ab60 0x56515de6af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de78130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f00ff9640 1 -- 172.21.15.16:0/3315555721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f4efc0d02b0 con 0x7f4f04106f10 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f03fff640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4eec0a3430 0x7f4eec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f02ffd640 1 -- 172.21.15.16:0/4218908950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 msgr2=0x56515de78130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de78130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f02ffd640 1 -- 172.21.15.16:0/4218908950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6b480 msgr2=0x56515de75800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6b480 0x56515de75800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f02ffd640 1 -- 172.21.15.16:0/4218908950 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515dd841f0 con 0x56515de6ab60 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f037fe640 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6b480 0x56515de75800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6ab60 0x56515de6af40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009230 tx=0x7f4ef8005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcd7254cae70757 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4f03fff640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4eec0a3430 0x7f4eec0a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4efc09c170 tx=0x7f4efc009270 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4ee9ffb640 1 -- 172.21.15.16:0/4218908950 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8035040 con 0x56515de6ab60 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4ee9ffb640 1 -- 172.21.15.16:0/4218908950 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef80035f0 con 0x56515de6ab60 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.943+0000 7f4ee9ffb640 1 -- 172.21.15.16:0/4218908950 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef803d6a0 con 0x56515de6ab60 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 -- 172.21.15.16:0/4218908950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6ab60 msgr2=0x56515de6af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6ab60 0x56515de6af40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009230 tx=0x7f4ef8005b50 comp rx=0 tx=0).stop 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 -- 172.21.15.16:0/4218908950 shutdown_connections 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de78130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6b480 0x56515de75800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 --2- 172.21.15.16:0/4218908950 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6ab60 0x56515de6af40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 -- 172.21.15.16:0/4218908950 >> 172.21.15.16:0/4218908950 conn(0x56515de670f0 msgr2=0x56515de674d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 -- 172.21.15.16:0/4218908950 shutdown_connections 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.944+0000 7f4f0c416740 1 -- 172.21.15.16:0/4218908950 wait complete. 2024-09-15T22:46:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 -- start start 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 Processor -- start 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de6ab60 0x56515de72c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de73160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de75d40 0x56515de736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56515de78910 con 0x56515de75d40 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56515dd8af10 con 0x56515de6ab60 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56515de78a90 con 0x56515de6b480 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f0c416740 10 client.0 _ll_register_callbacks cb 0x56515dd53a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f037fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de73160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de75d40 0x56515de736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f037fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de73160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:45050/0 (socket says 172.21.15.16:45050) 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f037fe640 1 -- 172.21.15.16:0/401626855 learned_addr learned my addr 172.21.15.16:0/401626855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.945+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de6ab60 0x56515de72c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f02ffd640 1 -- 172.21.15.16:0/401626855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 msgr2=0x56515de73160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de73160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f02ffd640 1 -- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de6ab60 msgr2=0x56515de72c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de6ab60 0x56515de72c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f02ffd640 1 -- 172.21.15.16:0/401626855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515db416a0 con 0x56515de75d40 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f037fe640 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de73160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de75d40 0x56515de736a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005ab0 tx=0x7f4ef8032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a8841a3250297e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4ee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/401626855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8051020 con 0x56515de75d40 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/401626855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f4ef80039a0 con 0x56515de75d40 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef80035f0 con 0x56515de75d40 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/401626855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ef8035040 con 0x56515de75d40 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f0c416740 10 client.17286 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56515de84510 con 0x56515de75d40 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/401626855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef80035f0 con 0x56515de75d40 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.946+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/401626855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef803a040 con 0x56515de75d40 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f0c416740 10 client.17286 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f0c416740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f0c416740 20 client.17286 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f0c416740 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515de85860 0x56515de87c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f0c416740 4 client.17286 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56515dddd7f0 con 0x56515de85860 2024-09-15T22:46:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f037fe640 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515de85860 0x56515de87c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4f037fe640 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515de85860 0x56515de87c50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81695405ce736a60 server_cookie=880a71842b1db33e in_seq=0 out_seq=0 2024-09-15T22:46:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.947+0000 7f4ee2ffd640 10 client.17286 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/401626855 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56515dddd7f0 con 0x56515de85860 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4ee2ffd640 10 client.17286 handle_command_reply: tid=0 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 client.17286 shutdown 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de75d40 msgr2=0x56515de736a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de75d40 0x56515de736a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8005ab0 tx=0x7f4ef8032e30 comp rx=0 tx=0).stop 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 shutdown_connections 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515de85860 0x56515de87c50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de75d40 0x56515de736a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de73160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4ee2ffd640 0 client.17286 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 --2- 172.21.15.16:0/401626855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de6ab60 0x56515de72c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 >> 172.21.15.16:0/401626855 conn(0x56515de670f0 msgr2=0x56515de66270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 shutdown_connections 2024-09-15T22:46:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 1 -- 172.21.15.16:0/401626855 wait complete. 2024-09-15T22:46:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.948+0000 7f4f0c416740 20 client.17286 trim_cache size 0 max 16384 2024-09-15T22:46:45.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 Processor -- start 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 -- start start 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de6fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de65a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b3b20 con 0x56515de6ab60 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b13f0 con 0x56515de75d40 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f0c416740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041af260 con 0x56515de6b480 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de6fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.966+0000 7f4f02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de6fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59338/0 (socket says 172.21.15.16:59338) 2024-09-15T22:46:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f02ffd640 1 -- 172.21.15.16:0/1381532190 learned_addr learned my addr 172.21.15.16:0/1381532190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f037fe640 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f02ffd640 1 -- 172.21.15.16:0/1381532190 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 msgr2=0x56515de703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f02ffd640 1 -- 172.21.15.16:0/1381532190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 msgr2=0x56515de65a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de65a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f02ffd640 1 -- 172.21.15.16:0/1381532190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515de62830 con 0x56515de6ab60 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de6fe80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80059d0 tx=0x7f4ef8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91772fa889d960f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4ee97fa640 1 -- 172.21.15.16:0/1381532190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8051040 con 0x56515de6ab60 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4ee97fa640 1 -- 172.21.15.16:0/1381532190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef804ad50 con 0x56515de6ab60 2024-09-15T22:46:45.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.967+0000 7f4ee97fa640 1 -- 172.21.15.16:0/1381532190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8003b50 con 0x56515de6ab60 2024-09-15T22:46:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 -- 172.21.15.16:0/1381532190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 msgr2=0x56515de6fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de6fe80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80059d0 tx=0x7f4ef8001e50 comp rx=0 tx=0).stop 2024-09-15T22:46:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 -- 172.21.15.16:0/1381532190 shutdown_connections 2024-09-15T22:46:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de65a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 --2- 172.21.15.16:0/1381532190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de6fe80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 -- 172.21.15.16:0/1381532190 >> 172.21.15.16:0/1381532190 conn(0x56515de670f0 msgr2=0x56515de74b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 -- 172.21.15.16:0/1381532190 shutdown_connections 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 -- 172.21.15.16:0/1381532190 wait complete. 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.968+0000 7f4f0c416740 1 -- start start 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 1 Processor -- start 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de8b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de88750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de88c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041af8b0 con 0x56515de6ab60 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041b0d20 con 0x56515de75d40 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4f041aff80 con 0x56515de6b480 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f0c416740 10 client.0 _ll_register_callbacks cb 0x7f4ef4009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de8b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de8b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59350/0 (socket says 172.21.15.16:59350) 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 -- 172.21.15.16:0/78488852 learned_addr learned my addr 172.21.15.16:0/78488852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f037fe640 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de88750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 -- 172.21.15.16:0/78488852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 msgr2=0x56515de88750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de88750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 -- 172.21.15.16:0/78488852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 msgr2=0x56515de88c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de88c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.969+0000 7f4f02ffd640 1 -- 172.21.15.16:0/78488852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515de894c0 con 0x56515de6ab60 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f02ffd640 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de8b3b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4ef803df30 tx=0x7f4ef8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19792fee60cc4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4ee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/78488852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef804aa80 con 0x56515de6ab60 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/78488852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7f4ef80038d0 con 0x56515de6ab60 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef804ad50 con 0x56515de6ab60 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/78488852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ef806d050 con 0x56515de6ab60 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 10 client.17292 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56515de894c0 con 0x56515de6ab60 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/78488852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef804ad50 con 0x56515de6ab60 2024-09-15T22:46:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/78488852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef803d560 con 0x56515de6ab60 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 10 client.17292 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 20 client.17292 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515def04c0 0x56515def28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 4 client.17292 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.970+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56515ddd7d00 con 0x56515def04c0 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.971+0000 7f4f037fe640 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515def04c0 0x56515def28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.971+0000 7f4ee2ffd640 10 client.17292 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.971+0000 7f4f037fe640 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515def04c0 0x56515def28b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4af5f9fd0a080d9f server_cookie=b3c8d19f3b53fc36 in_seq=0 out_seq=0 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.972+0000 7f4ee2ffd640 1 -- 172.21.15.16:0/78488852 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56515ddd7d00 con 0x56515def04c0 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.972+0000 7f4ee2ffd640 10 client.17292 handle_command_reply: tid=0 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.972+0000 7f4f0c416740 1 client.17292 shutdown 2024-09-15T22:46:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.972+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 msgr2=0x56515de8b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.972+0000 7f4f0c416740 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de8b3b0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4ef803df30 tx=0x7f4ef8003f10 comp rx=0 tx=0).stop 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.972+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 shutdown_connections 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56515def04c0 0x56515def28b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4ee2ffd640 0 client.17292 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56515de75d40 0x56515de88c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56515de6b480 0x56515de88750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 1 --2- 172.21.15.16:0/78488852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56515de6ab60 0x56515de8b3b0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 >> 172.21.15.16:0/78488852 conn(0x56515de670f0 msgr2=0x56515de74290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 shutdown_connections 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 1 -- 172.21.15.16:0/78488852 wait complete. 2024-09-15T22:46:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0c416740 20 client.17292 trim_cache size 0 max 16384 2024-09-15T22:46:45.980 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:46:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:46:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "be1ecf07-f445-4ee6-a85d-28ac7aabf9d4", 2024-09-15T22:46:45.980 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:46:45.980 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4eec0a3430 msgr2=0x7f4eec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4eec0a3430 0x7f4eec0a58f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4efc09c170 tx=0x7f4efc009270 comp rx=0 tx=0).stop 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04106f10 msgr2=0x7f4f04074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04106f10 0x7f4f04074f70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00eaa0 tx=0x7f4efc00ef60 comp rx=0 tx=0).stop 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 shutdown_connections 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f04106f10 0x7f4f04074f70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4eec0a3430 0x7f4eec0a58f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4f04102790 0x7f4f04074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 --2- 172.21.15.16:0/3315555721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4f04101de0 0x7f4f040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.973+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 >> 172.21.15.16:0/3315555721 conn(0x7f4f040fd8c0 msgr2=0x7f4f040fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.974+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 shutdown_connections 2024-09-15T22:46:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:45.974+0000 7f4f0a03d640 1 -- 172.21.15.16:0/3315555721 wait complete. 2024-09-15T22:46:46.677 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:46 smithi016 ceph-mon[31492]: pgmap v257: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:46.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:46 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:46:46.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:46 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:46.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:46 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:46:46.678 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:46 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:46.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:46:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:46 smithi130 ceph-mon[35742]: pgmap v257: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:46 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:46:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:46 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:46 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:46:46.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:46 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:46.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:46 smithi059 ceph-mon[34872]: pgmap v257: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:46.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:46 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:46:46.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:46 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:46.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:46 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:46:46.945 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:46 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:46:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.404+0000 7fb0c7643640 1 -- 172.21.15.16:0/1433997707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c010c170 msgr2=0x7fb0c010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.404+0000 7fb0c7643640 1 --2- 172.21.15.16:0/1433997707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c010c170 0x7fb0c010e610 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009ca0 tx=0x7fb0b802a950 comp rx=0 tx=0).stop 2024-09-15T22:46:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.404+0000 7fb0c7643640 1 -- 172.21.15.16:0/1433997707 shutdown_connections 2024-09-15T22:46:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.404+0000 7fb0c7643640 1 --2- 172.21.15.16:0/1433997707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c010c170 0x7fb0c010e610 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.404+0000 7fb0c7643640 1 --2- 172.21.15.16:0/1433997707 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0c0103660 0x7fb0c010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.404+0000 7fb0c7643640 1 --2- 172.21.15.16:0/1433997707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c0102d40 0x7fb0c0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.404+0000 7fb0c7643640 1 -- 172.21.15.16:0/1433997707 >> 172.21.15.16:0/1433997707 conn(0x7fb0c00fc7a0 msgr2=0x7fb0c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.405+0000 7fb0c7643640 1 -- 172.21.15.16:0/1433997707 shutdown_connections 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.405+0000 7fb0c7643640 1 -- 172.21.15.16:0/1433997707 wait complete. 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.405+0000 7fb0c7643640 1 Processor -- start 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.405+0000 7fb0c7643640 1 -- start start 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0102d40 0x7fb0c019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c0103660 0x7fb0c019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0c010c170 0x7fb0c019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0101d20 con 0x7fb0c0102d40 2024-09-15T22:46:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0101ba0 con 0x7fb0c0103660 2024-09-15T22:46:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0101ea0 con 0x7fb0c010c170 2024-09-15T22:46:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0102d40 0x7fb0c019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0102d40 0x7fb0c019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59354/0 (socket says 172.21.15.16:59354) 2024-09-15T22:46:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 -- 172.21.15.16:0/3191381684 learned_addr learned my addr 172.21.15.16:0/3191381684 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c0103660 0x7fb0c019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 -- 172.21.15.16:0/3191381684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0c010c170 msgr2=0x7fb0c019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0c010c170 0x7fb0c019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 -- 172.21.15.16:0/3191381684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c0103660 msgr2=0x7fb0c019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c0103660 0x7fb0c019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 -- 172.21.15.16:0/3191381684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c019f5a0 con 0x7fb0c0102d40 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c0103660 0x7fb0c019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c53df640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0102d40 0x7fb0c019a5f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e9f0 tx=0x7fb0bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0b27fc640 1 -- 172.21.15.16:0/3191381684 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00cd80 con 0x7fb0c0102d40 2024-09-15T22:46:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0b27fc640 1 -- 172.21.15.16:0/3191381684 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0bc010910 con 0x7fb0c0102d40 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c019f890 con 0x7fb0c0102d40 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0b27fc640 1 -- 172.21.15.16:0/3191381684 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc018990 con 0x7fb0c0102d40 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.406+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c01a7170 con 0x7fb0c0102d40 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0b27fc640 1 -- 172.21.15.16:0/3191381684 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb0bc010ac0 con 0x7fb0c0102d40 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 Processor -- start 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 -- start start 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b529c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52a7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586b52022e0 con 0x5586b529c4a0 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5586b51a9ce0 con 0x5586b52a7680 2024-09-15T22:46:47.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.409+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5586b51fe2a0 con 0x5586b529cdc0 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0b27fc640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0ac0a33b0 0x7fb0ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c5be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b529c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c5be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b529c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59368/0 (socket says 172.21.15.16:59368) 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c5be0640 1 -- 172.21.15.16:0/623658678 learned_addr learned my addr 172.21.15.16:0/623658678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c53df640 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c5be0640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0ac0a33b0 0x7fb0ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0b27fc640 1 -- 172.21.15.16:0/3191381684 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7fb0bc0d04a0 con 0x7fb0c0102d40 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c4bde640 1 -- 172.21.15.16:0/623658678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 msgr2=0x5586b52a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c4bde640 1 -- 172.21.15.16:0/623658678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 msgr2=0x5586b529c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b529c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c4bde640 1 -- 172.21.15.16:0/623658678 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b518e440 con 0x5586b52a7680 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c5be0640 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b529c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c5be0640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0ac0a33b0 0x7fb0ac0a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8004af0 tx=0x7fb0b80095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.410+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52a9a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00fe400 tx=0x7fb0b400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306d013cc725479b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0ab7fe640 1 -- 172.21.15.16:0/623658678 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400be60 con 0x5586b52a7680 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0ab7fe640 1 -- 172.21.15.16:0/623658678 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b4011920 con 0x5586b52a7680 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0ab7fe640 1 -- 172.21.15.16:0/623658678 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b401b9e0 con 0x5586b52a7680 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/623658678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 msgr2=0x5586b52a9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52a9a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00fe400 tx=0x7fb0b400f730 comp rx=0 tx=0).stop 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/623658678 shutdown_connections 2024-09-15T22:46:47.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52a9a70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/623658678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b529c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/623658678 >> 172.21.15.16:0/623658678 conn(0x5586b5298a30 msgr2=0x5586b5298e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/623658678 shutdown_connections 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.411+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/623658678 wait complete. 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 -- start start 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 Processor -- start 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52a6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b52a7680 0x5586b52a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5586b52aa250 con 0x5586b529c4a0 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5586b51bd050 con 0x5586b529cdc0 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5586b52aa3d0 con 0x5586b52a7680 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c9a1c740 10 client.0 _ll_register_callbacks cb 0x5586b51fd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:47.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c53df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52a6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c53df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52a6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59376/0 (socket says 172.21.15.16:59376) 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c53df640 1 -- 172.21.15.16:0/2234845923 learned_addr learned my addr 172.21.15.16:0/2234845923 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.412+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.413+0000 7fb0c4bde640 1 -- 172.21.15.16:0/2234845923 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b52a7680 msgr2=0x5586b52a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.413+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b52a7680 0x5586b52a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.413+0000 7fb0c4bde640 1 -- 172.21.15.16:0/2234845923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 msgr2=0x5586b52a6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.413+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52a6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.413+0000 7fb0c4bde640 1 -- 172.21.15.16:0/2234845923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b4f733c0 con 0x5586b529cdc0 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a4270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400fc30 tx=0x7fb0b400bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72884d225ae0b133 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:47.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/2234845923 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b402a060 con 0x5586b529cdc0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b4011920 con 0x5586b529cdc0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/2234845923 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7fb0b4011650 con 0x5586b529cdc0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/2234845923 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0b4031050 con 0x5586b529cdc0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0c9a1c740 10 client.26576 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586b52b5c00 con 0x5586b529cdc0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.414+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/2234845923 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b4011920 con 0x5586b529cdc0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/2234845923 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb0b401b7a0 con 0x5586b529cdc0 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c9a1c740 10 client.26576 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c9a1c740 10 client.26576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c9a1c740 20 client.26576 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b52b7010 0x5586b52b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c9a1c740 4 client.26576 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:47.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586b520f850 con 0x5586b52b7010 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c53df640 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b52b7010 0x5586b52b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0c53df640 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b52b7010 0x5586b52b9400 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e53f9dcbd5adab server_cookie=45cca6b3667716ef in_seq=0 out_seq=0 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.415+0000 7fb0a8ff9640 10 client.26576 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/2234845923 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586b520f850 con 0x5586b52b7010 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0a8ff9640 10 client.26576 handle_command_reply: tid=0 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 client.26576 shutdown 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 msgr2=0x5586b52a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a4270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400fc30 tx=0x7fb0b400bc60 comp rx=0 tx=0).stop 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 shutdown_connections 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b52b7010 0x5586b52b9400 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b52a7680 0x5586b52a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a4270 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2234845923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52a6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 >> 172.21.15.16:0/2234845923 conn(0x5586b5298a30 msgr2=0x5586b529e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.416+0000 7fb0a8ff9640 0 client.26576 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.417+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 shutdown_connections 2024-09-15T22:46:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.417+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2234845923 wait complete. 2024-09-15T22:46:47.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.417+0000 7fb0c9a1c740 20 client.26576 trim_cache size 0 max 16384 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 Processor -- start 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 -- start start 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529c4a0 0x5586b52b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b52a7680 0x5586b52bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01b3b90 con 0x5586b52a7680 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01b1460 con 0x5586b529cdc0 2024-09-15T22:46:47.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01af2d0 con 0x5586b529c4a0 2024-09-15T22:46:47.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.434+0000 7fb0c5be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b52a7680 0x5586b52bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b52a7680 0x5586b52bea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59392/0 (socket says 172.21.15.16:59392) 2024-09-15T22:46:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 -- 172.21.15.16:0/2466638797 learned_addr learned my addr 172.21.15.16:0/2466638797 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c53df640 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529c4a0 0x5586b52b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 -- 172.21.15.16:0/2466638797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529c4a0 msgr2=0x5586b52b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529c4a0 0x5586b52b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 -- 172.21.15.16:0/2466638797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 msgr2=0x5586b52a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 -- 172.21.15.16:0/2466638797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b5298860 con 0x5586b52a7680 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c53df640 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529c4a0 0x5586b52b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0c5be0640 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b52a7680 0x5586b52bea00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802fc90 tx=0x7fb0b8004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6968bf131248553 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0aaffd640 1 -- 172.21.15.16:0/2466638797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b804e030 con 0x5586b52a7680 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0aaffd640 1 -- 172.21.15.16:0/2466638797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b8003e50 con 0x5586b52a7680 2024-09-15T22:46:47.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.435+0000 7fb0aaffd640 1 -- 172.21.15.16:0/2466638797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8034070 con 0x5586b52a7680 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2466638797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b52a7680 msgr2=0x5586b52bea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b52a7680 0x5586b52bea00 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802fc90 tx=0x7fb0b8004a80 comp rx=0 tx=0).stop 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2466638797 shutdown_connections 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b52a7680 0x5586b52bea00 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b529cdc0 0x5586b52a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/2466638797 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529c4a0 0x5586b52b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2466638797 >> 172.21.15.16:0/2466638797 conn(0x5586b5298a30 msgr2=0x5586b52a5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2466638797 shutdown_connections 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/2466638797 wait complete. 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.436+0000 7fb0c9a1c740 1 -- start start 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 1 Processor -- start 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52aad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01af920 con 0x5586b529c4a0 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01b0d90 con 0x5586b52a7680 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01afff0 con 0x5586b529cdc0 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c9a1c740 10 client.0 _ll_register_callbacks cb 0x7fb0b4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59406/0 (socket says 172.21.15.16:59406) 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 -- 172.21.15.16:0/3605921812 learned_addr learned my addr 172.21.15.16:0/3605921812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c5be0640 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 -- 172.21.15.16:0/3605921812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 msgr2=0x5586b52afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 -- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 msgr2=0x5586b52aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:47.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.437+0000 7fb0c53df640 1 -- 172.21.15.16:0/3605921812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b52ab5a0 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c53df640 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52af940 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc098e40 tx=0x7fb0bc014cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991f8bf667c0015d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/3605921812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc09c470 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/3605921812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9467+0+0 (secure 0 0 0) 0x7fb0bc098ae0 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b8003e50 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/3605921812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0bc0b5050 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 10 client.17310 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5586b5321620 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/3605921812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b8003e50 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/3605921812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb0bc0b51f0 con 0x5586b529c4a0 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 10 client.17310 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:47.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 20 client.17310 populate_metadata read hostname 'smithi016' 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b5321cf0 0x5586b53240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 4 client.17310 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.438+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586b5209d60 con 0x5586b5321cf0 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b5321cf0 0x5586b53240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c4bde640 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b5321cf0 0x5586b53240e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01fb9e8c5510c44 server_cookie=2a9b914b6c525154 in_seq=0 out_seq=0 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0a8ff9640 10 client.17310 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0a8ff9640 1 -- 172.21.15.16:0/3605921812 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5586b5209d60 con 0x5586b5321cf0 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0a8ff9640 10 client.17310 handle_command_reply: tid=0 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 client.17310 shutdown 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 msgr2=0x5586b52af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52af940 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc098e40 tx=0x7fb0bc014cb0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 shutdown_connections 2024-09-15T22:46:47.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5586b5321cf0 0x5586b53240e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5586b52a7680 0x5586b52aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5586b529cdc0 0x5586b52afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0a8ff9640 0 client.17310 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 --2- 172.21.15.16:0/3605921812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5586b529c4a0 0x5586b52af940 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.439+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 >> 172.21.15.16:0/3605921812 conn(0x5586b5298a30 msgr2=0x5586b52a62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 shutdown_connections 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c9a1c740 1 -- 172.21.15.16:0/3605921812 wait complete. 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c9a1c740 20 client.17310 trim_cache size 0 max 16384 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:46:47.447 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout: "be1ecf07-f445-4ee6-a85d-28ac7aabf9d4": { 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "be1ecf07-f445-4ee6-a85d-28ac7aabf9d4", 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:46:47.448 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0ac0a33b0 msgr2=0x7fb0ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0ac0a33b0 0x7fb0ac0a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8004af0 tx=0x7fb0b80095c0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0102d40 msgr2=0x7fb0c019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0102d40 0x7fb0c019a5f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00e9f0 tx=0x7fb0bc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 shutdown_connections 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb0ac0a33b0 0x7fb0ac0a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb0c010c170 0x7fb0c019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0c0103660 0x7fb0c019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 --2- 172.21.15.16:0/3191381684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0c0102d40 0x7fb0c019a5f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 >> 172.21.15.16:0/3191381684 conn(0x7fb0c00fc7a0 msgr2=0x7fb0c00fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:47.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 shutdown_connections 2024-09-15T22:46:47.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:46:47.440+0000 7fb0c7643640 1 -- 172.21.15.16:0/3191381684 wait complete. 2024-09-15T22:46:48.153 INFO:tasks.cephfs.filesystem:scrub status for tag:be1ecf07-f445-4ee6-a85d-28ac7aabf9d4 - {'path': '/', 'tag': 'be1ecf07-f445-4ee6-a85d-28ac7aabf9d4', 'options': 'recursive,force'} 2024-09-15T22:46:48.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:48 smithi130 ceph-mon[35742]: pgmap v258: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:48.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:48 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:46:48.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:48 smithi016 ceph-mon[31492]: pgmap v258: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:48 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:46:48.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:48 smithi059 ceph-mon[34872]: pgmap v258: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:48.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:48 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:46:50.779 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:50 smithi130 ceph-mon[35742]: pgmap v259: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:50.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:50 smithi016 ceph-mon[31492]: pgmap v259: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:50.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:50 smithi059 ceph-mon[34872]: pgmap v259: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:52.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:52 smithi130 ceph-mon[35742]: pgmap v260: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:52.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:52 smithi016 ceph-mon[31492]: pgmap v260: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:52.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:52 smithi059 ceph-mon[34872]: pgmap v260: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:54.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:54 smithi130 ceph-mon[35742]: pgmap v261: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:54.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:54 smithi016 ceph-mon[31492]: pgmap v261: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:54.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:54 smithi059 ceph-mon[34872]: pgmap v261: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:56.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:56 smithi130 ceph-mon[35742]: pgmap v262: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:56.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:56 smithi016 ceph-mon[31492]: pgmap v262: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:56.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:56 smithi059 ceph-mon[34872]: pgmap v262: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:46:58.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:46:58 smithi130 ceph-mon[35742]: pgmap v263: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:58.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:46:58 smithi016 ceph-mon[31492]: pgmap v263: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:46:58.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:46:58 smithi059 ceph-mon[34872]: pgmap v263: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:00.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:00 smithi130 ceph-mon[35742]: pgmap v264: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:47:00.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:00 smithi016 ceph-mon[31492]: pgmap v264: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:47:00.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:00 smithi059 ceph-mon[34872]: pgmap v264: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:47:02.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:02 smithi130 ceph-mon[35742]: pgmap v265: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:47:02.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:02 smithi016 ceph-mon[31492]: pgmap v265: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:47:02.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:02 smithi059 ceph-mon[34872]: pgmap v265: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:47:04.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:04 smithi016 ceph-mon[31492]: pgmap v266: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:47:04.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:04 smithi059 ceph-mon[34872]: pgmap v266: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:47:05.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:04 smithi130 ceph-mon[35742]: pgmap v266: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:47:06.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:06 smithi016 ceph-mon[31492]: pgmap v267: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:47:06.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:06 smithi059 ceph-mon[34872]: pgmap v267: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:47:07.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:06 smithi130 ceph-mon[35742]: pgmap v267: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:47:08.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:08 smithi016 ceph-mon[31492]: pgmap v268: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:47:08.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:08 smithi059 ceph-mon[34872]: pgmap v268: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:47:09.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:08 smithi130 ceph-mon[35742]: pgmap v268: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:47:09.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:09 smithi016 ceph-mon[31492]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:47:09.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:09 smithi059 ceph-mon[34872]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:47:10.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:09 smithi130 ceph-mon[35742]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:47:10.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:10 smithi016 ceph-mon[31492]: pgmap v270: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:47:10.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:10 smithi059 ceph-mon[34872]: pgmap v270: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:47:11.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:10 smithi130 ceph-mon[35742]: pgmap v270: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:47:12.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:12 smithi016 ceph-mon[31492]: pgmap v271: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:12.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:12 smithi059 ceph-mon[34872]: pgmap v271: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:13.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:12 smithi130 ceph-mon[35742]: pgmap v271: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:14.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:14 smithi016 ceph-mon[31492]: pgmap v272: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:14.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:14 smithi016 ceph-mon[31492]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:47:14.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:14 smithi059 ceph-mon[34872]: pgmap v272: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:14 smithi059 ceph-mon[34872]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:47:15.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:14 smithi130 ceph-mon[35742]: pgmap v272: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:14 smithi130 ceph-mon[35742]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:47:16.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:16 smithi016 ceph-mon[31492]: pgmap v274: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:47:16.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:16 smithi059 ceph-mon[34872]: pgmap v274: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:47:17.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:16 smithi130 ceph-mon[35742]: pgmap v274: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:47:18.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:47:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:18.810 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:18 smithi016 ceph-mon[31492]: pgmap v275: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 237 B/s wr, 7 op/s 2024-09-15T22:47:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 -- 172.21.15.16:0/3561755128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c102d40 msgr2=0x7ff26c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 --2- 172.21.15.16:0/3561755128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c102d40 0x7ff26c103120 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009900 tx=0x7ff25c02a480 comp rx=0 tx=0).stop 2024-09-15T22:47:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 -- 172.21.15.16:0/3561755128 shutdown_connections 2024-09-15T22:47:18.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 --2- 172.21.15.16:0/3561755128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff26c10c170 0x7ff26c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 --2- 172.21.15.16:0/3561755128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff26c103660 0x7ff26c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 --2- 172.21.15.16:0/3561755128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c102d40 0x7ff26c103120 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 -- 172.21.15.16:0/3561755128 >> 172.21.15.16:0/3561755128 conn(0x7ff26c0fc7a0 msgr2=0x7ff26c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 -- 172.21.15.16:0/3561755128 shutdown_connections 2024-09-15T22:47:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.881+0000 7ff271f0a640 1 -- 172.21.15.16:0/3561755128 wait complete. 2024-09-15T22:47:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 Processor -- start 2024-09-15T22:47:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 -- start start 2024-09-15T22:47:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff26c102d40 0x7ff26c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff26c103660 0x7ff26c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c10c170 0x7ff26c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c111fb0 con 0x7ff26c10c170 2024-09-15T22:47:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c111e30 con 0x7ff26c102d40 2024-09-15T22:47:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff271f0a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c112130 con 0x7ff26c103660 2024-09-15T22:47:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff26bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c10c170 0x7ff26c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff26bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c10c170 0x7ff26c19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52138/0 (socket says 172.21.15.16:52138) 2024-09-15T22:47:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff26bfff640 1 -- 172.21.15.16:0/1156677386 learned_addr learned my addr 172.21.15.16:0/1156677386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff26affd640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff26c103660 0x7ff26c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.882+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff26c102d40 0x7ff26c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff26bfff640 1 -- 172.21.15.16:0/1156677386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff26c103660 msgr2=0x7ff26c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff26bfff640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff26c103660 0x7ff26c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff26bfff640 1 -- 172.21.15.16:0/1156677386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff26c102d40 msgr2=0x7ff26c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff26bfff640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff26c102d40 0x7ff26c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff26bfff640 1 -- 172.21.15.16:0/1156677386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c19d460 con 0x7ff26c10c170 2024-09-15T22:47:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff26bfff640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c10c170 0x7ff26c19cd80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff26400eaa0 tx=0x7ff26400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff268ff9640 1 -- 172.21.15.16:0/1156677386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26400cdb0 con 0x7ff26c10c170 2024-09-15T22:47:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c19d750 con 0x7ff26c10c170 2024-09-15T22:47:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff268ff9640 1 -- 172.21.15.16:0/1156677386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff264005020 con 0x7ff26c10c170 2024-09-15T22:47:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff268ff9640 1 -- 172.21.15.16:0/1156677386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640116b0 con 0x7ff26c10c170 2024-09-15T22:47:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.883+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c1a5030 con 0x7ff26c10c170 2024-09-15T22:47:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff268ff9640 1 -- 172.21.15.16:0/1156677386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff264005560 con 0x7ff26c10c170 2024-09-15T22:47:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 Processor -- start 2024-09-15T22:47:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 -- start start 2024-09-15T22:47:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e67d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e688140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e68aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56215e5e32e0 con 0x56215e67ddc0 2024-09-15T22:47:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56215e58ace0 con 0x56215e67d4a0 2024-09-15T22:47:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff2742e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56215e5df2a0 con 0x56215e688680 2024-09-15T22:47:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e688140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e688140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52146/0 (socket says 172.21.15.16:52146) 2024-09-15T22:47:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff26b7fe640 1 -- 172.21.15.16:0/2131120624 learned_addr learned my addr 172.21.15.16:0/2131120624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.886+0000 7ff268ff9640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff2540a33b0 0x7ff2540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26affd640 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e67d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26affd640 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e68aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26b7fe640 1 -- 172.21.15.16:0/2131120624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 msgr2=0x56215e68aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26bfff640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff2540a33b0 0x7ff2540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e68aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26b7fe640 1 -- 172.21.15.16:0/2131120624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 msgr2=0x56215e67d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e67d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26b7fe640 1 -- 172.21.15.16:0/2131120624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215e56f440 con 0x56215e67ddc0 2024-09-15T22:47:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26affd640 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e67d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff268ff9640 1 -- 172.21.15.16:0/1156677386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7ff2640d02e0 con 0x7ff26c10c170 2024-09-15T22:47:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e688140 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009900 tx=0x7ff25c031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1d9fcf67592b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff251ffb640 1 -- 172.21.15.16:0/2131120624 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c040070 con 0x56215e67ddc0 2024-09-15T22:47:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff26bfff640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff2540a33b0 0x7ff2540a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff26409bf60 tx=0x7ff2640bc040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff251ffb640 1 -- 172.21.15.16:0/2131120624 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff25c005020 con 0x56215e67ddc0 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.887+0000 7ff251ffb640 1 -- 172.21.15.16:0/2131120624 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c035040 con 0x56215e67ddc0 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 -- 172.21.15.16:0/2131120624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 msgr2=0x56215e688140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e688140 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009900 tx=0x7ff25c031630 comp rx=0 tx=0).stop 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 -- 172.21.15.16:0/2131120624 shutdown_connections 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e68aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e688140 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 --2- 172.21.15.16:0/2131120624 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e67d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 -- 172.21.15.16:0/2131120624 >> 172.21.15.16:0/2131120624 conn(0x56215e679a30 msgr2=0x56215e679e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 -- 172.21.15.16:0/2131120624 shutdown_connections 2024-09-15T22:47:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 -- 172.21.15.16:0/2131120624 wait complete. 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.888+0000 7ff2742e3740 1 -- start start 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 1 Processor -- start 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e687f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e685220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e688680 0x56215e685760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56215e68b250 con 0x56215e688680 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56215e59e050 con 0x56215e67d4a0 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56215e68b3d0 con 0x56215e67ddc0 2024-09-15T22:47:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff2742e3740 10 client.0 _ll_register_callbacks cb 0x56215e5de160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e688680 0x56215e685760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e688680 0x56215e685760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52162/0 (socket says 172.21.15.16:52162) 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e685220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 -- 172.21.15.16:0/478437059 learned_addr learned my addr 172.21.15.16:0/478437059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e687f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 -- 172.21.15.16:0/478437059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 msgr2=0x56215e685220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e685220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 -- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 msgr2=0x56215e687f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e687f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 -- 172.21.15.16:0/478437059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215e3543c0 con 0x56215e688680 2024-09-15T22:47:18.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e685220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.889+0000 7ff26affd640 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e688680 0x56215e685760 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff260009e80 tx=0x7ff2600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8321e386bf3bea6e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff24affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff24affd640 1 -- 172.21.15.16:0/478437059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260019110 con 0x56215e688680 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff24affd640 1 -- 172.21.15.16:0/478437059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7ff260014b20 con 0x56215e688680 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff25c005020 con 0x56215e688680 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff24affd640 1 -- 172.21.15.16:0/478437059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff260028050 con 0x56215e688680 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 10 client.17322 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56215e696c20 con 0x56215e688680 2024-09-15T22:47:18.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff24affd640 1 -- 172.21.15.16:0/478437059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff25c005020 con 0x56215e688680 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff24affd640 1 -- 172.21.15.16:0/478437059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff260017bc0 con 0x56215e688680 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 10 client.17322 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 20 client.17322 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e6980e0 0x56215e69a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 4 client.17322 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.890+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56215e5f0850 con 0x56215e6980e0 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.891+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e6980e0 0x56215e69a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.891+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e6980e0 0x56215e69a4f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bfef9a9917894b3 server_cookie=915ee0c09271d9cd in_seq=0 out_seq=0 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.891+0000 7ff24affd640 10 client.17322 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff24affd640 1 -- 172.21.15.16:0/478437059 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56215e5f0850 con 0x56215e6980e0 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff24affd640 10 client.17322 handle_command_reply: tid=0 2024-09-15T22:47:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 client.17322 shutdown 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e688680 msgr2=0x56215e685760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e688680 0x56215e685760 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff260009e80 tx=0x7ff2600046c0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 shutdown_connections 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e6980e0 0x56215e69a4f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e688680 0x56215e685760 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e685220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 --2- 172.21.15.16:0/478437059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e687f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 >> 172.21.15.16:0/478437059 conn(0x56215e679a30 msgr2=0x56215e678b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff24affd640 0 client.17322 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 shutdown_connections 2024-09-15T22:47:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 1 -- 172.21.15.16:0/478437059 wait complete. 2024-09-15T22:47:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.892+0000 7ff2742e3740 20 client.17322 trim_cache size 0 max 16384 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.909+0000 7ff2742e3740 1 Processor -- start 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.909+0000 7ff2742e3740 1 -- start start 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67d4a0 0x56215e6944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e69a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e688680 0x56215e6806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2742e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1b1980 con 0x56215e67d4a0 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2742e3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1af250 con 0x56215e688680 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2742e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1ad0c0 con 0x56215e67ddc0 2024-09-15T22:47:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67d4a0 0x56215e6944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67d4a0 0x56215e6944b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52170/0 (socket says 172.21.15.16:52170) 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 -- 172.21.15.16:0/1430794863 learned_addr learned my addr 172.21.15.16:0/1430794863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e69a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 -- 172.21.15.16:0/1430794863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 msgr2=0x56215e69a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e69a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 -- 172.21.15.16:0/1430794863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e688680 msgr2=0x56215e6806d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e688680 0x56215e6806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 -- 172.21.15.16:0/1430794863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215e6841d0 con 0x56215e67d4a0 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff26affd640 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67d4a0 0x56215e6944b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff260009b00 tx=0x7ff260006ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182fb74ee898c44c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2517fa640 1 -- 172.21.15.16:0/1430794863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260019110 con 0x56215e67d4a0 2024-09-15T22:47:18.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2517fa640 1 -- 172.21.15.16:0/1430794863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26001f070 con 0x56215e67d4a0 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.910+0000 7ff2517fa640 1 -- 172.21.15.16:0/1430794863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600059c0 con 0x56215e67d4a0 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 -- 172.21.15.16:0/1430794863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67d4a0 msgr2=0x56215e6944b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67d4a0 0x56215e6944b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7ff260009b00 tx=0x7ff260006ec0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 -- 172.21.15.16:0/1430794863 shutdown_connections 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e688680 0x56215e6806d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e67ddc0 0x56215e69a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1430794863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67d4a0 0x56215e6944b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 -- 172.21.15.16:0/1430794863 >> 172.21.15.16:0/1430794863 conn(0x56215e679a30 msgr2=0x56215e67ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 -- 172.21.15.16:0/1430794863 shutdown_connections 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 -- 172.21.15.16:0/1430794863 wait complete. 2024-09-15T22:47:18.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.911+0000 7ff2742e3740 1 -- start start 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 1 Processor -- start 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e68bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e68c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e6ff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1ad710 con 0x56215e67ddc0 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1aeb80 con 0x56215e67d4a0 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1adde0 con 0x56215e688680 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff2742e3740 10 client.0 _ll_register_callbacks cb 0x7ff2600097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e68c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e68bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e68c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52184/0 (socket says 172.21.15.16:52184) 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 -- 172.21.15.16:0/1972423988 learned_addr learned my addr 172.21.15.16:0/1972423988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26affd640 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e6ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 -- 172.21.15.16:0/1972423988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 msgr2=0x56215e6ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e6ff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 -- 172.21.15.16:0/1972423988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 msgr2=0x56215e68bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e68bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.912+0000 7ff26b7fe640 1 -- 172.21.15.16:0/1972423988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215e6882e0 con 0x56215e67ddc0 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff26b7fe640 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e68c2e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0096c0 tx=0x7ff25c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f609efb8348e8358 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff24affd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff24affd640 1 -- 172.21.15.16:0/1972423988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c040070 con 0x56215e67ddc0 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff24affd640 1 -- 172.21.15.16:0/1972423988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7ff25c0035f0 con 0x56215e67ddc0 2024-09-15T22:47:18.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff26001f070 con 0x56215e67ddc0 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff24affd640 1 -- 172.21.15.16:0/1972423988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff25c05f050 con 0x56215e67ddc0 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 10 client.17328 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56215e7069c0 con 0x56215e67ddc0 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff24affd640 1 -- 172.21.15.16:0/1972423988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26001f070 con 0x56215e67ddc0 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff24affd640 1 -- 172.21.15.16:0/1972423988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff25c04f780 con 0x56215e67ddc0 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 10 client.17328 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 10 client.17328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 20 client.17328 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:18.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e706dc0 0x56215e7091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 4 client.17328 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.913+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56215e5ead60 con 0x56215e706dc0 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.914+0000 7ff26affd640 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e706dc0 0x56215e7091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.914+0000 7ff26affd640 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e706dc0 0x56215e7091b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0914f58a5d22ae server_cookie=5263f8a879257551 in_seq=0 out_seq=0 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.914+0000 7ff24affd640 10 client.17328 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.914+0000 7ff24affd640 1 -- 172.21.15.16:0/1972423988 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56215e5ead60 con 0x56215e706dc0 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.914+0000 7ff24affd640 10 client.17328 handle_command_reply: tid=0 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.914+0000 7ff2742e3740 1 client.17328 shutdown 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 msgr2=0x56215e68c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e68c2e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0096c0 tx=0x7ff25c005850 comp rx=0 tx=0).stop 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 shutdown_connections 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56215e706dc0 0x56215e7091b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56215e688680 0x56215e6ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56215e67ddc0 0x56215e68c2e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff24affd640 0 client.17328 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 --2- 172.21.15.16:0/1972423988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56215e67d4a0 0x56215e68bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 >> 172.21.15.16:0/1972423988 conn(0x56215e679a30 msgr2=0x56215e6807e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 shutdown_connections 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 1 -- 172.21.15.16:0/1972423988 wait complete. 2024-09-15T22:47:18.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff2742e3740 20 client.17328 trim_cache size 0 max 16384 2024-09-15T22:47:18.930 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:47:18.930 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:47:18.930 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:47:18.930 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:47:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff2540a33b0 msgr2=0x7ff2540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff271f0a640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff2540a33b0 0x7ff2540a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff26409bf60 tx=0x7ff2640bc040 comp rx=0 tx=0).stop 2024-09-15T22:47:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c10c170 msgr2=0x7ff26c19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff271f0a640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c10c170 0x7ff26c19cd80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff26400eaa0 tx=0x7ff26400ef60 comp rx=0 tx=0).stop 2024-09-15T22:47:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.915+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 shutdown_connections 2024-09-15T22:47:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.916+0000 7ff271f0a640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c10c170 0x7ff26c19cd80 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.916+0000 7ff271f0a640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff2540a33b0 0x7ff2540a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.916+0000 7ff271f0a640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff26c103660 0x7ff26c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.916+0000 7ff271f0a640 1 --2- 172.21.15.16:0/1156677386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff26c102d40 0x7ff26c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.916+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 >> 172.21.15.16:0/1156677386 conn(0x7ff26c0fc7a0 msgr2=0x7ff26c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.916+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 shutdown_connections 2024-09-15T22:47:18.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:18.916+0000 7ff271f0a640 1 -- 172.21.15.16:0/1156677386 wait complete. 2024-09-15T22:47:18.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:18 smithi059 ceph-mon[34872]: pgmap v275: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 237 B/s wr, 7 op/s 2024-09-15T22:47:19.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:18 smithi130 ceph-mon[35742]: pgmap v275: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 237 B/s wr, 7 op/s 2024-09-15T22:47:19.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:47:19.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:47:20.051 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 -- 172.21.15.16:0/3122308899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101de0 msgr2=0x7f6b601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 --2- 172.21.15.16:0/3122308899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101de0 0x7f6b601021c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009900 tx=0x7f6b5002a410 comp rx=0 tx=0).stop 2024-09-15T22:47:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 -- 172.21.15.16:0/3122308899 shutdown_connections 2024-09-15T22:47:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 --2- 172.21.15.16:0/3122308899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b60106f10 0x7f6b6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 --2- 172.21.15.16:0/3122308899 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b60102790 0x7f6b601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 --2- 172.21.15.16:0/3122308899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60101de0 0x7f6b601021c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 -- 172.21.15.16:0/3122308899 >> 172.21.15.16:0/3122308899 conn(0x7f6b600fd8c0 msgr2=0x7f6b600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 -- 172.21.15.16:0/3122308899 shutdown_connections 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 -- 172.21.15.16:0/3122308899 wait complete. 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.514+0000 7f6b654da640 1 Processor -- start 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b654da640 1 -- start start 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b654da640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b654da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b654da640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b654da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60111f70 con 0x7f6b60102790 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b654da640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6b60111df0 con 0x7f6b60106f10 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b654da640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f6b601120f0 con 0x7f6b60101de0 2024-09-15T22:47:20.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52194/0 (socket says 172.21.15.16:52194) 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 -- 172.21.15.16:0/713130374 learned_addr learned my addr 172.21.15.16:0/713130374 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5effd640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5f7fe640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 -- 172.21.15.16:0/713130374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b60101de0 msgr2=0x7f6b600744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 -- 172.21.15.16:0/713130374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b60106f10 msgr2=0x7f6b60074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.515+0000 7f6b5e7fc640 1 -- 172.21.15.16:0/713130374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b601a68f0 con 0x7f6b60102790 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.516+0000 7f6b5e7fc640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400d770 tx=0x7f6b5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:20.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.516+0000 7f6b47fff640 1 -- 172.21.15.16:0/713130374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54017070 con 0x7f6b60102790 2024-09-15T22:47:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.516+0000 7f6b47fff640 1 -- 172.21.15.16:0/713130374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b54005500 con 0x7f6b60102790 2024-09-15T22:47:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.516+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b601a6be0 con 0x7f6b60102790 2024-09-15T22:47:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.516+0000 7f6b47fff640 1 -- 172.21.15.16:0/713130374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004e00 con 0x7f6b60102790 2024-09-15T22:47:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.516+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601a70d0 con 0x7f6b60102790 2024-09-15T22:47:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.517+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b601021c0 con 0x7f6b60102790 2024-09-15T22:47:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.518+0000 7f6b47fff640 1 -- 172.21.15.16:0/713130374 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6b54004fa0 con 0x7f6b60102790 2024-09-15T22:47:20.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.519+0000 7f6b47fff640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6b480a33b0 0x7f6b480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.519+0000 7f6b47fff640 1 -- 172.21.15.16:0/713130374 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f6b540cfb00 con 0x7f6b60102790 2024-09-15T22:47:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.519+0000 7f6b5effd640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6b480a33b0 0x7f6b480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.519+0000 7f6b5effd640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6b480a33b0 0x7f6b480a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009230 tx=0x7f6b500058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:20.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.523+0000 7f6b47fff640 1 -- 172.21.15.16:0/713130374 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b5409c130 con 0x7f6b60102790 2024-09-15T22:47:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.739+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b6010a0f0 con 0x7f6b60102790 2024-09-15T22:47:20.748 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:20 smithi016 ceph-mon[31492]: pgmap v276: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.740+0000 7f6b47fff640 1 -- 172.21.15.16:0/713130374 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f6b5409ba30 con 0x7f6b60102790 2024-09-15T22:47:20.750 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:47:20.750 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:47:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:47:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6b480a33b0 msgr2=0x7f6b480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6b480a33b0 0x7f6b480a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009230 tx=0x7f6b500058d0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 msgr2=0x7f6b60074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400d770 tx=0x7f6b5400dc30 comp rx=0 tx=0).stop 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 shutdown_connections 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b60106f10 0x7f6b60074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b60102790 0x7f6b60074a30 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f6b480a33b0 0x7f6b480a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 --2- 172.21.15.16:0/713130374 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f6b60101de0 0x7f6b600744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.742+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 >> 172.21.15.16:0/713130374 conn(0x7f6b600fd8c0 msgr2=0x7f6b600fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.743+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 shutdown_connections 2024-09-15T22:47:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:20.743+0000 7f6b654da640 1 -- 172.21.15.16:0/713130374 wait complete. 2024-09-15T22:47:20.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:20 smithi059 ceph-mon[34872]: pgmap v276: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:21.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:20 smithi130 ceph-mon[35742]: pgmap v276: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:47:21.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:47:21.729 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:21.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:21 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/713130374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:21.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:21 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/713130374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:22.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:21 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/713130374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.322+0000 7fafd0b68640 1 -- 172.21.15.16:0/545270482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc101de0 msgr2=0x7fafcc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.322+0000 7fafd0b68640 1 --2- 172.21.15.16:0/545270482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc101de0 0x7fafcc1021c0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009900 tx=0x7fafbc02a410 comp rx=0 tx=0).stop 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 -- 172.21.15.16:0/545270482 shutdown_connections 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 --2- 172.21.15.16:0/545270482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc106f10 0x7fafcc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 --2- 172.21.15.16:0/545270482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafcc102790 0x7fafcc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 --2- 172.21.15.16:0/545270482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc101de0 0x7fafcc1021c0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 -- 172.21.15.16:0/545270482 >> 172.21.15.16:0/545270482 conn(0x7fafcc0fd8c0 msgr2=0x7fafcc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 -- 172.21.15.16:0/545270482 shutdown_connections 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 -- 172.21.15.16:0/545270482 wait complete. 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 Processor -- start 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.323+0000 7fafd0b68640 1 -- start start 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafcc101de0 0x7fafcc19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc102790 0x7fafcc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc106f10 0x7fafcc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc111f70 con 0x7fafcc102790 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc111df0 con 0x7fafcc106f10 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1120f0 con 0x7fafcc101de0 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc102790 0x7fafcc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc102790 0x7fafcc19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37362/0 (socket says 172.21.15.16:37362) 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 -- 172.21.15.16:0/3653769775 learned_addr learned my addr 172.21.15.16:0/3653769775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafca59c640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafcc101de0 0x7fafcc19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafcad9d640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc106f10 0x7fafcc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 -- 172.21.15.16:0/3653769775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafcc101de0 msgr2=0x7fafcc19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafcc101de0 0x7fafcc19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 -- 172.21.15.16:0/3653769775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc106f10 msgr2=0x7fafcc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc106f10 0x7fafcc19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 -- 172.21.15.16:0/3653769775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc19f630 con 0x7fafcc102790 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc102790 0x7fafcc19abc0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fafc000d950 tx=0x7fafc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafbb7fe640 1 -- 172.21.15.16:0/3653769775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0017030 con 0x7fafcc102790 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc19f920 con 0x7fafcc102790 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafbb7fe640 1 -- 172.21.15.16:0/3653769775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc0004020 con 0x7fafcc102790 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafbb7fe640 1 -- 172.21.15.16:0/3653769775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00047a0 con 0x7fafcc102790 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.324+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a7200 con 0x7fafcc102790 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafbb7fe640 1 -- 172.21.15.16:0/3653769775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fafc00041d0 con 0x7fafcc102790 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 Processor -- start 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 -- start start 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c35d0 0x55c01d7c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7ce370 0x55c01d7d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c01d7292f0 con 0x55c01d7c35d0 2024-09-15T22:47:22.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c01d6d0cf0 con 0x55c01d7ce370 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafd2f41740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c01d7252b0 con 0x55c01d7c4b40 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.327+0000 7fafca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c35d0 0x55c01d7c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c35d0 0x55c01d7c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37364/0 (socket says 172.21.15.16:37364) 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 -- 172.21.15.16:0/2261450115 learned_addr learned my addr 172.21.15.16:0/2261450115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7ce370 0x55c01d7d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafcad9d640 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafbb7fe640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fafb40a33b0 0x7fafb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafcad9d640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fafb40a33b0 0x7fafb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 -- 172.21.15.16:0/2261450115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 msgr2=0x55c01d7c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 -- 172.21.15.16:0/2261450115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7ce370 msgr2=0x55c01d7d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7ce370 0x55c01d7d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 -- 172.21.15.16:0/2261450115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01d6b53a0 con 0x55c01d7c35d0 2024-09-15T22:47:22.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafbb7fe640 1 -- 172.21.15.16:0/3653769775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fafc00cfb70 con 0x7fafcc102790 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafca59c640 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c35d0 0x55c01d7c3a10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009900 tx=0x7fafbc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11963e04f72c187 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafb8ff9640 1 -- 172.21.15.16:0/2261450115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc03f070 con 0x55c01d7c35d0 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafb8ff9640 1 -- 172.21.15.16:0/2261450115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafbc004030 con 0x55c01d7c35d0 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafcad9d640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fafb40a33b0 0x7fafb40a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005fd0 tx=0x7fafc4004ab0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.328+0000 7fafb8ff9640 1 -- 172.21.15.16:0/2261450115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc035070 con 0x55c01d7c35d0 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 -- 172.21.15.16:0/2261450115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c35d0 msgr2=0x55c01d7c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c35d0 0x55c01d7c3a10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009900 tx=0x7fafbc005e80 comp rx=0 tx=0).stop 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 -- 172.21.15.16:0/2261450115 shutdown_connections 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7ce370 0x55c01d7d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c35d0 0x55c01d7c3a10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 --2- 172.21.15.16:0/2261450115 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 -- 172.21.15.16:0/2261450115 >> 172.21.15.16:0/2261450115 conn(0x55c01d78c310 msgr2=0x55c01d7bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 -- 172.21.15.16:0/2261450115 shutdown_connections 2024-09-15T22:47:22.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.329+0000 7fafd2f41740 1 -- 172.21.15.16:0/2261450115 wait complete. 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 -- start start 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 Processor -- start 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c35d0 0x55c01d7cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c4b40 0x55c01d7cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c01d7d1010 con 0x55c01d7ce370 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c01d6e4060 con 0x55c01d7c4b40 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c01d7d1190 con 0x55c01d7c35d0 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafd2f41740 10 client.0 _ll_register_callbacks cb 0x55c01d4378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafcad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafcad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37372/0 (socket says 172.21.15.16:37372) 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafcad9d640 1 -- 172.21.15.16:0/464966123 learned_addr learned my addr 172.21.15.16:0/464966123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:22.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafca59c640 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c35d0 0x55c01d7cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c4b40 0x55c01d7cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafcad9d640 1 -- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c35d0 msgr2=0x55c01d7cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafcad9d640 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c35d0 0x55c01d7cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafcad9d640 1 -- 172.21.15.16:0/464966123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c4b40 msgr2=0x55c01d7cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.330+0000 7fafcad9d640 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c4b40 0x55c01d7cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafcad9d640 1 -- 172.21.15.16:0/464966123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c01d49a3c0 con 0x55c01d7ce370 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafcad9d640 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7cb5d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fafc40126a0 tx=0x7fafc4009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89c6647858f856f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafb1ffb640 1 -- 172.21.15.16:0/464966123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4020640 con 0x55c01d7ce370 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafb1ffb640 1 -- 172.21.15.16:0/464966123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fafc40207e0 con 0x55c01d7ce370 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafbc004030 con 0x55c01d7ce370 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafb1ffb640 1 -- 172.21.15.16:0/464966123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafc4024020 con 0x55c01d7ce370 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafd2f41740 10 client.17346 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:22.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c01d7dcad0 con 0x55c01d7ce370 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafb1ffb640 1 -- 172.21.15.16:0/464966123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafbc004030 con 0x55c01d7ce370 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafb1ffb640 1 -- 172.21.15.16:0/464966123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fafc401cc20 con 0x55c01d7ce370 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafd2f41740 10 client.17346 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.331+0000 7fafd2f41740 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.332+0000 7fafd2f41740 20 client.17346 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.332+0000 7fafd2f41740 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d7ddf90 0x55c01d7e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.332+0000 7fafd2f41740 4 client.17346 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.332+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c01d736860 con 0x55c01d7ddf90 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.332+0000 7fafca59c640 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d7ddf90 0x55c01d7e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.332+0000 7fafca59c640 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d7ddf90 0x55c01d7e0380 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d28b8febc700c6ad server_cookie=3e31907f5745c673 in_seq=0 out_seq=0 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.332+0000 7fafb1ffb640 10 client.17346 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafb1ffb640 1 -- 172.21.15.16:0/464966123 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c01d736860 con 0x55c01d7ddf90 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafb1ffb640 10 client.17346 handle_command_reply: tid=0 2024-09-15T22:47:22.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 client.17346 shutdown 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 msgr2=0x55c01d7cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7cb5d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fafc40126a0 tx=0x7fafc4009490 comp rx=0 tx=0).stop 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 shutdown_connections 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d7ddf90 0x55c01d7e0380 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7cb5d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c4b40 0x55c01d7cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafb1ffb640 0 client.17346 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 --2- 172.21.15.16:0/464966123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c35d0 0x55c01d7cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 >> 172.21.15.16:0/464966123 conn(0x55c01d78c310 msgr2=0x55c01d7c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 shutdown_connections 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 1 -- 172.21.15.16:0/464966123 wait complete. 2024-09-15T22:47:22.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.333+0000 7fafd2f41740 20 client.17346 trim_cache size 0 max 16384 2024-09-15T22:47:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.346+0000 7fafd2f41740 1 Processor -- start 2024-09-15T22:47:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafd2f41740 1 -- start start 2024-09-15T22:47:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c4b40 0x55c01d7cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7ce370 0x55c01d7cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafd2f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1b3b70 con 0x55c01d7c4b40 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafd2f41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1b1440 con 0x55c01d7c35d0 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafd2f41740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1af2b0 con 0x55c01d7ce370 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c4b40 0x55c01d7cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c4b40 0x55c01d7cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37386/0 (socket says 172.21.15.16:37386) 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 -- 172.21.15.16:0/1959309289 learned_addr learned my addr 172.21.15.16:0/1959309289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafcad9d640 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7ce370 0x55c01d7cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafca59c640 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 -- 172.21.15.16:0/1959309289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7ce370 msgr2=0x55c01d7cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7ce370 0x55c01d7cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 -- 172.21.15.16:0/1959309289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 msgr2=0x55c01d7da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 -- 172.21.15.16:0/1959309289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c01d7e0960 con 0x55c01d7c4b40 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.347+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c4b40 0x55c01d7cf760 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fafc0097f00 tx=0x7fafc0086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32828a04665ce9e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafb3fff640 1 -- 172.21.15.16:0/1959309289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00b4070 con 0x55c01d7c4b40 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafb3fff640 1 -- 172.21.15.16:0/1959309289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc009baa0 con 0x55c01d7c4b40 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafb3fff640 1 -- 172.21.15.16:0/1959309289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00cd320 con 0x55c01d7c4b40 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 -- 172.21.15.16:0/1959309289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c4b40 msgr2=0x55c01d7cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c4b40 0x55c01d7cf760 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fafc0097f00 tx=0x7fafc0086040 comp rx=0 tx=0).stop 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 -- 172.21.15.16:0/1959309289 shutdown_connections 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7ce370 0x55c01d7cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7c4b40 0x55c01d7cf760 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 --2- 172.21.15.16:0/1959309289 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 -- 172.21.15.16:0/1959309289 >> 172.21.15.16:0/1959309289 conn(0x55c01d78c310 msgr2=0x55c01d7cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:22.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 -- 172.21.15.16:0/1959309289 shutdown_connections 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.348+0000 7fafd2f41740 1 -- 172.21.15.16:0/1959309289 wait complete. 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 -- start start 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 Processor -- start 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7d6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7e5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1af900 con 0x55c01d7ce370 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1b0d70 con 0x55c01d7c35d0 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1affd0 con 0x55c01d7c4b40 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafd2f41740 10 client.0 _ll_register_callbacks cb 0x7fafc4003f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafcad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafcad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37402/0 (socket says 172.21.15.16:37402) 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafcad9d640 1 -- 172.21.15.16:0/655689500 learned_addr learned my addr 172.21.15.16:0/655689500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:22.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafc9d9b640 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7d6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafca59c640 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafcad9d640 1 -- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 msgr2=0x55c01d7d6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.349+0000 7fafcad9d640 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7d6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafcad9d640 1 -- 172.21.15.16:0/655689500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 msgr2=0x55c01d7d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafcad9d640 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7d6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafcad9d640 1 -- 172.21.15.16:0/655689500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c01d7e5a90 con 0x55c01d7ce370 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafcad9d640 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7e5260 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fafc4004a80 tx=0x7fafc40208f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59c729a0de0b5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafb1ffb640 1 -- 172.21.15.16:0/655689500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4020cf0 con 0x55c01d7ce370 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc009baa0 con 0x55c01d7ce370 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafb1ffb640 1 -- 172.21.15.16:0/655689500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fafc401e680 con 0x55c01d7ce370 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafb1ffb640 1 -- 172.21.15.16:0/655689500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fafc401e880 con 0x55c01d7ce370 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafd2f41740 10 client.17352 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:22.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c01d7d60d0 con 0x55c01d7ce370 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafb1ffb640 1 -- 172.21.15.16:0/655689500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc009baa0 con 0x55c01d7ce370 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.350+0000 7fafb1ffb640 1 -- 172.21.15.16:0/655689500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fafc402aa90 con 0x55c01d7ce370 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafd2f41740 10 client.17352 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafd2f41740 10 client.17352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafd2f41740 20 client.17352 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafd2f41740 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d848f40 0x55c01d84b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafd2f41740 4 client.17352 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c01d730d70 con 0x55c01d848f40 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafca59c640 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d848f40 0x55c01d84b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafca59c640 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d848f40 0x55c01d84b360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6160bb1425b2fbe9 server_cookie=aff3be2096c25891 in_seq=0 out_seq=0 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafb1ffb640 10 client.17352 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.351+0000 7fafb1ffb640 1 -- 172.21.15.16:0/655689500 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c01d730d70 con 0x55c01d848f40 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafb1ffb640 10 client.17352 handle_command_reply: tid=0 2024-09-15T22:47:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 client.17352 shutdown 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 msgr2=0x55c01d7e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7e5260 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fafc4004a80 tx=0x7fafc40208f0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 shutdown_connections 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c01d848f40 0x55c01d84b360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c01d7ce370 0x55c01d7e5260 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c01d7c4b40 0x55c01d7d6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 --2- 172.21.15.16:0/655689500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c01d7c35d0 0x55c01d7d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 >> 172.21.15.16:0/655689500 conn(0x55c01d78c310 msgr2=0x55c01d7c9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafb1ffb640 0 client.17352 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 shutdown_connections 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 1 -- 172.21.15.16:0/655689500 wait complete. 2024-09-15T22:47:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd2f41740 20 client.17352 trim_cache size 0 max 16384 2024-09-15T22:47:22.360 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:22.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fafb40a33b0 msgr2=0x7fafb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd0b68640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fafb40a33b0 0x7fafb40a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fafc4005fd0 tx=0x7fafc4004ab0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc102790 msgr2=0x7fafcc19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:22.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.352+0000 7fafd0b68640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc102790 0x7fafcc19abc0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fafc000d950 tx=0x7fafc000de10 comp rx=0 tx=0).stop 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 shutdown_connections 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafcc106f10 0x7fafcc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafcc102790 0x7fafcc19abc0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fafb40a33b0 0x7fafb40a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 --2- 172.21.15.16:0/3653769775 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fafcc101de0 0x7fafcc19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 >> 172.21.15.16:0/3653769775 conn(0x7fafcc0fd8c0 msgr2=0x7fafcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 shutdown_connections 2024-09-15T22:47:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:22.353+0000 7fafd0b68640 1 -- 172.21.15.16:0/3653769775 wait complete. 2024-09-15T22:47:22.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:22 smithi059 ceph-mon[34872]: pgmap v277: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:47:23.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:22 smithi130 ceph-mon[35742]: pgmap v277: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:47:23.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:22 smithi016 ceph-mon[31492]: pgmap v277: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:47:23.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:47:23.385 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:23.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 -- 172.21.15.16:0/3736236155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988106a80 msgr2=0x7f1988106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 --2- 172.21.15.16:0/3736236155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988106a80 0x7f1988106e60 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f197c009930 tx=0x7f197c02a420 comp rx=0 tx=0).stop 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 -- 172.21.15.16:0/3736236155 shutdown_connections 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 --2- 172.21.15.16:0/3736236155 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1988101420 0x7f198810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 --2- 172.21.15.16:0/3736236155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1988100a80 0x7f1988100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 --2- 172.21.15.16:0/3736236155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988106a80 0x7f1988106e60 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 -- 172.21.15.16:0/3736236155 >> 172.21.15.16:0/3736236155 conn(0x7f19880fc7a0 msgr2=0x7f19880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 -- 172.21.15.16:0/3736236155 shutdown_connections 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.872+0000 7f198f022640 1 -- 172.21.15.16:0/3736236155 wait complete. 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 Processor -- start 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 -- start start 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988100a80 0x7f19881984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1988101420 0x7f1988198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1988106a80 0x7f198819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1988110ea0 con 0x7f1988100a80 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1988110d20 con 0x7f1988101420 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198f022640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1988111020 con 0x7f1988106a80 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988100a80 0x7f19881984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988100a80 0x7f19881984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37418/0 (socket says 172.21.15.16:37418) 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 -- 172.21.15.16:0/776159159 learned_addr learned my addr 172.21.15.16:0/776159159 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198d5bf640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1988106a80 0x7f198819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f1987fff640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1988101420 0x7f1988198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 -- 172.21.15.16:0/776159159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1988106a80 msgr2=0x7f198819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1988106a80 0x7f198819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 -- 172.21.15.16:0/776159159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1988101420 msgr2=0x7f1988198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1988101420 0x7f1988198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198cdbe640 1 -- 172.21.15.16:0/776159159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198819d480 con 0x7f1988100a80 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f1987fff640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1988101420 0x7f1988198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.873+0000 7f198d5bf640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1988106a80 0x7f198819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.874+0000 7f198cdbe640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988100a80 0x7f19881984d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f197c009a50 tx=0x7f197c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.874+0000 7f1985ffb640 1 -- 172.21.15.16:0/776159159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c03f030 con 0x7f1988100a80 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.874+0000 7f1985ffb640 1 -- 172.21.15.16:0/776159159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f197c003ce0 con 0x7f1988100a80 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.874+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198819d710 con 0x7f1988100a80 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.874+0000 7f1985ffb640 1 -- 172.21.15.16:0/776159159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c035070 con 0x7f1988100a80 2024-09-15T22:47:23.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.874+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a5050 con 0x7f1988100a80 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.876+0000 7f1985ffb640 1 -- 172.21.15.16:0/776159159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f197c042030 con 0x7f1988100a80 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 Processor -- start 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 -- start start 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571757222f0 con 0x5571757bdb40 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571756c9cf0 con 0x5571757bc5d0 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f19913fb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55717571e2b0 con 0x5571757c7370 2024-09-15T22:47:23.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37430/0 (socket says 172.21.15.16:37430) 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 -- 172.21.15.16:0/1665974147 learned_addr learned my addr 172.21.15.16:0/1665974147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f1985ffb640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19780a3430 0x7f19780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f1987fff640 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198cdbe640 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 -- 172.21.15.16:0/1665974147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 msgr2=0x5571757c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 -- 172.21.15.16:0/1665974147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 msgr2=0x5571757bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 -- 172.21.15.16:0/1665974147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571756ae3a0 con 0x5571757bdb40 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198cdbe640 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:23.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f1985ffb640 1 -- 172.21.15.16:0/776159159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f197c0f0740 con 0x7f1988100a80 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.877+0000 7f198d5bf640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19780a3430 0x7f19780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.878+0000 7f198d5bf640 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bc090 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f198000c550 tx=0x7f198000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f1fd26b883a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.878+0000 7f1972ffd640 1 -- 172.21.15.16:0/1665974147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198000a600 con 0x5571757bdb40 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.878+0000 7f1972ffd640 1 -- 172.21.15.16:0/1665974147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198000ac80 con 0x5571757bdb40 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.878+0000 7f198d5bf640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19780a3430 0x7f19780a58f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f198001f000 tx=0x7f1980018000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.878+0000 7f1972ffd640 1 -- 172.21.15.16:0/1665974147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980017a40 con 0x5571757bdb40 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- 172.21.15.16:0/1665974147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 msgr2=0x5571757bc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bc090 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f198000c550 tx=0x7f198000ca10 comp rx=0 tx=0).stop 2024-09-15T22:47:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- 172.21.15.16:0/1665974147 shutdown_connections 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 --2- 172.21.15.16:0/1665974147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bc090 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- 172.21.15.16:0/1665974147 >> 172.21.15.16:0/1665974147 conn(0x557175785310 msgr2=0x5571757b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- 172.21.15.16:0/1665974147 shutdown_connections 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- 172.21.15.16:0/1665974147 wait complete. 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- start start 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 Processor -- start 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bc5d0 0x5571757c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bdb40 0x5571757c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5571757ca070 con 0x5571757bc5d0 2024-09-15T22:47:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5571756dd060 con 0x5571757bdb40 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.879+0000 7f19913fb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5571757ca1f0 con 0x5571757c7370 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f19913fb740 10 client.0 _ll_register_callbacks cb 0x5571754308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bc5d0 0x5571757c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bc5d0 0x5571757c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37440/0 (socket says 172.21.15.16:37440) 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 -- 172.21.15.16:0/3438808529 learned_addr learned my addr 172.21.15.16:0/3438808529 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f1987fff640 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bdb40 0x5571757c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198d5bf640 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 -- 172.21.15.16:0/3438808529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 msgr2=0x5571757c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 -- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bdb40 msgr2=0x5571757c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bdb40 0x5571757c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 -- 172.21.15.16:0/3438808529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571754933c0 con 0x5571757bc5d0 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f198cdbe640 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bc5d0 0x5571757c6d80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f197c0bc5a0 tx=0x7f197c08b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9382a7127f7d906 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f1967fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f1967fff640 1 -- 172.21.15.16:0/3438808529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c049040 con 0x5571757bc5d0 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f1967fff640 1 -- 172.21.15.16:0/3438808529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f197c0b8c80 con 0x5571757bc5d0 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f198000ac80 con 0x5571757bc5d0 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f1967fff640 1 -- 172.21.15.16:0/3438808529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f197c0bbf70 con 0x5571757bc5d0 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f19913fb740 10 client.17364 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.880+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571757d5ab0 con 0x5571757bc5d0 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f1967fff640 1 -- 172.21.15.16:0/3438808529 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198000ac80 con 0x5571757bc5d0 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f1967fff640 1 -- 172.21.15.16:0/3438808529 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f197c03b460 con 0x5571757bc5d0 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f19913fb740 10 client.17364 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f19913fb740 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f19913fb740 20 client.17364 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f19913fb740 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x5571757d92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f19913fb740 4 client.17364 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55717572f860 con 0x5571757d6ec0 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.881+0000 7f1987fff640 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x5571757d92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.882+0000 7f1987fff640 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x5571757d92d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a61485ab0257dbfa server_cookie=fcc2a0bd288744af in_seq=0 out_seq=0 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.882+0000 7f1967fff640 10 client.17364 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.882+0000 7f1967fff640 1 -- 172.21.15.16:0/3438808529 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55717572f860 con 0x5571757d6ec0 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.882+0000 7f1967fff640 10 client.17364 handle_command_reply: tid=0 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.882+0000 7f19913fb740 1 client.17364 shutdown 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bc5d0 msgr2=0x5571757c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bc5d0 0x5571757c6d80 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f197c0bc5a0 tx=0x7f197c08b300 comp rx=0 tx=0).stop 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 shutdown_connections 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x5571757d92d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x5571757c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f1967fff640 0 client.17364 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bdb40 0x5571757c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 --2- 172.21.15.16:0/3438808529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bc5d0 0x5571757c6d80 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 >> 172.21.15.16:0/3438808529 conn(0x557175785310 msgr2=0x5571757be930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 shutdown_connections 2024-09-15T22:47:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 1 -- 172.21.15.16:0/3438808529 wait complete. 2024-09-15T22:47:23.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.883+0000 7f19913fb740 20 client.17364 trim_cache size 0 max 16384 2024-09-15T22:47:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 Processor -- start 2024-09-15T22:47:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 -- start start 2024-09-15T22:47:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757bc5d0 0x5571757bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757c7370 0x5571757debe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19881b1910 con 0x5571757bdb40 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19881af1e0 con 0x5571757c7370 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f19913fb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ad050 con 0x5571757bc5d0 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f1987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f1987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37444/0 (socket says 172.21.15.16:37444) 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f1987fff640 1 -- 172.21.15.16:0/500011520 learned_addr learned my addr 172.21.15.16:0/500011520 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.896+0000 7f198d5bf640 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757c7370 0x5571757debe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f198cdbe640 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757bc5d0 0x5571757bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f1987fff640 1 -- 172.21.15.16:0/500011520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757bc5d0 msgr2=0x5571757bf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f1987fff640 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757bc5d0 0x5571757bf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f1987fff640 1 -- 172.21.15.16:0/500011520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757c7370 msgr2=0x5571757debe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f1987fff640 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757c7370 0x5571757debe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f1987fff640 1 -- 172.21.15.16:0/500011520 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571757bc2d0 con 0x5571757bdb40 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f198d5bf640 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757c7370 0x5571757debe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f1987fff640 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bf660 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f19880fecf0 tx=0x7f19740138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e456d8c3f20ab103 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f19727fc640 1 -- 172.21.15.16:0/500011520 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974013190 con 0x5571757bdb40 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f19727fc640 1 -- 172.21.15.16:0/500011520 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19740035f0 con 0x5571757bdb40 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.897+0000 7f19727fc640 1 -- 172.21.15.16:0/500011520 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740116a0 con 0x5571757bdb40 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 -- 172.21.15.16:0/500011520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 msgr2=0x5571757bf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bf660 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f19880fecf0 tx=0x7f19740138f0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 -- 172.21.15.16:0/500011520 shutdown_connections 2024-09-15T22:47:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757c7370 0x5571757debe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757bf660 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 --2- 172.21.15.16:0/500011520 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757bc5d0 0x5571757bf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 -- 172.21.15.16:0/500011520 >> 172.21.15.16:0/500011520 conn(0x557175785310 msgr2=0x5571757b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 -- 172.21.15.16:0/500011520 shutdown_connections 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 -- 172.21.15.16:0/500011520 wait complete. 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 -- start start 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.898+0000 7f19913fb740 1 Processor -- start 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f19913fb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f19913fb740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f19913fb740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x55717583abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f19913fb740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19881ad6a0 con 0x5571757bdb40 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f19913fb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f19881aeb10 con 0x5571757bc5d0 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f19913fb740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f19881add70 con 0x5571757c7370 2024-09-15T22:47:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f19913fb740 10 client.0 _ll_register_callbacks cb 0x7f197c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37448/0 (socket says 172.21.15.16:37448) 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 -- 172.21.15.16:0/4112910034 learned_addr learned my addr 172.21.15.16:0/4112910034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f198cdbe640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f198d5bf640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x55717583abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 -- 172.21.15.16:0/4112910034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 msgr2=0x55717583abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x55717583abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 -- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 msgr2=0x5571757cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 -- 172.21.15.16:0/4112910034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571757b9960 con 0x5571757bdb40 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f198d5bf640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x55717583abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1987fff640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757cbfd0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x5571757b8530 tx=0x7f197400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292d2e3205b39dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:23.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1967fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.899+0000 7f1967fff640 1 -- 172.21.15.16:0/4112910034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740037a0 con 0x5571757bdb40 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f1967fff640 1 -- 172.21.15.16:0/4112910034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f1974021040 con 0x5571757bdb40 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19740035f0 con 0x5571757bdb40 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f1967fff640 1 -- 172.21.15.16:0/4112910034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f197403a050 con 0x5571757bdb40 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 10 client.17370 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571758423b0 con 0x5571757bdb40 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f1967fff640 1 -- 172.21.15.16:0/4112910034 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19740035f0 con 0x5571757bdb40 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f1967fff640 1 -- 172.21.15.16:0/4112910034 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1974011a90 con 0x5571757bdb40 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 10 client.17370 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 10 client.17370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 20 client.17370 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x557175844b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 4 client.17370 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557175729d70 con 0x5571757d6ec0 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.900+0000 7f198cdbe640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x557175844b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f198cdbe640 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x557175844b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efe1dc18eaee326c server_cookie=93bd9a4a3ac51c20 in_seq=0 out_seq=0 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f1967fff640 10 client.17370 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f1967fff640 1 -- 172.21.15.16:0/4112910034 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557175729d70 con 0x5571757d6ec0 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f1967fff640 10 client.17370 handle_command_reply: tid=0 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 client.17370 shutdown 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 msgr2=0x5571757cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757cbfd0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x5571757b8530 tx=0x7f197400caf0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 shutdown_connections 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5571757d6ec0 0x557175844b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f1967fff640 0 client.17370 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5571757c7370 0x55717583abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5571757bdb40 0x5571757cbfd0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 --2- 172.21.15.16:0/4112910034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5571757bc5d0 0x5571757cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 >> 172.21.15.16:0/4112910034 conn(0x557175785310 msgr2=0x5571757c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 shutdown_connections 2024-09-15T22:47:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 1 -- 172.21.15.16:0/4112910034 wait complete. 2024-09-15T22:47:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.901+0000 7f19913fb740 20 client.17370 trim_cache size 0 max 16384 2024-09-15T22:47:23.914 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19780a3430 msgr2=0x7f19780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19780a3430 0x7f19780a58f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f198001f000 tx=0x7f1980018000 comp rx=0 tx=0).stop 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988100a80 msgr2=0x7f19881984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988100a80 0x7f19881984d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f197c009a50 tx=0x7f197c004270 comp rx=0 tx=0).stop 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 shutdown_connections 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f19780a3430 0x7f19780a58f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1988106a80 0x7f198819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1988101420 0x7f1988198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 --2- 172.21.15.16:0/776159159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1988100a80 0x7f19881984d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 >> 172.21.15.16:0/776159159 conn(0x7f19880fc7a0 msgr2=0x7f198806a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 shutdown_connections 2024-09-15T22:47:23.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:23.902+0000 7f198f022640 1 -- 172.21.15.16:0/776159159 wait complete. 2024-09-15T22:47:24.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:47:24.850 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:24.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:24 smithi059 ceph-mon[34872]: pgmap v278: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:47:24.971 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:24 smithi016 ceph-mon[31492]: pgmap v278: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:47:25.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:24 smithi130 ceph-mon[35742]: pgmap v278: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 -- 172.21.15.16:0/3332638507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57901036c0 msgr2=0x7f579010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 --2- 172.21.15.16:0/3332638507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57901036c0 0x7f579010bbb0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5780009900 tx=0x7f578002a480 comp rx=0 tx=0).stop 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 -- 172.21.15.16:0/3332638507 shutdown_connections 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 --2- 172.21.15.16:0/3332638507 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f579010c180 0x7f579010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 --2- 172.21.15.16:0/3332638507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57901036c0 0x7f579010bbb0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 --2- 172.21.15.16:0/3332638507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5790102da0 0x7f5790103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 -- 172.21.15.16:0/3332638507 >> 172.21.15.16:0/3332638507 conn(0x7f57900fc7a0 msgr2=0x7f57900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:25.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 -- 172.21.15.16:0/3332638507 shutdown_connections 2024-09-15T22:47:25.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.483+0000 7f57977b9640 1 -- 172.21.15.16:0/3332638507 wait complete. 2024-09-15T22:47:25.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 Processor -- start 2024-09-15T22:47:25.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 -- start start 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5790102da0 0x7f579019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57901036c0 0x7f579019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579010c180 0x7f579019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5790111f60 con 0x7f579010c180 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5790111de0 con 0x7f5790102da0 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f57977b9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f57901120e0 con 0x7f57901036c0 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f5795d56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579010c180 0x7f579019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f5795d56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579010c180 0x7f579019eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37476/0 (socket says 172.21.15.16:37476) 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f5795d56640 1 -- 172.21.15.16:0/3284900421 learned_addr learned my addr 172.21.15.16:0/3284900421 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:25.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f5795555640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5790102da0 0x7f579019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.484+0000 7f5794d54640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57901036c0 0x7f579019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f5795d56640 1 -- 172.21.15.16:0/3284900421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57901036c0 msgr2=0x7f579019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f5795d56640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57901036c0 0x7f579019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f5795d56640 1 -- 172.21.15.16:0/3284900421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5790102da0 msgr2=0x7f579019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f5795d56640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5790102da0 0x7f579019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f5795d56640 1 -- 172.21.15.16:0/3284900421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579019f5b0 con 0x7f579010c180 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f5795555640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5790102da0 0x7f579019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f5795d56640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579010c180 0x7f579019eed0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f578800cce0 tx=0x7f578800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f57867fc640 1 -- 172.21.15.16:0/3284900421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578800f450 con 0x7f579010c180 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579019f8a0 con 0x7f579010c180 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f57867fc640 1 -- 172.21.15.16:0/3284900421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578800fad0 con 0x7f579010c180 2024-09-15T22:47:25.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f57867fc640 1 -- 172.21.15.16:0/3284900421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788018c10 con 0x7f579010c180 2024-09-15T22:47:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.485+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57901a7180 con 0x7f579010c180 2024-09-15T22:47:25.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.487+0000 7f57867fc640 1 -- 172.21.15.16:0/3284900421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f578800f5f0 con 0x7f579010c180 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 Processor -- start 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 -- start start 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c531b40 0x55925c530090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c5305d0 0x55925c530a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c53d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55925c4962f0 con 0x55925c5305d0 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55925c43dcf0 con 0x55925c531b40 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5799b92740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55925c4922b0 con 0x55925c53b370 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c5305d0 0x55925c530a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f57867fc640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f577c0a33b0 0x7f577c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c5305d0 0x55925c530a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37488/0 (socket says 172.21.15.16:37488) 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5795555640 1 -- 172.21.15.16:0/3157493631 learned_addr learned my addr 172.21.15.16:0/3157493631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5794d54640 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c531b40 0x55925c530090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.488+0000 7f5795d56640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f577c0a33b0 0x7f577c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f57867fc640 1 -- 172.21.15.16:0/3284900421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f57880d0820 con 0x7f579010c180 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5794d54640 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c53d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5795555640 1 -- 172.21.15.16:0/3157493631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 msgr2=0x55925c53d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5795555640 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c53d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5795555640 1 -- 172.21.15.16:0/3157493631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c531b40 msgr2=0x55925c530090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5795555640 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c531b40 0x55925c530090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5795555640 1 -- 172.21.15.16:0/3157493631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925c4223a0 con 0x55925c5305d0 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5794d54640 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c531b40 0x55925c530090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5795555640 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c5305d0 0x55925c530a10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f578c00bd40 tx=0x7f578c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca521449fc1f1b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f5795d56640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f577c0a33b0 0x7f577c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f578809c680 tx=0x7f578806e470 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f577b7fe640 1 -- 172.21.15.16:0/3157493631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c015070 con 0x55925c5305d0 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f577b7fe640 1 -- 172.21.15.16:0/3157493631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578c00be30 con 0x55925c5305d0 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.489+0000 7f577b7fe640 1 -- 172.21.15.16:0/3157493631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c011a00 con 0x55925c5305d0 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 -- 172.21.15.16:0/3157493631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c5305d0 msgr2=0x55925c530a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c5305d0 0x55925c530a10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f578c00bd40 tx=0x7f578c00f730 comp rx=0 tx=0).stop 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 -- 172.21.15.16:0/3157493631 shutdown_connections 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c53d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c5305d0 0x55925c530a10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 --2- 172.21.15.16:0/3157493631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c531b40 0x55925c530090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 -- 172.21.15.16:0/3157493631 >> 172.21.15.16:0/3157493631 conn(0x55925c4f9310 msgr2=0x55925c52cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 -- 172.21.15.16:0/3157493631 shutdown_connections 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 -- 172.21.15.16:0/3157493631 wait complete. 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.490+0000 7f5799b92740 1 -- start start 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 1 Processor -- start 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c5305d0 0x55925c53ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c537fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c5385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55925c53e070 con 0x55925c531b40 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55925c451060 con 0x55925c5305d0 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55925c53e1f0 con 0x55925c53b370 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5799b92740 10 client.0 _ll_register_callbacks cb 0x55925c1a48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c537fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c537fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37504/0 (socket says 172.21.15.16:37504) 2024-09-15T22:47:25.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5795555640 1 -- 172.21.15.16:0/3966162687 learned_addr learned my addr 172.21.15.16:0/3966162687 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5794d54640 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c5305d0 0x55925c53ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.491+0000 7f5794d54640 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c5385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5795555640 1 -- 172.21.15.16:0/3966162687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 msgr2=0x55925c5385d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5795555640 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c5385d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5795555640 1 -- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c5305d0 msgr2=0x55925c53ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5795555640 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c5305d0 0x55925c53ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5795555640 1 -- 172.21.15.16:0/3966162687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925c2073c0 con 0x55925c531b40 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5795555640 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c537fe0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f578c011620 tx=0x7f578c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b19aac6984d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5778ff9640 1 -- 172.21.15.16:0/3966162687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c028060 con 0x55925c531b40 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5778ff9640 1 -- 172.21.15.16:0/3966162687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f578c005480 con 0x55925c531b40 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f578c00be30 con 0x55925c531b40 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5778ff9640 1 -- 172.21.15.16:0/3966162687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f578c02f050 con 0x55925c531b40 2024-09-15T22:47:25.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5799b92740 10 client.17382 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55925c549ad0 con 0x55925c531b40 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.492+0000 7f5778ff9640 1 -- 172.21.15.16:0/3966162687 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578c011c60 con 0x55925c531b40 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5778ff9640 1 -- 172.21.15.16:0/3966162687 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f578c004cd0 con 0x55925c531b40 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5799b92740 10 client.17382 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5799b92740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5799b92740 20 client.17382 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5799b92740 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c54ae20 0x55925c54d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5799b92740 4 client.17382 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f578c00be30 con 0x55925c54ae20 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5794d54640 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c54ae20 0x55925c54d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5794d54640 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c54ae20 0x55925c54d210 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e0e84d8be42a83 server_cookie=31f30b34cefe12db in_seq=0 out_seq=0 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.493+0000 7f5778ff9640 10 client.17382 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5778ff9640 1 -- 172.21.15.16:0/3966162687 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f578c00be30 con 0x55925c54ae20 2024-09-15T22:47:25.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5778ff9640 10 client.17382 handle_command_reply: tid=0 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 client.17382 shutdown 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 msgr2=0x55925c537fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c537fe0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f578c011620 tx=0x7f578c005040 comp rx=0 tx=0).stop 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 shutdown_connections 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c54ae20 0x55925c54d210 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c53b370 0x55925c5385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c537fe0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5778ff9640 0 client.17382 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 --2- 172.21.15.16:0/3966162687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c5305d0 0x55925c53ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.494+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 >> 172.21.15.16:0/3966162687 conn(0x55925c4f9310 msgr2=0x55925c532960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.495+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 shutdown_connections 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.495+0000 7f5799b92740 1 -- 172.21.15.16:0/3966162687 wait complete. 2024-09-15T22:47:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.495+0000 7f5799b92740 20 client.17382 trim_cache size 0 max 16384 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 Processor -- start 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 -- start start 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c539f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c533120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c533660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57901b3b00 con 0x55925c531b40 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57901b13d0 con 0x55925c53b370 2024-09-15T22:47:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5799b92740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f57901af240 con 0x55925c5305d0 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c533120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c533120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37518/0 (socket says 172.21.15.16:37518) 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 -- 172.21.15.16:0/4222237264 learned_addr learned my addr 172.21.15.16:0/4222237264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5794d54640 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c539f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5794d54640 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c533660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 -- 172.21.15.16:0/4222237264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 msgr2=0x55925c539f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c539f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 -- 172.21.15.16:0/4222237264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 msgr2=0x55925c533660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c533660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5795555640 1 -- 172.21.15.16:0/4222237264 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925c52c4f0 con 0x55925c531b40 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.508+0000 7f5794d54640 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c539f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5795555640 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c533120 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f578c01a650 tx=0x7f578c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8282428261eabd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f577affd640 1 -- 172.21.15.16:0/4222237264 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c028040 con 0x55925c531b40 2024-09-15T22:47:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f577affd640 1 -- 172.21.15.16:0/4222237264 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578c01a850 con 0x55925c531b40 2024-09-15T22:47:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f577affd640 1 -- 172.21.15.16:0/4222237264 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c015070 con 0x55925c531b40 2024-09-15T22:47:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 -- 172.21.15.16:0/4222237264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 msgr2=0x55925c533120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c533120 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f578c01a650 tx=0x7f578c0050b0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 -- 172.21.15.16:0/4222237264 shutdown_connections 2024-09-15T22:47:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c533660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c533120 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 --2- 172.21.15.16:0/4222237264 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c539f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 -- 172.21.15.16:0/4222237264 >> 172.21.15.16:0/4222237264 conn(0x55925c4f9310 msgr2=0x55925c539840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 -- 172.21.15.16:0/4222237264 shutdown_connections 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.509+0000 7f5799b92740 1 -- 172.21.15.16:0/4222237264 wait complete. 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 -- start start 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 Processor -- start 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c5350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c535610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c535b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57901af890 con 0x55925c531b40 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f57901b0d00 con 0x55925c53b370 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f57901aff60 con 0x55925c5305d0 2024-09-15T22:47:25.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5799b92740 10 client.0 _ll_register_callbacks cb 0x7f578c00bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c535610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5795555640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c535610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37534/0 (socket says 172.21.15.16:37534) 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.510+0000 7f5795555640 1 -- 172.21.15.16:0/1963907591 learned_addr learned my addr 172.21.15.16:0/1963907591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5794d54640 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c535b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5794d54640 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c5350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5795555640 1 -- 172.21.15.16:0/1963907591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 msgr2=0x55925c5350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5795555640 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c5350d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5795555640 1 -- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 msgr2=0x55925c535b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5795555640 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c535b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5795555640 1 -- 172.21.15.16:0/1963907591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925c5b4860 con 0x55925c531b40 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5795555640 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c535610 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f578c00bc90 tx=0x7f578c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541139249d398ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5778ff9640 1 -- 172.21.15.16:0/1963907591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c015070 con 0x55925c531b40 2024-09-15T22:47:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5778ff9640 1 -- 172.21.15.16:0/1963907591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f578c01add0 con 0x55925c531b40 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f578c01a850 con 0x55925c531b40 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5778ff9640 1 -- 172.21.15.16:0/1963907591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f578c043050 con 0x55925c531b40 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5799b92740 10 client.17388 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.511+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55925c5b5ab0 con 0x55925c531b40 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5778ff9640 1 -- 172.21.15.16:0/1963907591 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f578c01a850 con 0x55925c531b40 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5778ff9640 1 -- 172.21.15.16:0/1963907591 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f578c039030 con 0x55925c531b40 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5799b92740 10 client.17388 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5799b92740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5799b92740 20 client.17388 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5799b92740 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c5b60c0 0x55925c5b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5799b92740 4 client.17388 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55925c4a3860 con 0x55925c5b60c0 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.512+0000 7f5794d54640 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c5b60c0 0x55925c5b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5794d54640 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c5b60c0 0x55925c5b84e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb8bd41b43f83004 server_cookie=8be4be177d675455 in_seq=0 out_seq=0 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5778ff9640 10 client.17388 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5778ff9640 1 -- 172.21.15.16:0/1963907591 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55925c4a3860 con 0x55925c5b60c0 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5778ff9640 10 client.17388 handle_command_reply: tid=0 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 client.17388 shutdown 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 msgr2=0x55925c535610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c535610 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f578c00bc90 tx=0x7f578c00fc30 comp rx=0 tx=0).stop 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 shutdown_connections 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55925c5b60c0 0x55925c5b84e0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55925c53b370 0x55925c535b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55925c531b40 0x55925c535610 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5778ff9640 0 client.17388 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 --2- 172.21.15.16:0/1963907591 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55925c5305d0 0x55925c5350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 >> 172.21.15.16:0/1963907591 conn(0x55925c4f9310 msgr2=0x55925c53a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 shutdown_connections 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 1 -- 172.21.15.16:0/1963907591 wait complete. 2024-09-15T22:47:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.513+0000 7f5799b92740 20 client.17388 trim_cache size 0 max 16384 2024-09-15T22:47:25.521 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f577c0a33b0 msgr2=0x7f577c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f577c0a33b0 0x7f577c0a5870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f578809c680 tx=0x7f578806e470 comp rx=0 tx=0).stop 2024-09-15T22:47:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579010c180 msgr2=0x7f579019eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:25.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579010c180 0x7f579019eed0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f578800cce0 tx=0x7f578800a620 comp rx=0 tx=0).stop 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 shutdown_connections 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579010c180 0x7f579019eed0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f577c0a33b0 0x7f577c0a5870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f57901036c0 0x7f579019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 --2- 172.21.15.16:0/3284900421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5790102da0 0x7f579019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 >> 172.21.15.16:0/3284900421 conn(0x7f57900fc7a0 msgr2=0x7f5790101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 shutdown_connections 2024-09-15T22:47:25.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:25.514+0000 7f57977b9640 1 -- 172.21.15.16:0/3284900421 wait complete. 2024-09-15T22:47:26.831 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:26 smithi130 ceph-mon[35742]: pgmap v279: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:47:26.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:26 smithi059 ceph-mon[34872]: pgmap v279: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:47:27.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:26 smithi016 ceph-mon[31492]: pgmap v279: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:47:27.277 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:47:27.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:47:27.551 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.017+0000 7f4b434a7640 1 -- 172.21.15.16:0/141689586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c1008d0 msgr2=0x7f4b3c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.017+0000 7f4b434a7640 1 --2- 172.21.15.16:0/141689586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c1008d0 0x7f4b3c100cb0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009900 tx=0x7f4b3802a410 comp rx=0 tx=0).stop 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.017+0000 7f4b434a7640 1 -- 172.21.15.16:0/141689586 shutdown_connections 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.017+0000 7f4b434a7640 1 --2- 172.21.15.16:0/141689586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b3c10e480 0x7f4b3c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.017+0000 7f4b434a7640 1 --2- 172.21.15.16:0/141689586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b3c101280 0x7f4b3c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.017+0000 7f4b434a7640 1 --2- 172.21.15.16:0/141689586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c1008d0 0x7f4b3c100cb0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.017+0000 7f4b434a7640 1 -- 172.21.15.16:0/141689586 >> 172.21.15.16:0/141689586 conn(0x7f4b3c0fc7a0 msgr2=0x7f4b3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 -- 172.21.15.16:0/141689586 shutdown_connections 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 -- 172.21.15.16:0/141689586 wait complete. 2024-09-15T22:47:28.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 Processor -- start 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 -- start start 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b3c1008d0 0x7f4b3c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c101280 0x7f4b3c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b3c10e480 0x7f4b3c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c114190 con 0x7f4b3c101280 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c114010 con 0x7f4b3c10e480 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b434a7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c114310 con 0x7f4b3c1008d0 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.018+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c101280 0x7f4b3c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c101280 0x7f4b3c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37552/0 (socket says 172.21.15.16:37552) 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 -- 172.21.15.16:0/2957771926 learned_addr learned my addr 172.21.15.16:0/2957771926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b41a44640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b3c10e480 0x7f4b3c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b41243640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b3c1008d0 0x7f4b3c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 -- 172.21.15.16:0/2957771926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b3c1008d0 msgr2=0x7f4b3c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b3c1008d0 0x7f4b3c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 -- 172.21.15.16:0/2957771926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b3c10e480 msgr2=0x7f4b3c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b3c10e480 0x7f4b3c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 -- 172.21.15.16:0/2957771926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3c19f620 con 0x7f4b3c101280 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b41243640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b3c1008d0 0x7f4b3c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b40a42640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c101280 0x7f4b3c19abb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c890 tx=0x7f4b2c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b327fc640 1 -- 172.21.15.16:0/2957771926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c004220 con 0x7f4b3c101280 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b327fc640 1 -- 172.21.15.16:0/2957771926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c0043c0 con 0x7f4b3c101280 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b327fc640 1 -- 172.21.15.16:0/2957771926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0049e0 con 0x7f4b3c101280 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3c19f910 con 0x7f4b3c101280 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.019+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3c1a71f0 con 0x7f4b3c101280 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.021+0000 7f4b327fc640 1 -- 172.21.15.16:0/2957771926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4b2c0079b0 con 0x7f4b3c101280 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 Processor -- start 2024-09-15T22:47:28.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 -- start start 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b327fc640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4b280a3430 0x7f4b280a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd2bb60 0x55812bd2bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd36800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd36d40 0x55812bd39130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55812bc91280 con 0x55812bd2c480 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55812bc38b10 con 0x55812bd36d40 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.022+0000 7f4b45880740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55812bc8d240 con 0x55812bd2bb60 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b41a44640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4b280a3430 0x7f4b280a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd36800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd36800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37556/0 (socket says 172.21.15.16:37556) 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 -- 172.21.15.16:0/1388892991 learned_addr learned my addr 172.21.15.16:0/1388892991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b327fc640 1 -- 172.21.15.16:0/2957771926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f4b2c0d00f0 con 0x7f4b3c101280 2024-09-15T22:47:28.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b41243640 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd2bb60 0x55812bd2bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 -- 172.21.15.16:0/1388892991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd2bb60 msgr2=0x55812bd2bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd2bb60 0x55812bd2bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 -- 172.21.15.16:0/1388892991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd36d40 msgr2=0x55812bd39130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd36d40 0x55812bd39130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 -- 172.21.15.16:0/1388892991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55812bc451f0 con 0x55812bd2c480 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b41a44640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4b280a3430 0x7f4b280a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c06a0f0 tx=0x7f4b34009a60 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b40a42640 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd36800 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c09bfb0 tx=0x7f4b2c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb14e3c04498243c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b277fe640 1 -- 172.21.15.16:0/1388892991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c01d040 con 0x55812bd2c480 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b277fe640 1 -- 172.21.15.16:0/1388892991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c09c720 con 0x55812bd2c480 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.023+0000 7f4b277fe640 1 -- 172.21.15.16:0/1388892991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0bd450 con 0x55812bd2c480 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 -- 172.21.15.16:0/1388892991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 msgr2=0x55812bd36800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd36800 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c09bfb0 tx=0x7f4b2c01b000 comp rx=0 tx=0).stop 2024-09-15T22:47:28.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 -- 172.21.15.16:0/1388892991 shutdown_connections 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd36d40 0x55812bd39130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd36800 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 --2- 172.21.15.16:0/1388892991 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd2bb60 0x55812bd2bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 -- 172.21.15.16:0/1388892991 >> 172.21.15.16:0/1388892991 conn(0x55812bd280f0 msgr2=0x55812bd284d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 -- 172.21.15.16:0/1388892991 shutdown_connections 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 -- 172.21.15.16:0/1388892991 wait complete. 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.024+0000 7f4b45880740 1 -- start start 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 1 Processor -- start 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd33c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd346f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55812bd39910 con 0x55812bd2c480 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55812bc4bf10 con 0x55812bd2bb60 2024-09-15T22:47:28.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55812bd39a90 con 0x55812bd36d40 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b45880740 10 client.0 _ll_register_callbacks cb 0x55812bc14a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd341b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37558/0 (socket says 172.21.15.16:37558) 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b40a42640 1 -- 172.21.15.16:0/2964609503 learned_addr learned my addr 172.21.15.16:0/2964609503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b41243640 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd33c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b41a44640 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd346f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b40a42640 1 -- 172.21.15.16:0/2964609503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 msgr2=0x55812bd346f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b40a42640 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd346f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b40a42640 1 -- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 msgr2=0x55812bd33c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.025+0000 7f4b40a42640 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd33c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b40a42640 1 -- 172.21.15.16:0/2964609503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812ba026a0 con 0x55812bd2c480 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b40a42640 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd341b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c007590 tx=0x7f4b2c0988a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd8dd104dd219d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:28.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b24ff9640 1 -- 172.21.15.16:0/2964609503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c01d040 con 0x55812bd2c480 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b24ff9640 1 -- 172.21.15.16:0/2964609503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f4b2c007ca0 con 0x55812bd2c480 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b2c09c720 con 0x55812bd2c480 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b24ff9640 1 -- 172.21.15.16:0/2964609503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b2c0a03d0 con 0x55812bd2c480 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 10 client.17400 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55812bd45550 con 0x55812bd2c480 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b24ff9640 1 -- 172.21.15.16:0/2964609503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b2c09c720 con 0x55812bd2c480 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b24ff9640 1 -- 172.21.15.16:0/2964609503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b2c0a4800 con 0x55812bd2c480 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 10 client.17400 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 20 client.17400 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bd48c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 4 client.17400 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:28.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.026+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55812bc9e7f0 con 0x55812bd468a0 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.027+0000 7f4b41243640 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bd48c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.027+0000 7f4b24ff9640 10 client.17400 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.027+0000 7f4b41243640 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bd48c90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2587e4576321300 server_cookie=f3a0956118df9d13 in_seq=0 out_seq=0 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b24ff9640 1 -- 172.21.15.16:0/2964609503 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55812bc9e7f0 con 0x55812bd468a0 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b24ff9640 10 client.17400 handle_command_reply: tid=0 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 client.17400 shutdown 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 msgr2=0x55812bd341b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd341b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c007590 tx=0x7f4b2c0988a0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 shutdown_connections 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bd48c90 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd346f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b24ff9640 0 client.17400 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd341b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 --2- 172.21.15.16:0/2964609503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd33c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 >> 172.21.15.16:0/2964609503 conn(0x55812bd280f0 msgr2=0x55812bd2de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:28.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 shutdown_connections 2024-09-15T22:47:28.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 1 -- 172.21.15.16:0/2964609503 wait complete. 2024-09-15T22:47:28.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.028+0000 7f4b45880740 20 client.17400 trim_cache size 0 max 16384 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.046+0000 7f4b45880740 1 Processor -- start 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.046+0000 7f4b45880740 1 -- start start 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b45880740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2bb60 0x55812bd270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b45880740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2c480 0x55812bd2e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b45880740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd35ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b45880740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b3bb0 con 0x55812bd2bb60 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b45880740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b1480 con 0x55812bd2c480 2024-09-15T22:47:28.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b45880740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1af2f0 con 0x55812bd36d40 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2bb60 0x55812bd270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2bb60 0x55812bd270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37564/0 (socket says 172.21.15.16:37564) 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 -- 172.21.15.16:0/302191977 learned_addr learned my addr 172.21.15.16:0/302191977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b40a42640 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2c480 0x55812bd2e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41a44640 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd35ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 -- 172.21.15.16:0/302191977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 msgr2=0x55812bd35ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd35ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 -- 172.21.15.16:0/302191977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2c480 msgr2=0x55812bd2e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2c480 0x55812bd2e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 -- 172.21.15.16:0/302191977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55812bd36540 con 0x55812bd2bb60 2024-09-15T22:47:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b41243640 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2bb60 0x55812bd270c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4b3802a930 tx=0x7f4b38031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ce51114f206afc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:28.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b26ffd640 1 -- 172.21.15.16:0/302191977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38035040 con 0x55812bd2bb60 2024-09-15T22:47:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.047+0000 7f4b26ffd640 1 -- 172.21.15.16:0/302191977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b38032a00 con 0x55812bd2bb60 2024-09-15T22:47:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b26ffd640 1 -- 172.21.15.16:0/302191977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38004920 con 0x55812bd2bb60 2024-09-15T22:47:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 -- 172.21.15.16:0/302191977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2bb60 msgr2=0x55812bd270c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2bb60 0x55812bd270c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4b3802a930 tx=0x7f4b38031400 comp rx=0 tx=0).stop 2024-09-15T22:47:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 -- 172.21.15.16:0/302191977 shutdown_connections 2024-09-15T22:47:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bd35ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2c480 0x55812bd2e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 --2- 172.21.15.16:0/302191977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2bb60 0x55812bd270c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 -- 172.21.15.16:0/302191977 >> 172.21.15.16:0/302191977 conn(0x55812bd280f0 msgr2=0x55812bd353c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:28.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 -- 172.21.15.16:0/302191977 shutdown_connections 2024-09-15T22:47:28.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.048+0000 7f4b45880740 1 -- 172.21.15.16:0/302191977 wait complete. 2024-09-15T22:47:28.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 -- start start 2024-09-15T22:47:28.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 Processor -- start 2024-09-15T22:47:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd4a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd4a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bdaa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1af940 con 0x55812bd2c480 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b0db0 con 0x55812bd2bb60 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3c1b0010 con 0x55812bd36d40 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b45880740 10 client.0 _ll_register_callbacks cb 0x7f4b38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd4a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b41243640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd4a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b41243640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd4a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:52628/0 (socket says 172.21.15.16:52628) 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b40a42640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd4a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37574/0 (socket says 172.21.15.16:37574) 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.049+0000 7f4b41243640 1 -- 172.21.15.16:0/789871252 learned_addr learned my addr 172.21.15.16:0/789871252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b41a44640 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bdaa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b41243640 1 -- 172.21.15.16:0/789871252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 msgr2=0x55812bdaa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b41243640 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bdaa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b41243640 1 -- 172.21.15.16:0/789871252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 msgr2=0x55812bd4a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:28.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b41243640 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd4a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b41243640 1 -- 172.21.15.16:0/789871252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812bdaa8c0 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b40a42640 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd4a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b41243640 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd4a370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009230 tx=0x7f4b38005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18bb0782db2242d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b24ff9640 1 -- 172.21.15.16:0/789871252 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38054070 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b38032a00 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b24ff9640 1 -- 172.21.15.16:0/789871252 <== mon.2 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f4b380458a0 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b24ff9640 1 -- 172.21.15.16:0/789871252 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b38060050 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b45880740 10 client.26660 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.050+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55812bdb1cc0 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b24ff9640 1 -- 172.21.15.16:0/789871252 <== mon.2 v2:172.21.15.59:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b38032a00 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b24ff9640 1 -- 172.21.15.16:0/789871252 <== mon.2 v2:172.21.15.59:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b3804e430 con 0x55812bd2bb60 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b45880740 10 client.26660 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:28.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b45880740 10 client.26660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b45880740 20 client.26660 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b45880740 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bdb43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b45880740 4 client.26660 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55812bc98d00 con 0x55812bd468a0 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b40a42640 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bdb43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b40a42640 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bdb43a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bba7fb3f3436ab8 server_cookie=dbdb0a7c23b22f7f in_seq=0 out_seq=0 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.051+0000 7f4b24ff9640 10 client.26660 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.052+0000 7f4b24ff9640 1 -- 172.21.15.16:0/789871252 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55812bc98d00 con 0x55812bd468a0 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.052+0000 7f4b24ff9640 10 client.26660 handle_command_reply: tid=0 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.052+0000 7f4b45880740 1 client.26660 shutdown 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 msgr2=0x55812bd4a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd4a370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4b38009230 tx=0x7f4b38005c00 comp rx=0 tx=0).stop 2024-09-15T22:47:28.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 shutdown_connections 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55812bd468a0 0x55812bdb43a0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55812bd36d40 0x55812bdaa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b24ff9640 0 client.26660 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55812bd2c480 0x55812bd4a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 --2- 172.21.15.16:0/789871252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55812bd2bb60 0x55812bd4a370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 >> 172.21.15.16:0/789871252 conn(0x55812bd280f0 msgr2=0x55812bd42500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 shutdown_connections 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 1 -- 172.21.15.16:0/789871252 wait complete. 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.053+0000 7f4b45880740 20 client.26660 trim_cache size 0 max 16384 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:47:28.061 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:47:28.062 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4d0bde3e-b66b-4ade-a7f4-c35a0f3fe209", 2024-09-15T22:47:28.062 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:47:28.062 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:47:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4b280a3430 msgr2=0x7f4b280a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4b280a3430 0x7f4b280a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c06a0f0 tx=0x7f4b34009a60 comp rx=0 tx=0).stop 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c101280 msgr2=0x7f4b3c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c101280 0x7f4b3c19abb0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c890 tx=0x7f4b2c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 shutdown_connections 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4b3c10e480 0x7f4b3c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b3c101280 0x7f4b3c19abb0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4b280a3430 0x7f4b280a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 --2- 172.21.15.16:0/2957771926 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4b3c1008d0 0x7f4b3c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:28.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 >> 172.21.15.16:0/2957771926 conn(0x7f4b3c0fc7a0 msgr2=0x7f4b3c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 shutdown_connections 2024-09-15T22:47:28.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:28.054+0000 7f4b434a7640 1 -- 172.21.15.16:0/2957771926 wait complete. 2024-09-15T22:47:28.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:47:28.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:28 smithi016 ceph-mon[31492]: pgmap v280: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:47:28.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:28 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:47:28.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:28 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:28.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:28 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:47:28.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:28 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:28.850 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:28 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:47:28.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:28 smithi059 ceph-mon[34872]: pgmap v280: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:47:28.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:28 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:47:28.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:28 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:28.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:28 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:47:28.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:28 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:28.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:28 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:47:28.985 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:29.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:28 smithi130 ceph-mon[35742]: pgmap v280: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:47:29.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:28 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:47:29.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:28 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:29.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:28 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:47:29.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:28 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:29.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:28 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 -- 172.21.15.16:0/411816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0102790 msgr2=0x7f38a01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 --2- 172.21.15.16:0/411816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0102790 0x7f38a01067e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3890009900 tx=0x7f389002a410 comp rx=0 tx=0).stop 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 -- 172.21.15.16:0/411816 shutdown_connections 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 --2- 172.21.15.16:0/411816 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a0106f10 0x7f38a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 --2- 172.21.15.16:0/411816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0102790 0x7f38a01067e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 --2- 172.21.15.16:0/411816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0101de0 0x7f38a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 -- 172.21.15.16:0/411816 >> 172.21.15.16:0/411816 conn(0x7f38a00fd8c0 msgr2=0x7f38a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 -- 172.21.15.16:0/411816 shutdown_connections 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.524+0000 7f38a72d7640 1 -- 172.21.15.16:0/411816 wait complete. 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 Processor -- start 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 -- start start 2024-09-15T22:47:29.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0101de0 0x7f38a0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0102790 0x7f38a01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a0106f10 0x7f38a019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0111f70 con 0x7f38a0101de0 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0111df0 con 0x7f38a0102790 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a72d7640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01120f0 con 0x7f38a0106f10 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a5073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0101de0 0x7f38a0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a5073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0101de0 0x7f38a0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37602/0 (socket says 172.21.15.16:37602) 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a5073640 1 -- 172.21.15.16:0/1300859167 learned_addr learned my addr 172.21.15.16:0/1300859167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.525+0000 7f38a4872640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0102790 0x7f38a01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5874640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a0106f10 0x7f38a019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5073640 1 -- 172.21.15.16:0/1300859167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a0106f10 msgr2=0x7f38a019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5073640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a0106f10 0x7f38a019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5073640 1 -- 172.21.15.16:0/1300859167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0102790 msgr2=0x7f38a01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5073640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0102790 0x7f38a01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5073640 1 -- 172.21.15.16:0/1300859167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a019d420 con 0x7f38a0101de0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5874640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a0106f10 0x7f38a019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a5073640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0101de0 0x7f38a0198470 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c890 tx=0x7f389c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38967fc640 1 -- 172.21.15.16:0/1300859167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c004220 con 0x7f38a0101de0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38967fc640 1 -- 172.21.15.16:0/1300859167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c0043c0 con 0x7f38a0101de0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a019d710 con 0x7f38a0101de0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38967fc640 1 -- 172.21.15.16:0/1300859167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0049e0 con 0x7f38a0101de0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.526+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01a4ff0 con 0x7f38a0101de0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38967fc640 1 -- 172.21.15.16:0/1300859167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f389c0079b0 con 0x7f38a0101de0 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 Processor -- start 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 -- start start 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746eb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642746512e0 con 0x5642746f6680 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5642745f8ce0 con 0x5642746ebdc0 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a96b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56427464d2a0 con 0x5642746eb4a0 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a4872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a4872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746f8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37606/0 (socket says 172.21.15.16:37606) 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a4872640 1 -- 172.21.15.16:0/631618422 learned_addr learned my addr 172.21.15.16:0/631618422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a5073640 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a5874640 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746eb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a4872640 1 -- 172.21.15.16:0/631618422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 msgr2=0x5642746eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.529+0000 7f38a4872640 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746eb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38967fc640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f388c0a33b0 0x7f388c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38a4872640 1 -- 172.21.15.16:0/631618422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 msgr2=0x5642746f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38a4872640 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38a4872640 1 -- 172.21.15.16:0/631618422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642745dd440 con 0x5642746f6680 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38a5073640 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38a5874640 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38a5874640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f388c0a33b0 0x7f388c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38a4872640 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746f8a70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3890009900 tx=0x7f38900316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16b6e7ec29bb0e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f38967fc640 1 -- 172.21.15.16:0/1300859167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f389c0d0130 con 0x7f38a0101de0 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f388b7fe640 1 -- 172.21.15.16:0/631618422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890040070 con 0x5642746f6680 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f388b7fe640 1 -- 172.21.15.16:0/631618422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3890005020 con 0x5642746f6680 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.530+0000 7f388b7fe640 1 -- 172.21.15.16:0/631618422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890035040 con 0x5642746f6680 2024-09-15T22:47:29.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 -- 172.21.15.16:0/631618422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 msgr2=0x5642746f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746f8a70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f3890009900 tx=0x7f38900316e0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a5874640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f388c0a33b0 0x7f388c0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3898009da0 tx=0x7f3898009320 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 -- 172.21.15.16:0/631618422 shutdown_connections 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746f8a70 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 --2- 172.21.15.16:0/631618422 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746eb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 -- 172.21.15.16:0/631618422 >> 172.21.15.16:0/631618422 conn(0x5642746e7a30 msgr2=0x5642746e7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 -- 172.21.15.16:0/631618422 shutdown_connections 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 -- 172.21.15.16:0/631618422 wait complete. 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 -- start start 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.531+0000 7f38a96b0740 1 Processor -- start 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746ebdc0 0x5642746f3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746f6680 0x5642746f3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a96b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642746f9250 con 0x5642746ebdc0 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a96b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56427460c050 con 0x5642746f6680 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a96b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5642746f93d0 con 0x5642746eb4a0 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a96b0740 10 client.0 _ll_register_callbacks cb 0x56427464c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746ebdc0 0x5642746f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746ebdc0 0x5642746f3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37618/0 (socket says 172.21.15.16:37618) 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 -- 172.21.15.16:0/3229865110 learned_addr learned my addr 172.21.15.16:0/3229865110 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a5073640 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a5874640 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746f6680 0x5642746f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 -- 172.21.15.16:0/3229865110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 msgr2=0x5642746f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 -- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746f6680 msgr2=0x5642746f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746f6680 0x5642746f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 -- 172.21.15.16:0/3229865110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642743c23c0 con 0x5642746ebdc0 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f38a4872640 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746ebdc0 0x5642746f3200 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3890004650 tx=0x7f3890009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651ae744fe20d8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.532+0000 7f3888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.533+0000 7f3888ff9640 1 -- 172.21.15.16:0/3229865110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389004c020 con 0x5642746ebdc0 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.533+0000 7f3888ff9640 1 -- 172.21.15.16:0/3229865110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f3890004af0 con 0x5642746ebdc0 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.533+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3890005020 con 0x5642746ebdc0 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.533+0000 7f3888ff9640 1 -- 172.21.15.16:0/3229865110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3890057050 con 0x5642746ebdc0 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.533+0000 7f38a96b0740 10 client.17418 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:29.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564274704b10 con 0x5642746ebdc0 2024-09-15T22:47:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f3888ff9640 1 -- 172.21.15.16:0/3229865110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3890005020 con 0x5642746ebdc0 2024-09-15T22:47:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f3888ff9640 1 -- 172.21.15.16:0/3229865110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3890044400 con 0x5642746ebdc0 2024-09-15T22:47:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a96b0740 10 client.17418 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a96b0740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a96b0740 20 client.17418 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:29.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a96b0740 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274705e60 0x564274708250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a96b0740 4 client.17418 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56427465e850 con 0x564274705e60 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.534+0000 7f38a5073640 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274705e60 0x564274708250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.535+0000 7f38a5073640 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274705e60 0x564274708250 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c5a5c8f0fbe46 server_cookie=406d2db1fdf2527d in_seq=0 out_seq=0 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.535+0000 7f3888ff9640 10 client.17418 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.535+0000 7f3888ff9640 1 -- 172.21.15.16:0/3229865110 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56427465e850 con 0x564274705e60 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.535+0000 7f3888ff9640 10 client.17418 handle_command_reply: tid=0 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.535+0000 7f38a96b0740 1 client.17418 shutdown 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746ebdc0 msgr2=0x5642746f3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746ebdc0 0x5642746f3200 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3890004650 tx=0x7f3890009670 comp rx=0 tx=0).stop 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 shutdown_connections 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274705e60 0x564274708250 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746f6680 0x5642746f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746ebdc0 0x5642746f3200 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 --2- 172.21.15.16:0/3229865110 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 >> 172.21.15.16:0/3229865110 conn(0x5642746e7a30 msgr2=0x5642746ed710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f3888ff9640 0 client.17418 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 shutdown_connections 2024-09-15T22:47:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 1 -- 172.21.15.16:0/3229865110 wait complete. 2024-09-15T22:47:29.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.536+0000 7f38a96b0740 20 client.17418 trim_cache size 0 max 16384 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 Processor -- start 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 -- start start 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746eb4a0 0x5642747024b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x56427470d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746f6680 0x56427470ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b19c0 con 0x5642746eb4a0 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01af290 con 0x5642746ebdc0 2024-09-15T22:47:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a96b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01ad100 con 0x5642746f6680 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.553+0000 7f38a5073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746eb4a0 0x5642747024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746eb4a0 0x5642747024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37632/0 (socket says 172.21.15.16:37632) 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 -- 172.21.15.16:0/1142446737 learned_addr learned my addr 172.21.15.16:0/1142446737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a4872640 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x56427470d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 -- 172.21.15.16:0/1142446737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746f6680 msgr2=0x56427470ddc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746f6680 0x56427470ddc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 -- 172.21.15.16:0/1142446737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 msgr2=0x56427470d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x56427470d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 -- 172.21.15.16:0/1142446737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642746e7860 con 0x5642746eb4a0 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a4872640 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x56427470d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f38a5073640 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746eb4a0 0x5642747024b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f389c005fd0 tx=0x7f389c0d2640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b391ee739536cd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f388affd640 1 -- 172.21.15.16:0/1142446737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c01d040 con 0x5642746eb4a0 2024-09-15T22:47:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f388affd640 1 -- 172.21.15.16:0/1142446737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c09c060 con 0x5642746eb4a0 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.554+0000 7f388affd640 1 -- 172.21.15.16:0/1142446737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0986d0 con 0x5642746eb4a0 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 -- 172.21.15.16:0/1142446737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746eb4a0 msgr2=0x5642747024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746eb4a0 0x5642747024b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f389c005fd0 tx=0x7f389c0d2640 comp rx=0 tx=0).stop 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 -- 172.21.15.16:0/1142446737 shutdown_connections 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746f6680 0x56427470ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x56427470d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1142446737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746eb4a0 0x5642747024b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 -- 172.21.15.16:0/1142446737 >> 172.21.15.16:0/1142446737 conn(0x5642746e7a30 msgr2=0x5642746f22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 -- 172.21.15.16:0/1142446737 shutdown_connections 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 -- 172.21.15.16:0/1142446737 wait complete. 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 -- start start 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.555+0000 7f38a96b0740 1 Processor -- start 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746feb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a96b0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a96b0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01ad750 con 0x5642746f6680 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a96b0740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01aebc0 con 0x5642746ebdc0 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a96b0740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01ade20 con 0x5642746eb4a0 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a96b0740 10 client.0 _ll_register_callbacks cb 0x7f38900315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746fa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37642/0 (socket says 172.21.15.16:37642) 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 -- 172.21.15.16:0/1754946845 learned_addr learned my addr 172.21.15.16:0/1754946845 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a4872640 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 -- 172.21.15.16:0/1754946845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 msgr2=0x5642746feb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746feb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 -- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 msgr2=0x5642746f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a5874640 1 -- 172.21.15.16:0/1754946845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642746fa9a0 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.556+0000 7f38a4872640 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a5874640 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746fa3e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f389800fe20 tx=0x7f3898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d14f2a160378923 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f3888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f3888ff9640 1 -- 172.21.15.16:0/1754946845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389801ce90 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f3888ff9640 1 -- 172.21.15.16:0/1754946845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f389801d430 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389c09c060 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f3888ff9640 1 -- 172.21.15.16:0/1754946845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f389801d630 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 10 client.17424 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5642747705e0 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f3888ff9640 1 -- 172.21.15.16:0/1754946845 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c09c060 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f3888ff9640 1 -- 172.21.15.16:0/1754946845 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3898026aa0 con 0x5642746f6680 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 10 client.17424 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 20 client.17424 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274770db0 0x5642747731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 4 client.17424 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.557+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564274658d60 con 0x564274770db0 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f38a5073640 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274770db0 0x5642747731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f38a5073640 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274770db0 0x5642747731a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12070d44d1d47917 server_cookie=633f2c82840b5345 in_seq=0 out_seq=0 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f3888ff9640 10 client.17424 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f3888ff9640 1 -- 172.21.15.16:0/1754946845 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564274658d60 con 0x564274770db0 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f3888ff9640 10 client.17424 handle_command_reply: tid=0 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f38a96b0740 1 client.17424 shutdown 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 msgr2=0x5642746fa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746fa3e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f389800fe20 tx=0x7f3898004a40 comp rx=0 tx=0).stop 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 shutdown_connections 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.558+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x564274770db0 0x5642747731a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642746f6680 0x5642746fa3e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5642746ebdc0 0x5642746f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f3888ff9640 0 client.17424 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a96b0740 1 --2- 172.21.15.16:0/1754946845 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5642746eb4a0 0x5642746feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 >> 172.21.15.16:0/1754946845 conn(0x5642746e7a30 msgr2=0x5642746f4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 shutdown_connections 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a96b0740 1 -- 172.21.15.16:0/1754946845 wait complete. 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a96b0740 20 client.17424 trim_cache size 0 max 16384 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:47:29.566 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:47:29.567 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:47:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f388c0a33b0 msgr2=0x7f388c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f388c0a33b0 0x7f388c0a5870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3898009da0 tx=0x7f3898009320 comp rx=0 tx=0).stop 2024-09-15T22:47:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0101de0 msgr2=0x7f38a0198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:29.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0101de0 0x7f38a0198470 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c890 tx=0x7f389c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 shutdown_connections 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f388c0a33b0 0x7f388c0a5870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f38a0106f10 0x7f38a019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38a0102790 0x7f38a01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 --2- 172.21.15.16:0/1300859167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a0101de0 0x7f38a0198470 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.559+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 >> 172.21.15.16:0/1300859167 conn(0x7f38a00fd8c0 msgr2=0x7f38a00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.560+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 shutdown_connections 2024-09-15T22:47:29.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:29.560+0000 7f38a72d7640 1 -- 172.21.15.16:0/1300859167 wait complete. 2024-09-15T22:47:30.277 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:47:30.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:47:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:30.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:30 smithi059 ceph-mon[34872]: pgmap v281: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:47:31.001 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:30 smithi016 ceph-mon[31492]: pgmap v281: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:47:31.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:30 smithi130 ceph-mon[35742]: pgmap v281: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:47:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 -- 172.21.15.16:0/4100524008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 msgr2=0x7f797c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 --2- 172.21.15.16:0/4100524008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c101e20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f7978009930 tx=0x7f797802a420 comp rx=0 tx=0).stop 2024-09-15T22:47:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 -- 172.21.15.16:0/4100524008 shutdown_connections 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 --2- 172.21.15.16:0/4100524008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f797c102360 0x7f797c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 --2- 172.21.15.16:0/4100524008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c101e20 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 --2- 172.21.15.16:0/4100524008 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f797c1079c0 0x7f797c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 -- 172.21.15.16:0/4100524008 >> 172.21.15.16:0/4100524008 conn(0x7f797c0fd740 msgr2=0x7f797c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 -- 172.21.15.16:0/4100524008 shutdown_connections 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.084+0000 7f798319d640 1 -- 172.21.15.16:0/4100524008 wait complete. 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 Processor -- start 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 -- start start 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f797c102360 0x7f797c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f797c1079c0 0x7f797c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f797c111dc0 con 0x7f797c1019c0 2024-09-15T22:47:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f797c111c40 con 0x7f797c1079c0 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798319d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f797c111f40 con 0x7f797c102360 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37658/0 (socket says 172.21.15.16:37658) 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 -- 172.21.15.16:0/4223011652 learned_addr learned my addr 172.21.15.16:0/4223011652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798299c640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f797c1079c0 0x7f797c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 -- 172.21.15.16:0/4223011652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f797c102360 msgr2=0x7f797c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f797c102360 0x7f797c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 -- 172.21.15.16:0/4223011652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f797c1079c0 msgr2=0x7f797c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f797c1079c0 0x7f797c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 -- 172.21.15.16:0/4223011652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c1aacf0 con 0x7f797c1019c0 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.085+0000 7f798219b640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c0744f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f796c00d770 tx=0x7f796c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.086+0000 7f79737fe640 1 -- 172.21.15.16:0/4223011652 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c017070 con 0x7f797c1019c0 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.086+0000 7f79737fe640 1 -- 172.21.15.16:0/4223011652 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f796c005500 con 0x7f797c1019c0 2024-09-15T22:47:31.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.086+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c1aaf20 con 0x7f797c1019c0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.086+0000 7f79737fe640 1 -- 172.21.15.16:0/4223011652 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c004e00 con 0x7f797c1019c0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.086+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c1ab380 con 0x7f797c1019c0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.087+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c103100 con 0x7f797c1019c0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.088+0000 7f79737fe640 1 -- 172.21.15.16:0/4223011652 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f796c004fa0 con 0x7f797c1019c0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.089+0000 7f79737fe640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f79680a33b0 0x7f79680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.089+0000 7f798199a640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f79680a33b0 0x7f79680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.089+0000 7f79737fe640 1 -- 172.21.15.16:0/4223011652 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f796c0cfef0 con 0x7f797c1019c0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.089+0000 7f798199a640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f79680a33b0 0x7f79680a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f797802a940 tx=0x7f7978000bc0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.092+0000 7f79737fe640 1 -- 172.21.15.16:0/4223011652 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f796c09c4c0 con 0x7f797c1019c0 2024-09-15T22:47:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.303+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f797c101e20 con 0x7f797c1019c0 2024-09-15T22:47:31.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.304+0000 7f79737fe640 1 -- 172.21.15.16:0/4223011652 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f797c101e20 con 0x7f797c1019c0 2024-09-15T22:47:31.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:47:31.306 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:47:31.306 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:47:31.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.306+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f79680a33b0 msgr2=0x7f79680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:31.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.306+0000 7f798319d640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f79680a33b0 0x7f79680a5870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f797802a940 tx=0x7f7978000bc0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 msgr2=0x7f797c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:31.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c0744f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f796c00d770 tx=0x7f796c00dc30 comp rx=0 tx=0).stop 2024-09-15T22:47:31.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 shutdown_connections 2024-09-15T22:47:31.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f797c1079c0 0x7f797c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f79680a33b0 0x7f79680a5870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f797c102360 0x7f797c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 --2- 172.21.15.16:0/4223011652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f797c1019c0 0x7f797c0744f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 >> 172.21.15.16:0/4223011652 conn(0x7f797c0fd740 msgr2=0x7f797c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 shutdown_connections 2024-09-15T22:47:31.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:31.307+0000 7f798319d640 1 -- 172.21.15.16:0/4223011652 wait complete. 2024-09-15T22:47:31.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:31 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4223011652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:31.964 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:31 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4223011652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:32.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:47:32.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:31 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4223011652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:32.263 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:32.718 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:32 smithi016 ceph-mon[31492]: pgmap v282: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 -- 172.21.15.16:0/2706821174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 msgr2=0x7f06341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 --2- 172.21.15.16:0/2706821174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f06341029b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f061c009900 tx=0x7f061c02a480 comp rx=0 tx=0).stop 2024-09-15T22:47:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 -- 172.21.15.16:0/2706821174 shutdown_connections 2024-09-15T22:47:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 --2- 172.21.15.16:0/2706821174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0634102ef0 0x7f063410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 --2- 172.21.15.16:0/2706821174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f06341029b0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 --2- 172.21.15.16:0/2706821174 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0634100e20 0x7f0634101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 -- 172.21.15.16:0/2706821174 >> 172.21.15.16:0/2706821174 conn(0x7f06340fc7d0 msgr2=0x7f06340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 -- 172.21.15.16:0/2706821174 shutdown_connections 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.801+0000 7f063aeca640 1 -- 172.21.15.16:0/2706821174 wait complete. 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 Processor -- start 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 -- start start 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0634100e20 0x7f06341984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f0634198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0634102ef0 0x7f063419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0634110f80 con 0x7f0634102570 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0634110e00 con 0x7f0634102ef0 2024-09-15T22:47:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f063aeca640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0634111100 con 0x7f0634100e20 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f0634198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f0634198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38110/0 (socket says 172.21.15.16:38110) 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 -- 172.21.15.16:0/4231677239 learned_addr learned my addr 172.21.15.16:0/4231677239 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0638c66640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0634100e20 0x7f06341984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0639467640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0634102ef0 0x7f063419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 -- 172.21.15.16:0/4231677239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0634100e20 msgr2=0x7f06341984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0634100e20 0x7f06341984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 -- 172.21.15.16:0/4231677239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0634102ef0 msgr2=0x7f063419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0634102ef0 0x7f063419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0633fff640 1 -- 172.21.15.16:0/4231677239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063419d480 con 0x7f0634102570 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0638c66640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0634100e20 0x7f06341984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.802+0000 7f0639467640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0634102ef0 0x7f063419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.803+0000 7f0633fff640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f0634198a10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f061c006010 tx=0x7f061c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.803+0000 7f0631ffb640 1 -- 172.21.15.16:0/4231677239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c03f070 con 0x7f0634102570 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.803+0000 7f0631ffb640 1 -- 172.21.15.16:0/4231677239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c031c30 con 0x7f0634102570 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.803+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063419d710 con 0x7f0634102570 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.803+0000 7f0631ffb640 1 -- 172.21.15.16:0/4231677239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c035030 con 0x7f0634102570 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.803+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341a5050 con 0x7f0634102570 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.805+0000 7f0631ffb640 1 -- 172.21.15.16:0/4231677239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f061c042070 con 0x7f0634102570 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.805+0000 7f063d2a3740 1 Processor -- start 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f063d2a3740 1 -- start start 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c6499090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c6499a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f063d2a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c7c63ff2f0 con 0x55c7c64a8780 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f063d2a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7c63a6cf0 con 0x55c7c649ab40 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f063d2a3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c7c63fb2b0 con 0x55c7c64995d0 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0631ffb640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06200a33b0 0x7f06200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0633fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0639467640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06200a33b0 0x7f06200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0633fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38120/0 (socket says 172.21.15.16:38120) 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0633fff640 1 -- 172.21.15.16:0/303336293 learned_addr learned my addr 172.21.15.16:0/303336293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0639467640 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c6499090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0638c66640 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c6499a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.806+0000 7f0631ffb640 1 -- 172.21.15.16:0/4231677239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f061c0f09e0 con 0x7f0634102570 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f0633fff640 1 -- 172.21.15.16:0/303336293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 msgr2=0x55c7c6499a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f0633fff640 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c6499a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f0633fff640 1 -- 172.21.15.16:0/303336293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 msgr2=0x55c7c6499090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f0633fff640 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c6499090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f0633fff640 1 -- 172.21.15.16:0/303336293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c638b3a0 con 0x55c7c64a8780 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f0639467640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06200a33b0 0x7f06200a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f062c009fd0 tx=0x7f062c009d60 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f0633fff640 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64aab70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f061c0bcf70 tx=0x7f061c03b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2747371acbc928bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f062affd640 1 -- 172.21.15.16:0/303336293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c049030 con 0x55c7c64a8780 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f062affd640 1 -- 172.21.15.16:0/303336293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c0bc910 con 0x55c7c64a8780 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.807+0000 7f062affd640 1 -- 172.21.15.16:0/303336293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0ee000 con 0x55c7c64a8780 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 -- 172.21.15.16:0/303336293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 msgr2=0x55c7c64aab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64aab70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f061c0bcf70 tx=0x7f061c03b870 comp rx=0 tx=0).stop 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 -- 172.21.15.16:0/303336293 shutdown_connections 2024-09-15T22:47:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64aab70 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c6499a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 --2- 172.21.15.16:0/303336293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c6499090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 -- 172.21.15.16:0/303336293 >> 172.21.15.16:0/303336293 conn(0x55c7c6462310 msgr2=0x55c7c6495c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 -- 172.21.15.16:0/303336293 shutdown_connections 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 -- 172.21.15.16:0/303336293 wait complete. 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 -- start start 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.808+0000 7f063d2a3740 1 Processor -- start 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c64a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c7c64ab240 con 0x55c7c64a8780 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c7c63ba060 con 0x55c7c649ab40 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c7c64ab3c0 con 0x55c7c64995d0 2024-09-15T22:47:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 10 client.0 _ll_register_callbacks cb 0x55c7c610d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64a5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38128/0 (socket says 172.21.15.16:38128) 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 -- 172.21.15.16:0/1347944128 learned_addr learned my addr 172.21.15.16:0/1347944128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0638c66640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c64a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0633fff640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 -- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 msgr2=0x55c7c64a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c64a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 -- 172.21.15.16:0/1347944128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 msgr2=0x55c7c64a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 -- 172.21.15.16:0/1347944128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c61703c0 con 0x55c7c64a8780 2024-09-15T22:47:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0633fff640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0638c66640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c64a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0639467640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64a5c60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f062c011d00 tx=0x7f062c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ab1d4a3f73ecdd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0613fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0613fff640 1 -- 172.21.15.16:0/1347944128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c024070 con 0x55c7c64a8780 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f0613fff640 1 -- 172.21.15.16:0/1347944128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f062c029660 con 0x55c7c64a8780 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.809+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7c640c860 con 0x55c7c64a8780 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f0613fff640 1 -- 172.21.15.16:0/1347944128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f062c029860 con 0x55c7c64a8780 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 10 client.17442 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7c64b6cd0 con 0x55c7c64a8780 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f0613fff640 1 -- 172.21.15.16:0/1347944128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c7c640c860 con 0x55c7c64a8780 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f0613fff640 1 -- 172.21.15.16:0/1347944128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f062c01d040 con 0x55c7c64a8780 2024-09-15T22:47:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 10 client.17442 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 10 client.17442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 20 client.17442 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c64b8190 0x55c7c64ba5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 4 client.17442 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7c6406d70 con 0x55c7c64b8190 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.810+0000 7f0638c66640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c64b8190 0x55c7c64ba5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.811+0000 7f0613fff640 10 client.17442 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.811+0000 7f0638c66640 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c64b8190 0x55c7c64ba5a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54a1f760586940ce server_cookie=ca677d99f7337c9f in_seq=0 out_seq=0 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f0613fff640 1 -- 172.21.15.16:0/1347944128 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7c6406d70 con 0x55c7c64b8190 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f0613fff640 10 client.17442 handle_command_reply: tid=0 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 client.17442 shutdown 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 msgr2=0x55c7c64a5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64a5c60 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f062c011d00 tx=0x7f062c005bd0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 shutdown_connections 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c64b8190 0x55c7c64ba5a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64a8780 0x55c7c64a5c60 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f0613fff640 0 client.17442 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 --2- 172.21.15.16:0/1347944128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64995d0 0x55c7c64a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 >> 172.21.15.16:0/1347944128 conn(0x55c7c6462310 msgr2=0x55c7c649b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 shutdown_connections 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 1 -- 172.21.15.16:0/1347944128 wait complete. 2024-09-15T22:47:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.812+0000 7f063d2a3740 20 client.17442 trim_cache size 0 max 16384 2024-09-15T22:47:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.825+0000 7f063d2a3740 1 Processor -- start 2024-09-15T22:47:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f063d2a3740 1 -- start start 2024-09-15T22:47:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c64995d0 0x55c7c649a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c649ab40 0x55c7c6490ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 0x55c7c64a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f063d2a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06341af610 con 0x55c7c649ab40 2024-09-15T22:47:32.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f063d2a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06341acee0 con 0x55c7c64995d0 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f063d2a3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f06341aad50 con 0x55c7c64a8780 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c649ab40 0x55c7c6490ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c649ab40 0x55c7c6490ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38142/0 (socket says 172.21.15.16:38142) 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 -- 172.21.15.16:0/577687824 learned_addr learned my addr 172.21.15.16:0/577687824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0638c66640 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c64995d0 0x55c7c649a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 -- 172.21.15.16:0/577687824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 msgr2=0x55c7c64a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 0x55c7c64a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 -- 172.21.15.16:0/577687824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c64995d0 msgr2=0x55c7c649a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c64995d0 0x55c7c649a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 -- 172.21.15.16:0/577687824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7c64b4410 con 0x55c7c649ab40 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0638c66640 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c64995d0 0x55c7c649a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.826+0000 7f0633fff640 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c649ab40 0x55c7c6490ab0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f061c0bcad0 tx=0x7f061c0bc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cf270216d7fac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:32.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f062a7fc640 1 -- 172.21.15.16:0/577687824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c049070 con 0x55c7c649ab40 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f062a7fc640 1 -- 172.21.15.16:0/577687824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f061c0c0cc0 con 0x55c7c649ab40 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f062a7fc640 1 -- 172.21.15.16:0/577687824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c03b460 con 0x55c7c649ab40 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 -- 172.21.15.16:0/577687824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c649ab40 msgr2=0x55c7c6490ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c649ab40 0x55c7c6490ab0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f061c0bcad0 tx=0x7f061c0bc370 comp rx=0 tx=0).stop 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 -- 172.21.15.16:0/577687824 shutdown_connections 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 0x55c7c64a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c649ab40 0x55c7c6490ab0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 --2- 172.21.15.16:0/577687824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c64995d0 0x55c7c649a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 -- 172.21.15.16:0/577687824 >> 172.21.15.16:0/577687824 conn(0x55c7c6462310 msgr2=0x55c7c64a3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 -- 172.21.15.16:0/577687824 shutdown_connections 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.827+0000 7f063d2a3740 1 -- 172.21.15.16:0/577687824 wait complete. 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 -- start start 2024-09-15T22:47:32.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 Processor -- start 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64995d0 0x55c7c64b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 0x55c7c64ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f06341ab3a0 con 0x55c7c64995d0 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f06341ac810 con 0x55c7c649ab40 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f06341aba70 con 0x55c7c64a8780 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.828+0000 7f063d2a3740 10 client.0 _ll_register_callbacks cb 0x7f062c0148b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64995d0 0x55c7c64b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64995d0 0x55c7c64b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38152/0 (socket says 172.21.15.16:38152) 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 -- 172.21.15.16:0/3491942271 learned_addr learned my addr 172.21.15.16:0/3491942271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0639467640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 0x55c7c64ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0633fff640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 -- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 msgr2=0x55c7c64ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 0x55c7c64ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 -- 172.21.15.16:0/3491942271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 msgr2=0x55c7c64abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 -- 172.21.15.16:0/3491942271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7c64acae0 con 0x55c7c64995d0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0633fff640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0638c66640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64995d0 0x55c7c64b0b80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x55c7c64bade0 tx=0x7f06240128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00a1c4a0f69d3bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0613fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0613fff640 1 -- 172.21.15.16:0/3491942271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062401b790 con 0x55c7c64995d0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0613fff640 1 -- 172.21.15.16:0/3491942271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f062401b930 con 0x55c7c64995d0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7c6383170 con 0x55c7c64995d0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f0613fff640 1 -- 172.21.15.16:0/3491942271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f062402c050 con 0x55c7c64995d0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f063d2a3740 10 client.17448 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.829+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c7c6522cb0 con 0x55c7c64995d0 2024-09-15T22:47:32.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f0613fff640 1 -- 172.21.15.16:0/3491942271 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c7c6383170 con 0x55c7c64995d0 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f0613fff640 1 -- 172.21.15.16:0/3491942271 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f062400f040 con 0x55c7c64995d0 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f063d2a3740 10 client.17448 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f063d2a3740 10 client.17448 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f063d2a3740 20 client.17448 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f063d2a3740 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c6523400 0x55c7c6525820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f063d2a3740 4 client.17448 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7c63d09b0 con 0x55c7c6523400 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.830+0000 7f0633fff640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c6523400 0x55c7c6525820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f0633fff640 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c6523400 0x55c7c6525820 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=818a71684a0f507b server_cookie=1fb95f4607570013 in_seq=0 out_seq=0 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f0613fff640 10 client.17448 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f0613fff640 1 -- 172.21.15.16:0/3491942271 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7c63d09b0 con 0x55c7c6523400 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f0613fff640 10 client.17448 handle_command_reply: tid=0 2024-09-15T22:47:32.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 client.17448 shutdown 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64995d0 msgr2=0x55c7c64b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64995d0 0x55c7c64b0b80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x55c7c64bade0 tx=0x7f06240128e0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 shutdown_connections 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55c7c6523400 0x55c7c6525820 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c7c64a8780 0x55c7c64ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c7c649ab40 0x55c7c64abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 --2- 172.21.15.16:0/3491942271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c7c64995d0 0x55c7c64b0b80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 >> 172.21.15.16:0/3491942271 conn(0x55c7c6462310 msgr2=0x55c7c64a45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f0613fff640 0 client.17448 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 shutdown_connections 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 1 -- 172.21.15.16:0/3491942271 wait complete. 2024-09-15T22:47:32.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.831+0000 7f063d2a3740 20 client.17448 trim_cache size 0 max 16384 2024-09-15T22:47:32.844 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06200a33b0 msgr2=0x7f06200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06200a33b0 0x7f06200a5870 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f062c009fd0 tx=0x7f062c009d60 comp rx=0 tx=0).stop 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 msgr2=0x7f0634198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f0634198a10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f061c006010 tx=0x7f061c003c10 comp rx=0 tx=0).stop 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 shutdown_connections 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0634102ef0 0x7f063419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0634102570 0x7f0634198a10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f06200a33b0 0x7f06200a5870 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 --2- 172.21.15.16:0/4231677239 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0634100e20 0x7f06341984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 >> 172.21.15.16:0/4231677239 conn(0x7f06340fc7d0 msgr2=0x7f063406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 shutdown_connections 2024-09-15T22:47:32.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:32.832+0000 7f063aeca640 1 -- 172.21.15.16:0/4231677239 wait complete. 2024-09-15T22:47:32.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:32 smithi059 ceph-mon[34872]: pgmap v282: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:33.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:32 smithi130 ceph-mon[35742]: pgmap v282: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:33.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:47:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 -- 172.21.15.16:0/3062359156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac100a80 msgr2=0x7ff8ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/3062359156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac100a80 0x7ff8ac100ee0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff894009930 tx=0x7ff89402a420 comp rx=0 tx=0).stop 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 -- 172.21.15.16:0/3062359156 shutdown_connections 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/3062359156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8ac101420 0x7ff8ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/3062359156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac100a80 0x7ff8ac100ee0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/3062359156 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8ac106a80 0x7ff8ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 -- 172.21.15.16:0/3062359156 >> 172.21.15.16:0/3062359156 conn(0x7ff8ac0fc7a0 msgr2=0x7ff8ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 -- 172.21.15.16:0/3062359156 shutdown_connections 2024-09-15T22:47:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.241+0000 7ff8b28fd640 1 -- 172.21.15.16:0/3062359156 wait complete. 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.242+0000 7ff8b28fd640 1 Processor -- start 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.242+0000 7ff8b28fd640 1 -- start start 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.242+0000 7ff8b28fd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8ac100a80 0x7ff8ac1a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.242+0000 7ff8b28fd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8ac101420 0x7ff8ac1a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b28fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac106a80 0x7ff8ac19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b28fd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac110e80 con 0x7ff8ac106a80 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8ac100a80 0x7ff8ac1a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b28fd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac110d00 con 0x7ff8ac100a80 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b28fd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac111000 con 0x7ff8ac101420 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8ac100a80 0x7ff8ac1a35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.16:60654/0 (socket says 172.21.15.16:60654) 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8abfff640 1 -- 172.21.15.16:0/885214639 learned_addr learned my addr 172.21.15.16:0/885214639 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:34.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b0e9a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac106a80 0x7ff8ac19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8ac101420 0x7ff8ac1a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b0e9a640 1 -- 172.21.15.16:0/885214639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8ac101420 msgr2=0x7ff8ac1a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b0e9a640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8ac101420 0x7ff8ac1a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b0e9a640 1 -- 172.21.15.16:0/885214639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8ac100a80 msgr2=0x7ff8ac1a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b0e9a640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8ac100a80 0x7ff8ac1a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8b0e9a640 1 -- 172.21.15.16:0/885214639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac19f3c0 con 0x7ff8ac106a80 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.243+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8ac101420 0x7ff8ac1a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.244+0000 7ff8b0e9a640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac106a80 0x7ff8ac19eb00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400cab0 tx=0x7ff8a400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.244+0000 7ff8a9ffb640 1 -- 172.21.15.16:0/885214639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400ae20 con 0x7ff8ac106a80 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.244+0000 7ff8a9ffb640 1 -- 172.21.15.16:0/885214639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a400f930 con 0x7ff8ac106a80 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.244+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac19f6b0 con 0x7ff8ac106a80 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.244+0000 7ff8a9ffb640 1 -- 172.21.15.16:0/885214639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4018b30 con 0x7ff8ac106a80 2024-09-15T22:47:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.244+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1ab6c0 con 0x7ff8ac106a80 2024-09-15T22:47:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.247+0000 7ff8a9ffb640 1 -- 172.21.15.16:0/885214639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff8a400fae0 con 0x7ff8ac106a80 2024-09-15T22:47:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.247+0000 7ff8b4cd6740 1 Processor -- start 2024-09-15T22:47:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.247+0000 7ff8b4cd6740 1 -- start start 2024-09-15T22:47:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.247+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945beb40 0x561a945bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945c8370 0x561a945ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a945232f0 con 0x561a945bd5d0 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a944cacf0 con 0x561a945c8370 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561a9451f2b0 con 0x561a945beb40 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945beb40 0x561a945bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945beb40 0x561a945bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:49592/0 (socket says 172.21.15.16:49592) 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8a3fff640 1 -- 172.21.15.16:0/1183219525 learned_addr learned my addr 172.21.15.16:0/1183219525 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945bda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38190/0 (socket says 172.21.15.16:38190) 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945c8370 0x561a945ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8a9ffb640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff8980a3430 0x7ff8980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 -- 172.21.15.16:0/1183219525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945beb40 msgr2=0x561a945bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945beb40 0x561a945bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 -- 172.21.15.16:0/1183219525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945c8370 msgr2=0x561a945ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945c8370 0x561a945ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 -- 172.21.15.16:0/1183219525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a944af3a0 con 0x561a945bd5d0 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8b0e9a640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff8980a3430 0x7ff8980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.248+0000 7ff8abfff640 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945bda10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00ac20 tx=0x7ff89c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3ea5a078dfd02d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.249+0000 7ff8a9ffb640 1 -- 172.21.15.16:0/885214639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7ff8a40d07c0 con 0x7ff8ac106a80 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.249+0000 7ff8a27fc640 1 -- 172.21.15.16:0/1183219525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c005220 con 0x561a945bd5d0 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.249+0000 7ff8a27fc640 1 -- 172.21.15.16:0/1183219525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89c0047a0 con 0x561a945bd5d0 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.249+0000 7ff8a27fc640 1 -- 172.21.15.16:0/1183219525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0187b0 con 0x561a945bd5d0 2024-09-15T22:47:34.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.249+0000 7ff8b0e9a640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff8980a3430 0x7ff8980a58f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4098b50 tx=0x7ff8a40674d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1183219525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 msgr2=0x561a945bda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945bda10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00ac20 tx=0x7ff89c00f790 comp rx=0 tx=0).stop 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1183219525 shutdown_connections 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945c8370 0x561a945ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945bda10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1183219525 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945beb40 0x561a945bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1183219525 >> 172.21.15.16:0/1183219525 conn(0x561a94586310 msgr2=0x561a945b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1183219525 shutdown_connections 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1183219525 wait complete. 2024-09-15T22:47:34.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 -- start start 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.250+0000 7ff8b4cd6740 1 Processor -- start 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945bd5d0 0x561a945c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 0x561a945c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945c8370 0x561a945c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a945cb070 con 0x561a945c8370 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a944de060 con 0x561a945beb40 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561a945cb1f0 con 0x561a945bd5d0 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 10 client.0 _ll_register_callbacks cb 0x561a942318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945c8370 0x561a945c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945c8370 0x561a945c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38192/0 (socket says 172.21.15.16:38192) 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 -- 172.21.15.16:0/3849336418 learned_addr learned my addr 172.21.15.16:0/3849336418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945bd5d0 0x561a945c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8abfff640 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 0x561a945c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 -- 172.21.15.16:0/3849336418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945bd5d0 msgr2=0x561a945c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945bd5d0 0x561a945c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 -- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 msgr2=0x561a945c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 0x561a945c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 -- 172.21.15.16:0/3849336418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a942943c0 con 0x561a945c8370 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945c8370 0x561a945c54d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff894009900 tx=0x7ff894032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e3b718488684b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff88b7fe640 1 -- 172.21.15.16:0/3849336418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894035040 con 0x561a945c8370 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff88b7fe640 1 -- 172.21.15.16:0/3849336418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7ff894005220 con 0x561a945c8370 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.251+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89c0047a0 con 0x561a945c8370 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff88b7fe640 1 -- 172.21.15.16:0/3849336418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff894052050 con 0x561a945c8370 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 10 client.17460 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a945d6ae0 con 0x561a945c8370 2024-09-15T22:47:34.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff88b7fe640 1 -- 172.21.15.16:0/3849336418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89c0047a0 con 0x561a945c8370 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff88b7fe640 1 -- 172.21.15.16:0/3849336418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff89403a040 con 0x561a945c8370 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 10 client.17460 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 10 client.17460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 20 client.17460 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a945d7ef0 0x561a945da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 4 client.17460 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a94530860 con 0x561a945d7ef0 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.252+0000 7ff8abfff640 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a945d7ef0 0x561a945da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.253+0000 7ff8abfff640 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a945d7ef0 0x561a945da320 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f398a705879e74a server_cookie=df3a39f3527fd05a in_seq=0 out_seq=0 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.253+0000 7ff88b7fe640 10 client.17460 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff88b7fe640 1 -- 172.21.15.16:0/3849336418 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a94530860 con 0x561a945d7ef0 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff88b7fe640 10 client.17460 handle_command_reply: tid=0 2024-09-15T22:47:34.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 client.17460 shutdown 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945c8370 msgr2=0x561a945c54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945c8370 0x561a945c54d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff894009900 tx=0x7ff894032c30 comp rx=0 tx=0).stop 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 shutdown_connections 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a945d7ef0 0x561a945da320 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945c8370 0x561a945c54d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 0x561a945c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff88b7fe640 0 client.17460 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/3849336418 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945bd5d0 0x561a945c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 >> 172.21.15.16:0/3849336418 conn(0x561a94586310 msgr2=0x561a945bf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 shutdown_connections 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/3849336418 wait complete. 2024-09-15T22:47:34.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.254+0000 7ff8b4cd6740 20 client.17460 trim_cache size 0 max 16384 2024-09-15T22:47:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 Processor -- start 2024-09-15T22:47:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 -- start start 2024-09-15T22:47:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 0x561a945c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945c0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac1b7e20 con 0x561a945bd5d0 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac1b56f0 con 0x561a945beb40 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac1b3560 con 0x561a945c8370 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38200/0 (socket says 172.21.15.16:38200) 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.267+0000 7ff8a3fff640 1 -- 172.21.15.16:0/580421768 learned_addr learned my addr 172.21.15.16:0/580421768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a3fff640 1 -- 172.21.15.16:0/580421768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 msgr2=0x561a945c0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/580421768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a3fff640 1 -- 172.21.15.16:0/580421768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 msgr2=0x561a945c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/580421768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 0x561a945c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a3fff640 1 -- 172.21.15.16:0/580421768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a945c3f40 con 0x561a945bd5d0 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/580421768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945c6ee0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff894005ab0 tx=0x7ff894032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b22f7f89d24d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a1ffb640 1 -- 172.21.15.16:0/580421768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894035070 con 0x561a945bd5d0 2024-09-15T22:47:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a1ffb640 1 -- 172.21.15.16:0/580421768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8940047a0 con 0x561a945bd5d0 2024-09-15T22:47:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8a1ffb640 1 -- 172.21.15.16:0/580421768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff894003800 con 0x561a945bd5d0 2024-09-15T22:47:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/580421768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 msgr2=0x561a945c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.268+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/580421768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945c6ee0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff894005ab0 tx=0x7ff894032db0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/580421768 shutdown_connections 2024-09-15T22:47:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/580421768 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/580421768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945beb40 0x561a945c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/580421768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945bd5d0 0x561a945c6ee0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/580421768 >> 172.21.15.16:0/580421768 conn(0x561a94586310 msgr2=0x561a945c67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/580421768 shutdown_connections 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/580421768 wait complete. 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- start start 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 Processor -- start 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945bd5d0 0x561a945d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945beb40 0x561a945d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac1b3bb0 con 0x561a945beb40 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac1b5020 con 0x561a945bd5d0 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac1b4280 con 0x561a945c8370 2024-09-15T22:47:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.269+0000 7ff8b4cd6740 10 client.0 _ll_register_callbacks cb 0x7ff894031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945beb40 0x561a945d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945beb40 0x561a945d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38210/0 (socket says 172.21.15.16:38210) 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945bd5d0 0x561a945d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 -- 172.21.15.16:0/1041570153 learned_addr learned my addr 172.21.15.16:0/1041570153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 -- 172.21.15.16:0/1041570153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 msgr2=0x561a945cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 -- 172.21.15.16:0/1041570153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945bd5d0 msgr2=0x561a945d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945bd5d0 0x561a945d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 -- 172.21.15.16:0/1041570153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a945cd9f0 con 0x561a945beb40 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8abfff640 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945beb40 0x561a945d0a20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff89c004b70 tx=0x7ff89c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d3c942b1388d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff88b7fe640 1 -- 172.21.15.16:0/1041570153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c004660 con 0x561a945beb40 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8940047a0 con 0x561a945beb40 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.270+0000 7ff88b7fe640 1 -- 172.21.15.16:0/1041570153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7ff89c01e040 con 0x561a945beb40 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff88b7fe640 1 -- 172.21.15.16:0/1041570153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff89c03b050 con 0x561a945beb40 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 10 client.17466 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a946432e0 con 0x561a945beb40 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff88b7fe640 1 -- 172.21.15.16:0/1041570153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8940047a0 con 0x561a945beb40 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff88b7fe640 1 -- 172.21.15.16:0/1041570153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff89c018460 con 0x561a945beb40 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 10 client.17466 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 10 client.17466 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 20 client.17466 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a946438a0 0x561a94645cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 4 client.17466 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a9452ad70 con 0x561a946438a0 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.271+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a946438a0 0x561a94645cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8a3fff640 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a946438a0 0x561a94645cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5cac1de4dd4b413 server_cookie=e09d97c8923d31fe in_seq=0 out_seq=0 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff88b7fe640 10 client.17466 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff88b7fe640 1 -- 172.21.15.16:0/1041570153 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a9452ad70 con 0x561a946438a0 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff88b7fe640 10 client.17466 handle_command_reply: tid=0 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 client.17466 shutdown 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945beb40 msgr2=0x561a945d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945beb40 0x561a945d0a20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7ff89c004b70 tx=0x7ff89c005f90 comp rx=0 tx=0).stop 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 shutdown_connections 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x561a946438a0 0x561a94645cc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a945c8370 0x561a945cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a945beb40 0x561a945d0a20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 --2- 172.21.15.16:0/1041570153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a945bd5d0 0x561a945d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 >> 172.21.15.16:0/1041570153 conn(0x561a94586310 msgr2=0x561a945c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff88b7fe640 0 client.17466 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 shutdown_connections 2024-09-15T22:47:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 1 -- 172.21.15.16:0/1041570153 wait complete. 2024-09-15T22:47:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.272+0000 7ff8b4cd6740 20 client.17466 trim_cache size 0 max 16384 2024-09-15T22:47:34.279 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff8980a3430 msgr2=0x7ff8980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff8980a3430 0x7ff8980a58f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4098b50 tx=0x7ff8a40674d0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac106a80 msgr2=0x7ff8ac19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac106a80 0x7ff8ac19eb00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400cab0 tx=0x7ff8a400cf70 comp rx=0 tx=0).stop 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 shutdown_connections 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ac106a80 0x7ff8ac19eb00 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff8980a3430 0x7ff8980a58f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff8ac101420 0x7ff8ac1a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 --2- 172.21.15.16:0/885214639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8ac100a80 0x7ff8ac1a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 >> 172.21.15.16:0/885214639 conn(0x7ff8ac0fc7a0 msgr2=0x7ff8ac06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 shutdown_connections 2024-09-15T22:47:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:34.273+0000 7ff8b28fd640 1 -- 172.21.15.16:0/885214639 wait complete. 2024-09-15T22:47:34.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:34 smithi059 ceph-mon[34872]: pgmap v283: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:34.998 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:34 smithi016 ceph-mon[31492]: pgmap v283: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:35.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:47:35.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:34 smithi130 ceph-mon[35742]: pgmap v283: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:35.282 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 -- 172.21.15.16:0/39352212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469010c2f0 msgr2=0x7f469010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 --2- 172.21.15.16:0/39352212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469010c2f0 0x7f469010e790 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4688009ca0 tx=0x7f468802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 -- 172.21.15.16:0/39352212 shutdown_connections 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 --2- 172.21.15.16:0/39352212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f469010c2f0 0x7f469010e790 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 --2- 172.21.15.16:0/39352212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46901037a0 0x7f469010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 --2- 172.21.15.16:0/39352212 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4690102df0 0x7f46901031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 -- 172.21.15.16:0/39352212 >> 172.21.15.16:0/39352212 conn(0x7f46900fc7d0 msgr2=0x7f46900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 -- 172.21.15.16:0/39352212 shutdown_connections 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 -- 172.21.15.16:0/39352212 wait complete. 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.768+0000 7f46952c4640 1 Processor -- start 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f46952c4640 1 -- start start 2024-09-15T22:47:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f46952c4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4690102df0 0x7f46901985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f46952c4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901037a0 0x7f4690198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f46952c4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f469010c2f0 0x7f469019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f46952c4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4690112110 con 0x7f4690102df0 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f46952c4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4690111f90 con 0x7f469010c2f0 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f46952c4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4690112290 con 0x7f46901037a0 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4690102df0 0x7f46901985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4690102df0 0x7f46901985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38222/0 (socket says 172.21.15.16:38222) 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 -- 172.21.15.16:0/105915286 learned_addr learned my addr 172.21.15.16:0/105915286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468e7fc640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901037a0 0x7f4690198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468f7fe640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f469010c2f0 0x7f469019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 -- 172.21.15.16:0/105915286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901037a0 msgr2=0x7f4690198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901037a0 0x7f4690198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 -- 172.21.15.16:0/105915286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f469010c2f0 msgr2=0x7f469019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f469010c2f0 0x7f469019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468effd640 1 -- 172.21.15.16:0/105915286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019d560 con 0x7f4690102df0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.769+0000 7f468e7fc640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901037a0 0x7f4690198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.770+0000 7f468effd640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4690102df0 0x7f46901985b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f468000e9f0 tx=0x7f468000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.770+0000 7f4677fff640 1 -- 172.21.15.16:0/105915286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468000cd80 con 0x7f4690102df0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.770+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469019d850 con 0x7f4690102df0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.770+0000 7f4677fff640 1 -- 172.21.15.16:0/105915286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680005500 con 0x7f4690102df0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.770+0000 7f4677fff640 1 -- 172.21.15.16:0/105915286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680010690 con 0x7f4690102df0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.770+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901a5130 con 0x7f4690102df0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.772+0000 7f4677fff640 1 -- 172.21.15.16:0/105915286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f46800046c0 con 0x7f4690102df0 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.772+0000 7f469769d740 1 Processor -- start 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.772+0000 7f469769d740 1 -- start start 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f469769d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9428090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f469769d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab94285d0 0x557ab9428a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f469769d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab9433370 0x557ab9435760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f469769d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ab938e2f0 con 0x557ab94285d0 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f469769d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557ab9335cf0 con 0x557ab9433370 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f469769d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557ab938a2b0 con 0x557ab9429b40 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab94285d0 0x557ab9428a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f4677fff640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a33b0 0x7f46780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab94285d0 0x557ab9428a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38238/0 (socket says 172.21.15.16:38238) 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 -- 172.21.15.16:0/2108368334 learned_addr learned my addr 172.21.15.16:0/2108368334 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468f7fe640 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9428090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468f7fe640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a33b0 0x7f46780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468e7fc640 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab9433370 0x557ab9435760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f4677fff640 1 -- 172.21.15.16:0/105915286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f46800d0690 con 0x7f4690102df0 2024-09-15T22:47:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 -- 172.21.15.16:0/2108368334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 msgr2=0x557ab9428090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9428090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 -- 172.21.15.16:0/2108368334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab9433370 msgr2=0x557ab9435760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab9433370 0x557ab9435760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 -- 172.21.15.16:0/2108368334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab931a3a0 con 0x557ab94285d0 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.773+0000 7f468effd640 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab94285d0 0x557ab9428a10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f46800a3bf0 tx=0x7f4680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e48d4e80614e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f468f7fe640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a33b0 0x7f46780a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f468802f540 tx=0x7f46880095c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f46757fa640 1 -- 172.21.15.16:0/2108368334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800a51d0 con 0x557ab94285d0 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f46757fa640 1 -- 172.21.15.16:0/2108368334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468009cc60 con 0x557ab94285d0 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f46757fa640 1 -- 172.21.15.16:0/2108368334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800d0220 con 0x557ab94285d0 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 -- 172.21.15.16:0/2108368334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab94285d0 msgr2=0x557ab9428a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab94285d0 0x557ab9428a10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f46800a3bf0 tx=0x7f4680004a40 comp rx=0 tx=0).stop 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 -- 172.21.15.16:0/2108368334 shutdown_connections 2024-09-15T22:47:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab9433370 0x557ab9435760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab94285d0 0x557ab9428a10 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 --2- 172.21.15.16:0/2108368334 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9428090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 -- 172.21.15.16:0/2108368334 >> 172.21.15.16:0/2108368334 conn(0x557ab93f1310 msgr2=0x557ab9424c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 -- 172.21.15.16:0/2108368334 shutdown_connections 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.774+0000 7f469769d740 1 -- 172.21.15.16:0/2108368334 wait complete. 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 -- start start 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 Processor -- start 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9432d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9429b40 0x557ab9430010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9433370 0x557ab9430600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ab9436010 con 0x557ab9429b40 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x557ab9349060 con 0x557ab94285d0 2024-09-15T22:47:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x557ab9436190 con 0x557ab9433370 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f469769d740 10 client.0 _ll_register_callbacks cb 0x557ab909c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f468e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9429b40 0x557ab9430010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f468e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9429b40 0x557ab9430010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38250/0 (socket says 172.21.15.16:38250) 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f468e7fc640 1 -- 172.21.15.16:0/3831633742 learned_addr learned my addr 172.21.15.16:0/3831633742 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f468effd640 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9432d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f468f7fe640 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9433370 0x557ab9430600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f468e7fc640 1 -- 172.21.15.16:0/3831633742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9433370 msgr2=0x557ab9430600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.775+0000 7f468e7fc640 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9433370 0x557ab9430600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f468e7fc640 1 -- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 msgr2=0x557ab9432d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f468e7fc640 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9432d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f468e7fc640 1 -- 172.21.15.16:0/3831633742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab90ff3c0 con 0x557ab9429b40 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f468e7fc640 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9429b40 0x557ab9430010 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4684012470 tx=0x7f4684012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f90ea0527a58ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f4672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f4672ff5640 1 -- 172.21.15.16:0/3831633742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468401f060 con 0x557ab9429b40 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f4672ff5640 1 -- 172.21.15.16:0/3831633742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f4684004460 con 0x557ab9429b40 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468009cc60 con 0x557ab9429b40 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f4672ff5640 1 -- 172.21.15.16:0/3831633742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4684026050 con 0x557ab9429b40 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f469769d740 10 client.17478 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ab9441b10 con 0x557ab9429b40 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f4672ff5640 1 -- 172.21.15.16:0/3831633742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468009cc60 con 0x557ab9429b40 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f4672ff5640 1 -- 172.21.15.16:0/3831633742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4684019030 con 0x557ab9429b40 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f469769d740 10 client.17478 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.776+0000 7f469769d740 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.777+0000 7f469769d740 20 client.17478 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.777+0000 7f469769d740 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab9443020 0x557ab9445440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.777+0000 7f469769d740 4 client.17478 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.777+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ab939b860 con 0x557ab9443020 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.777+0000 7f468effd640 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab9443020 0x557ab9445440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.777+0000 7f468effd640 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab9443020 0x557ab9445440 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7357a1df2f60380 server_cookie=5086e4754de783aa in_seq=0 out_seq=0 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.777+0000 7f4672ff5640 10 client.17478 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f4672ff5640 1 -- 172.21.15.16:0/3831633742 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ab939b860 con 0x557ab9443020 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f4672ff5640 10 client.17478 handle_command_reply: tid=0 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 client.17478 shutdown 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9429b40 msgr2=0x557ab9430010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9429b40 0x557ab9430010 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4684012470 tx=0x7f4684012930 comp rx=0 tx=0).stop 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 shutdown_connections 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab9443020 0x557ab9445440 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9433370 0x557ab9430600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9429b40 0x557ab9430010 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 --2- 172.21.15.16:0/3831633742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9432d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 >> 172.21.15.16:0/3831633742 conn(0x557ab93f1310 msgr2=0x557ab942a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f4672ff5640 0 client.17478 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 shutdown_connections 2024-09-15T22:47:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 1 -- 172.21.15.16:0/3831633742 wait complete. 2024-09-15T22:47:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.778+0000 7f469769d740 20 client.17478 trim_cache size 0 max 16384 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 Processor -- start 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 -- start start 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab942ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab943f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab944ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46901b19f0 con 0x557ab9433370 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46901af2c0 con 0x557ab94285d0 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.791+0000 7f469769d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46901ad130 con 0x557ab9429b40 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab944ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab944ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38260/0 (socket says 172.21.15.16:38260) 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 -- 172.21.15.16:0/1242955615 learned_addr learned my addr 172.21.15.16:0/1242955615 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468e7fc640 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab943f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468effd640 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab942ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 -- 172.21.15.16:0/1242955615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 msgr2=0x557ab943f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab943f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 -- 172.21.15.16:0/1242955615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 msgr2=0x557ab942ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab942ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 -- 172.21.15.16:0/1242955615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab942b120 con 0x557ab9433370 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f468f7fe640 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab944ad10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4688035000 tx=0x7f4688041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dda21b8c050d158 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f4674ff9640 1 -- 172.21.15.16:0/1242955615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688031450 con 0x557ab9433370 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f4674ff9640 1 -- 172.21.15.16:0/1242955615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4688036040 con 0x557ab9433370 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.792+0000 7f4674ff9640 1 -- 172.21.15.16:0/1242955615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468802f870 con 0x557ab9433370 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 -- 172.21.15.16:0/1242955615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 msgr2=0x557ab944ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab944ad10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4688035000 tx=0x7f4688041040 comp rx=0 tx=0).stop 2024-09-15T22:47:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 -- 172.21.15.16:0/1242955615 shutdown_connections 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab944ad10 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab943f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 --2- 172.21.15.16:0/1242955615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab942ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 -- 172.21.15.16:0/1242955615 >> 172.21.15.16:0/1242955615 conn(0x557ab93f1310 msgr2=0x557ab942f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 -- 172.21.15.16:0/1242955615 shutdown_connections 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 -- 172.21.15.16:0/1242955615 wait complete. 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 -- start start 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.793+0000 7f469769d740 1 Processor -- start 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f469769d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9436bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f469769d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9437100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f469769d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab943b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f469769d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46901ad780 con 0x557ab9433370 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f469769d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46901aebf0 con 0x557ab94285d0 2024-09-15T22:47:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f469769d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46901ade50 con 0x557ab9429b40 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f469769d740 10 client.0 _ll_register_callbacks cb 0x7f4684009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab943b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab943b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38264/0 (socket says 172.21.15.16:38264) 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 -- 172.21.15.16:0/2985318924 learned_addr learned my addr 172.21.15.16:0/2985318924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468effd640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9436bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468e7fc640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9437100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 -- 172.21.15.16:0/2985318924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 msgr2=0x557ab9437100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9437100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 -- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 msgr2=0x557ab9436bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9436bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 -- 172.21.15.16:0/2985318924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab943b8b0 con 0x557ab9433370 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468e7fc640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9437100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f468f7fe640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab943b240 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4688009ca0 tx=0x7f4688031d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a81f379695c5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f4672ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.794+0000 7f4672ff5640 1 -- 172.21.15.16:0/2985318924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468805c070 con 0x557ab9433370 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f4672ff5640 1 -- 172.21.15.16:0/2985318924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f4688031650 con 0x557ab9433370 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4688036040 con 0x557ab9433370 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f4672ff5640 1 -- 172.21.15.16:0/2985318924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4688068050 con 0x557ab9433370 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 10 client.17484 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ab94ad800 con 0x557ab9433370 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f4672ff5640 1 -- 172.21.15.16:0/2985318924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4688036040 con 0x557ab9433370 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f4672ff5640 1 -- 172.21.15.16:0/2985318924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f46880305c0 con 0x557ab9433370 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 10 client.17484 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:35.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 20 client.17484 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab94adef0 0x557ab94b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 4 client.17484 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ab9395d70 con 0x557ab94adef0 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.795+0000 7f468effd640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab94adef0 0x557ab94b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f4672ff5640 10 client.17484 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f468effd640 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab94adef0 0x557ab94b0310 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9d434312e4d352 server_cookie=821d2c398a128621 in_seq=0 out_seq=0 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f4672ff5640 1 -- 172.21.15.16:0/2985318924 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ab9395d70 con 0x557ab94adef0 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f4672ff5640 10 client.17484 handle_command_reply: tid=0 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 client.17484 shutdown 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 msgr2=0x557ab943b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab943b240 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4688009ca0 tx=0x7f4688031d90 comp rx=0 tx=0).stop 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 shutdown_connections 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x557ab94adef0 0x557ab94b0310 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ab9433370 0x557ab943b240 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f4672ff5640 0 client.17484 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x557ab9429b40 0x557ab9437100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 --2- 172.21.15.16:0/2985318924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557ab94285d0 0x557ab9436bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 >> 172.21.15.16:0/2985318924 conn(0x557ab93f1310 msgr2=0x557ab942e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 shutdown_connections 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 1 -- 172.21.15.16:0/2985318924 wait complete. 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.796+0000 7f469769d740 20 client.17484 trim_cache size 0 max 16384 2024-09-15T22:47:35.806 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a33b0 msgr2=0x7f46780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a33b0 0x7f46780a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f468802f540 tx=0x7f46880095c0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4690102df0 msgr2=0x7f46901985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4690102df0 0x7f46901985b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f468000e9f0 tx=0x7f468000edf0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 shutdown_connections 2024-09-15T22:47:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f469010c2f0 0x7f469019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a33b0 0x7f46780a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901037a0 0x7f4690198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 --2- 172.21.15.16:0/105915286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4690102df0 0x7f46901985b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 >> 172.21.15.16:0/105915286 conn(0x7f46900fc7d0 msgr2=0x7f46900fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 shutdown_connections 2024-09-15T22:47:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:35.797+0000 7f46952c4640 1 -- 172.21.15.16:0/105915286 wait complete. 2024-09-15T22:47:36.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:36 smithi016 ceph-mon[31492]: pgmap v284: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:36.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:36 smithi059 ceph-mon[34872]: pgmap v284: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:37.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:36 smithi130 ceph-mon[35742]: pgmap v284: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:37.470 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:47:37.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:47:37.743 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.202+0000 7f33f79af640 1 -- 172.21.15.16:0/1052608808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0101280 msgr2=0x7f33f010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.202+0000 7f33f79af640 1 --2- 172.21.15.16:0/1052608808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0101280 0x7f33f010df40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009900 tx=0x7f33e002a410 comp rx=0 tx=0).stop 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 -- 172.21.15.16:0/1052608808 shutdown_connections 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 --2- 172.21.15.16:0/1052608808 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33f010e480 0x7f33f0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 --2- 172.21.15.16:0/1052608808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f0101280 0x7f33f010df40 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 --2- 172.21.15.16:0/1052608808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33f01008d0 0x7f33f0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 -- 172.21.15.16:0/1052608808 >> 172.21.15.16:0/1052608808 conn(0x7f33f00fc7a0 msgr2=0x7f33f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 -- 172.21.15.16:0/1052608808 shutdown_connections 2024-09-15T22:47:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 -- 172.21.15.16:0/1052608808 wait complete. 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 Processor -- start 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 -- start start 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33f01008d0 0x7f33f019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.203+0000 7f33f79af640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33f0101280 0x7f33f019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f79af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f010e480 0x7f33f019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f79af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01141a0 con 0x7f33f010e480 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f79af640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0114020 con 0x7f33f0101280 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f79af640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0114320 con 0x7f33f01008d0 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f010e480 0x7f33f019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f010e480 0x7f33f019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38278/0 (socket says 172.21.15.16:38278) 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 -- 172.21.15.16:0/4205023481 learned_addr learned my addr 172.21.15.16:0/4205023481 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f574b640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33f01008d0 0x7f33f019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33f0101280 0x7f33f019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 -- 172.21.15.16:0/4205023481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33f01008d0 msgr2=0x7f33f019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33f01008d0 0x7f33f019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 -- 172.21.15.16:0/4205023481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33f0101280 msgr2=0x7f33f019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33f0101280 0x7f33f019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 -- 172.21.15.16:0/4205023481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f019f650 con 0x7f33f010e480 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33f0101280 0x7f33f019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f5f4c640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f010e480 0x7f33f019ef70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009fd0 tx=0x7f33e800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33e67fc640 1 -- 172.21.15.16:0/4205023481 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8004d20 con 0x7f33f010e480 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33e67fc640 1 -- 172.21.15.16:0/4205023481 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e8005020 con 0x7f33f010e480 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f019f940 con 0x7f33f010e480 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33e67fc640 1 -- 172.21.15.16:0/4205023481 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e80116d0 con 0x7f33f010e480 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.204+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f01a7220 con 0x7f33f010e480 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33e67fc640 1 -- 172.21.15.16:0/4205023481 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f33e80051d0 con 0x7f33f010e480 2024-09-15T22:47:38.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 Processor -- start 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 -- start start 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5baf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e40b520280 con 0x55e40b5bb480 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e40b4c7b10 con 0x55e40b5bab60 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f9d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e40b51c240 con 0x55e40b5c5d40 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38284/0 (socket says 172.21.15.16:38284) 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.207+0000 7f33f574b640 1 -- 172.21.15.16:0/2804697252 learned_addr learned my addr 172.21.15.16:0/2804697252 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33e67fc640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f33dc0a33b0 0x7f33dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f574b640 1 -- 172.21.15.16:0/2804697252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 msgr2=0x55e40b5c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f574b640 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f5f4c640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f33dc0a33b0 0x7f33dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f574b640 1 -- 172.21.15.16:0/2804697252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 msgr2=0x55e40b5baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f574b640 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f574b640 1 -- 172.21.15.16:0/2804697252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40b4d41f0 con 0x55e40b5bb480 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33f574b640 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c5800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00bcb0 tx=0x7f33ec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e83521d45180f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33e67fc640 1 -- 172.21.15.16:0/4205023481 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f33e80d0660 con 0x7f33f010e480 2024-09-15T22:47:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33db7fe640 1 -- 172.21.15.16:0/2804697252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec015070 con 0x55e40b5bb480 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33db7fe640 1 -- 172.21.15.16:0/2804697252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ec01a940 con 0x55e40b5bb480 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.208+0000 7f33db7fe640 1 -- 172.21.15.16:0/2804697252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec011aa0 con 0x55e40b5bb480 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 -- 172.21.15.16:0/2804697252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 msgr2=0x55e40b5c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f5f4c640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f33dc0a33b0 0x7f33dc0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f33e8098f60 tx=0x7f33e80c7360 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c5800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00bcb0 tx=0x7f33ec00f730 comp rx=0 tx=0).stop 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 -- 172.21.15.16:0/2804697252 shutdown_connections 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c5800 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2804697252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5baf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 -- 172.21.15.16:0/2804697252 >> 172.21.15.16:0/2804697252 conn(0x55e40b5b70f0 msgr2=0x55e40b5b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 -- 172.21.15.16:0/2804697252 shutdown_connections 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.209+0000 7f33f9d88740 1 -- 172.21.15.16:0/2804697252 wait complete. 2024-09-15T22:47:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 -- start start 2024-09-15T22:47:38.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 Processor -- start 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e40b5c8910 con 0x55e40b5bb480 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e40b4daf10 con 0x55e40b5bab60 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e40b5c8a90 con 0x55e40b5c5d40 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f9d88740 10 client.0 _ll_register_callbacks cb 0x55e40b4a3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38298/0 (socket says 172.21.15.16:38298) 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f574b640 1 -- 172.21.15.16:0/2530830511 learned_addr learned my addr 172.21.15.16:0/2530830511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.210+0000 7f33f574b640 1 -- 172.21.15.16:0/2530830511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 msgr2=0x55e40b5c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f574b640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f574b640 1 -- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 msgr2=0x55e40b5c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f574b640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f574b640 1 -- 172.21.15.16:0/2530830511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40b2916a0 con 0x55e40b5bb480 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f574b640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c3190 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f33ec000c00 tx=0x7f33ec00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb706e1456d7b098 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33d8ff9640 1 -- 172.21.15.16:0/2530830511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec015070 con 0x55e40b5bb480 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33d8ff9640 1 -- 172.21.15.16:0/2530830511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f33ec01a670 con 0x55e40b5bb480 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec01a940 con 0x55e40b5bb480 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33d8ff9640 1 -- 172.21.15.16:0/2530830511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f33ec01aaf0 con 0x55e40b5bb480 2024-09-15T22:47:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f9d88740 10 client.17496 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e40b5d45a0 con 0x55e40b5bb480 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33d8ff9640 1 -- 172.21.15.16:0/2530830511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ec01a940 con 0x55e40b5bb480 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.211+0000 7f33d8ff9640 1 -- 172.21.15.16:0/2530830511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f33ec011a80 con 0x55e40b5bb480 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f9d88740 10 client.17496 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f9d88740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f9d88740 20 client.17496 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b5d5a60 0x55e40b5d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f9d88740 4 client.17496 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e40b52d7f0 con 0x55e40b5d5a60 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b5d5a60 0x55e40b5d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33d8ff9640 10 client.17496 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.212+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b5d5a60 0x55e40b5d7e50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bf950faaaa6606 server_cookie=1e6f1b2ea7074975 in_seq=0 out_seq=0 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33d8ff9640 1 -- 172.21.15.16:0/2530830511 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e40b52d7f0 con 0x55e40b5d5a60 2024-09-15T22:47:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33d8ff9640 10 client.17496 handle_command_reply: tid=0 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 client.17496 shutdown 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 msgr2=0x55e40b5c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c3190 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f33ec000c00 tx=0x7f33ec00bfd0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 shutdown_connections 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b5d5a60 0x55e40b5d7e50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5c5d40 0x55e40b5c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c3190 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2530830511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5bab60 0x55e40b5c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33d8ff9640 0 client.17496 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 >> 172.21.15.16:0/2530830511 conn(0x55e40b5b70f0 msgr2=0x55e40b5b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.213+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 shutdown_connections 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.214+0000 7f33f9d88740 1 -- 172.21.15.16:0/2530830511 wait complete. 2024-09-15T22:47:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.214+0000 7f33f9d88740 20 client.17496 trim_cache size 0 max 16384 2024-09-15T22:47:38.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.231+0000 7f33f9d88740 1 Processor -- start 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.231+0000 7f33f9d88740 1 -- start start 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bab60 0x55e40b5bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bb480 0x55e40b5dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f9d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01b3c20 con 0x55e40b5bab60 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01b14f0 con 0x55e40b5c5d40 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f9d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01af360 con 0x55e40b5bb480 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bab60 0x55e40b5bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bab60 0x55e40b5bcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38308/0 (socket says 172.21.15.16:38308) 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 -- 172.21.15.16:0/2424771811 learned_addr learned my addr 172.21.15.16:0/2424771811 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:38.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f574b640 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bb480 0x55e40b5dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 -- 172.21.15.16:0/2424771811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bb480 msgr2=0x55e40b5dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bb480 0x55e40b5dd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 -- 172.21.15.16:0/2424771811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 msgr2=0x55e40b5ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ddb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 -- 172.21.15.16:0/2424771811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e40b5c5520 con 0x55e40b5bab60 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bab60 0x55e40b5bcd10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f33e002a450 tx=0x7f33e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b744181f5209e0c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.232+0000 7f33daffd640 1 -- 172.21.15.16:0/2424771811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0035040 con 0x55e40b5bab60 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33daffd640 1 -- 172.21.15.16:0/2424771811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e00035f0 con 0x55e40b5bab60 2024-09-15T22:47:38.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33daffd640 1 -- 172.21.15.16:0/2424771811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e004a520 con 0x55e40b5bab60 2024-09-15T22:47:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 -- 172.21.15.16:0/2424771811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bab60 msgr2=0x55e40b5bcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bab60 0x55e40b5bcd10 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f33e002a450 tx=0x7f33e0005fd0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 -- 172.21.15.16:0/2424771811 shutdown_connections 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bb480 0x55e40b5dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 --2- 172.21.15.16:0/2424771811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bab60 0x55e40b5bcd10 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 -- 172.21.15.16:0/2424771811 >> 172.21.15.16:0/2424771811 conn(0x55e40b5b70f0 msgr2=0x55e40b5bc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 -- 172.21.15.16:0/2424771811 shutdown_connections 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 -- 172.21.15.16:0/2424771811 wait complete. 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.233+0000 7f33f9d88740 1 -- start start 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 1 Processor -- start 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bab60 0x55e40b5c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01af9b0 con 0x55e40b5bb480 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01b0e20 con 0x55e40b5c5d40 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f33f01b0080 con 0x55e40b5bab60 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f9d88740 10 client.0 _ll_register_callbacks cb 0x7f33e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38310/0 (socket says 172.21.15.16:38310) 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 -- 172.21.15.16:0/1080336331 learned_addr learned my addr 172.21.15.16:0/1080336331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bab60 0x55e40b5c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 -- 172.21.15.16:0/1080336331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bab60 msgr2=0x55e40b5c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bab60 0x55e40b5c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 -- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 msgr2=0x55e40b5ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f574b640 1 -- 172.21.15.16:0/1080336331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e40b63f5c0 con 0x55e40b5bb480 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.234+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33f574b640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c9fa0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f33ec011ef0 tx=0x7f33ec011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d6772126f60f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33d8ff9640 1 -- 172.21.15.16:0/1080336331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec036020 con 0x55e40b5bb480 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33d8ff9640 1 -- 172.21.15.16:0/1080336331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f33ec011440 con 0x55e40b5bb480 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e00035f0 con 0x55e40b5bb480 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33d8ff9640 1 -- 172.21.15.16:0/1080336331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f33ec015040 con 0x55e40b5bb480 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33f9d88740 10 client.17502 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:38.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e40b6408c0 con 0x55e40b5bb480 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.235+0000 7f33d8ff9640 1 -- 172.21.15.16:0/1080336331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33e00035f0 con 0x55e40b5bb480 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33d8ff9640 1 -- 172.21.15.16:0/1080336331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f33ec032920 con 0x55e40b5bb480 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33f9d88740 10 client.17502 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33f9d88740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33f9d88740 20 client.17502 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33f9d88740 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b640ed0 0x55e40b6432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33f9d88740 4 client.17502 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e40b527d00 con 0x55e40b640ed0 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.236+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b640ed0 0x55e40b6432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.237+0000 7f33d8ff9640 10 client.17502 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:38.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.237+0000 7f33f4f4a640 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b640ed0 0x55e40b6432f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b6cc6d677638c6 server_cookie=8657598d88151d8 in_seq=0 out_seq=0 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33d8ff9640 1 -- 172.21.15.16:0/1080336331 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e40b527d00 con 0x55e40b640ed0 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33d8ff9640 10 client.17502 handle_command_reply: tid=0 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 client.17502 shutdown 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 msgr2=0x55e40b5c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c9fa0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f33ec011ef0 tx=0x7f33ec011cd0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 shutdown_connections 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e40b640ed0 0x55e40b6432f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e40b5c5d40 0x55e40b5ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e40b5bb480 0x55e40b5c9fa0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 --2- 172.21.15.16:0/1080336331 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e40b5bab60 0x55e40b5c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 >> 172.21.15.16:0/1080336331 conn(0x55e40b5b70f0 msgr2=0x55e40b5bcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33d8ff9640 0 client.17502 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 shutdown_connections 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.238+0000 7f33f9d88740 1 -- 172.21.15.16:0/1080336331 wait complete. 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f9d88740 20 client.17502 trim_cache size 0 max 16384 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "a245eaf9-bbd8-4fdb-b856-7d501acd603b", 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:47:38.246 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:47:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f33dc0a33b0 msgr2=0x7f33dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f33dc0a33b0 0x7f33dc0a5870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f33e8098f60 tx=0x7f33e80c7360 comp rx=0 tx=0).stop 2024-09-15T22:47:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f010e480 msgr2=0x7f33f019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f010e480 0x7f33f019ef70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009fd0 tx=0x7f33e800ee60 comp rx=0 tx=0).stop 2024-09-15T22:47:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 shutdown_connections 2024-09-15T22:47:38.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f010e480 0x7f33f019ef70 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33f0101280 0x7f33f019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f33dc0a33b0 0x7f33dc0a5870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 --2- 172.21.15.16:0/4205023481 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f33f01008d0 0x7f33f019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 >> 172.21.15.16:0/4205023481 conn(0x7f33f00fc7a0 msgr2=0x7f33f00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.239+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 shutdown_connections 2024-09-15T22:47:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:38.240+0000 7f33f79af640 1 -- 172.21.15.16:0/4205023481 wait complete. 2024-09-15T22:47:38.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:38 smithi016 ceph-mon[31492]: pgmap v285: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:38.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:38 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:47:38.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:38 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:38.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:38 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:47:38.878 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:38 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:38.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:38 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:47:38.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:38 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:47:38.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:38 smithi059 ceph-mon[34872]: pgmap v285: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:38 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:47:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:38 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:38 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:47:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:38 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:38 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:47:38.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:38 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:47:39.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:38 smithi130 ceph-mon[35742]: pgmap v285: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:38 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:47:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:38 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:38 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:47:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:38 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:38 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:47:39.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:38 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:47:39.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:47:39.329 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:39.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:39 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:47:39.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:39 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:47:39.745 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:39 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 -- 172.21.15.16:0/848027890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 msgr2=0x7f0450100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 --2- 172.21.15.16:0/848027890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f0450100ca0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0440009ca0 tx=0x7f044002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 -- 172.21.15.16:0/848027890 shutdown_connections 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 --2- 172.21.15.16:0/848027890 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0450105ec0 0x7f0450106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 --2- 172.21.15.16:0/848027890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04501011e0 0x7f0450105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 --2- 172.21.15.16:0/848027890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f0450100ca0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 -- 172.21.15.16:0/848027890 >> 172.21.15.16:0/848027890 conn(0x7f04500fc7d0 msgr2=0x7f04500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 -- 172.21.15.16:0/848027890 shutdown_connections 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 -- 172.21.15.16:0/848027890 wait complete. 2024-09-15T22:47:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 Processor -- start 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.828+0000 7f04579af640 1 -- start start 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f04579af640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f04501a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f04579af640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04501011e0 0x7f04501a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f04579af640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0450105ec0 0x7f045019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f04579af640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0450118630 con 0x7f04501008c0 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f04579af640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04501184b0 con 0x7f0450105ec0 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f04579af640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04501187b0 con 0x7f04501011e0 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f04501a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f04501a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38332/0 (socket says 172.21.15.16:38332) 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 -- 172.21.15.16:0/3168289333 learned_addr learned my addr 172.21.15.16:0/3168289333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f0454f4a640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04501011e0 0x7f04501a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f0455f4c640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0450105ec0 0x7f045019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 -- 172.21.15.16:0/3168289333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04501011e0 msgr2=0x7f04501a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04501011e0 0x7f04501a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 -- 172.21.15.16:0/3168289333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0450105ec0 msgr2=0x7f045019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0450105ec0 0x7f045019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 -- 172.21.15.16:0/3168289333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045019f070 con 0x7f04501008c0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f0454f4a640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04501011e0 0x7f04501a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.829+0000 7f045574b640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f04501a4780 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0440006010 tx=0x7f0440003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.830+0000 7f044e7fc640 1 -- 172.21.15.16:0/3168289333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440031650 con 0x7f04501008c0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.830+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045019f300 con 0x7f04501008c0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.830+0000 7f044e7fc640 1 -- 172.21.15.16:0/3168289333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04400317f0 con 0x7f04501008c0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.830+0000 7f044e7fc640 1 -- 172.21.15.16:0/3168289333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440030bb0 con 0x7f04501008c0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.830+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501ab650 con 0x7f04501008c0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.832+0000 7f044e7fc640 1 -- 172.21.15.16:0/3168289333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0440004690 con 0x7f04501008c0 2024-09-15T22:47:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.832+0000 7f0459d88740 1 Processor -- start 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0459d88740 1 -- start start 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0459d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0459d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0459d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7e8680 0x56386c7eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0459d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56386c7432e0 con 0x56386c7e8680 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0459d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56386c6eace0 con 0x56386c7dddc0 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0459d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56386c73f2a0 con 0x56386c7dd4a0 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f044e7fc640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f04380a33b0 0x7f04380a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0454f4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7e8680 0x56386c7eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0454f4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7e8680 0x56386c7eaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38342/0 (socket says 172.21.15.16:38342) 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0454f4a640 1 -- 172.21.15.16:0/811622997 learned_addr learned my addr 172.21.15.16:0/811622997 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0455f4c640 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f045574b640 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0455f4c640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f04380a33b0 0x7f04380a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f044e7fc640 1 -- 172.21.15.16:0/3168289333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f04400f1540 con 0x7f04501008c0 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0454f4a640 1 -- 172.21.15.16:0/811622997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 msgr2=0x56386c7dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0454f4a640 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7dd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.833+0000 7f0454f4a640 1 -- 172.21.15.16:0/811622997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 msgr2=0x56386c7e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f0454f4a640 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f0454f4a640 1 -- 172.21.15.16:0/811622997 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386c6cf440 con 0x56386c7e8680 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f045574b640 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f0454f4a640 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7e8680 0x56386c7eaa70 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f044400cc40 tx=0x7f044400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c7a8347ecacef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f0455f4c640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f04380a33b0 0x7f04380a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0448005fd0 tx=0x7f0448005850 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f04377fe640 1 -- 172.21.15.16:0/811622997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444014070 con 0x56386c7e8680 2024-09-15T22:47:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f04377fe640 1 -- 172.21.15.16:0/811622997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04440047a0 con 0x56386c7e8680 2024-09-15T22:47:39.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.834+0000 7f04377fe640 1 -- 172.21.15.16:0/811622997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444010720 con 0x56386c7e8680 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 -- 172.21.15.16:0/811622997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7e8680 msgr2=0x56386c7eaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7e8680 0x56386c7eaa70 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f044400cc40 tx=0x7f044400e720 comp rx=0 tx=0).stop 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 -- 172.21.15.16:0/811622997 shutdown_connections 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7e8680 0x56386c7eaa70 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 --2- 172.21.15.16:0/811622997 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 -- 172.21.15.16:0/811622997 >> 172.21.15.16:0/811622997 conn(0x56386c7d9a30 msgr2=0x56386c7d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 -- 172.21.15.16:0/811622997 shutdown_connections 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 -- 172.21.15.16:0/811622997 wait complete. 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 -- start start 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.835+0000 7f0459d88740 1 Processor -- start 2024-09-15T22:47:39.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dd4a0 0x56386c7e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7e8680 0x56386c7e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56386c7eb2a0 con 0x56386c7dd4a0 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56386c6fe050 con 0x56386c7dddc0 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56386c7eb420 con 0x56386c7e8680 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 10 client.0 _ll_register_callbacks cb 0x56386c73e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dd4a0 0x56386c7e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dd4a0 0x56386c7e7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38352/0 (socket says 172.21.15.16:38352) 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 -- 172.21.15.16:0/161928250 learned_addr learned my addr 172.21.15.16:0/161928250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0454f4a640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0455f4c640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7e8680 0x56386c7e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 -- 172.21.15.16:0/161928250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7e8680 msgr2=0x56386c7e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7e8680 0x56386c7e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 -- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 msgr2=0x56386c7e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 -- 172.21.15.16:0/161928250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386c4b43c0 con 0x56386c7dd4a0 2024-09-15T22:47:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0454f4a640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f045574b640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dd4a0 0x56386c7e7f70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f044002ade0 tx=0x7f044008c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8314e19d137ee062 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0434ff9640 1 -- 172.21.15.16:0/161928250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440088460 con 0x56386c7dd4a0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04440047a0 con 0x56386c7dd4a0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.836+0000 7f0434ff9640 1 -- 172.21.15.16:0/161928250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f0440030d50 con 0x56386c7dd4a0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0434ff9640 1 -- 172.21.15.16:0/161928250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04400bdd10 con 0x56386c7dd4a0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 10 client.17514 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56386c7eb2a0 con 0x56386c7dd4a0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0434ff9640 1 -- 172.21.15.16:0/161928250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04440047a0 con 0x56386c7dd4a0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0434ff9640 1 -- 172.21.15.16:0/161928250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f04400eeb60 con 0x56386c7dd4a0 2024-09-15T22:47:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 10 client.17514 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 20 client.17514 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7f8100 0x56386c7fa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 4 client.17514 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56386c750850 con 0x56386c7f8100 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.837+0000 7f0454f4a640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7f8100 0x56386c7fa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.838+0000 7f0434ff9640 10 client.17514 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.838+0000 7f0454f4a640 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7f8100 0x56386c7fa550 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f7516ee829af45 server_cookie=3fba9f46eb742431 in_seq=0 out_seq=0 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.838+0000 7f0434ff9640 1 -- 172.21.15.16:0/161928250 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56386c750850 con 0x56386c7f8100 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.838+0000 7f0434ff9640 10 client.17514 handle_command_reply: tid=0 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 client.17514 shutdown 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dd4a0 msgr2=0x56386c7e7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dd4a0 0x56386c7e7f70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f044002ade0 tx=0x7f044008c300 comp rx=0 tx=0).stop 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 shutdown_connections 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7f8100 0x56386c7fa550 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7e8680 0x56386c7e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dddc0 0x56386c7e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 --2- 172.21.15.16:0/161928250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dd4a0 0x56386c7e7f70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 >> 172.21.15.16:0/161928250 conn(0x56386c7d9a30 msgr2=0x56386c7df710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0434ff9640 0 client.17514 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 shutdown_connections 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 1 -- 172.21.15.16:0/161928250 wait complete. 2024-09-15T22:47:39.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.839+0000 7f0459d88740 20 client.17514 trim_cache size 0 max 16384 2024-09-15T22:47:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.856+0000 7f0459d88740 1 Processor -- start 2024-09-15T22:47:39.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.856+0000 7f0459d88740 1 -- start start 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0459d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7d9a30 0x56386c7f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0459d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dd4a0 0x56386c7e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0459d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dddc0 0x56386c7ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0459d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04501b7e30 con 0x56386c7d9a30 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0459d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04501b5700 con 0x56386c7dd4a0 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0459d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04501b3570 con 0x56386c7dddc0 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7d9a30 0x56386c7f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7d9a30 0x56386c7f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38354/0 (socket says 172.21.15.16:38354) 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 -- 172.21.15.16:0/4155924998 learned_addr learned my addr 172.21.15.16:0/4155924998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 -- 172.21.15.16:0/4155924998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dddc0 msgr2=0x56386c7ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0454f4a640 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dd4a0 0x56386c7e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dddc0 0x56386c7ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 -- 172.21.15.16:0/4155924998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dd4a0 msgr2=0x56386c7e07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dd4a0 0x56386c7e07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 -- 172.21.15.16:0/4155924998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386c7d9860 con 0x56386c7d9a30 2024-09-15T22:47:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f045574b640 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7d9a30 0x56386c7f4530 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f04400b9a70 tx=0x7f044002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e988c32c7f011f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0436ffd640 1 -- 172.21.15.16:0/4155924998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400098b0 con 0x56386c7d9a30 2024-09-15T22:47:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0436ffd640 1 -- 172.21.15.16:0/4155924998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0440036070 con 0x56386c7d9a30 2024-09-15T22:47:39.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.857+0000 7f0436ffd640 1 -- 172.21.15.16:0/4155924998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04400eead0 con 0x56386c7d9a30 2024-09-15T22:47:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 -- 172.21.15.16:0/4155924998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7d9a30 msgr2=0x56386c7f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7d9a30 0x56386c7f4530 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f04400b9a70 tx=0x7f044002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 -- 172.21.15.16:0/4155924998 shutdown_connections 2024-09-15T22:47:39.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dddc0 0x56386c7ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7dd4a0 0x56386c7e07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 --2- 172.21.15.16:0/4155924998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7d9a30 0x56386c7f4530 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 -- 172.21.15.16:0/4155924998 >> 172.21.15.16:0/4155924998 conn(0x56386c7e58c0 msgr2=0x56386c7e00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 -- 172.21.15.16:0/4155924998 shutdown_connections 2024-09-15T22:47:39.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 -- 172.21.15.16:0/4155924998 wait complete. 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.858+0000 7f0459d88740 1 -- start start 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 1 Processor -- start 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7d9a30 0x56386c7f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7ec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dddc0 0x56386c7eca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f04501b3bc0 con 0x56386c7dddc0 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f04501b5030 con 0x56386c7d9a30 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f04501b4290 con 0x56386c7dd4a0 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0459d88740 10 client.0 _ll_register_callbacks cb 0x7f0440005b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dddc0 0x56386c7eca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dddc0 0x56386c7eca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38366/0 (socket says 172.21.15.16:38366) 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 -- 172.21.15.16:0/3861396771 learned_addr learned my addr 172.21.15.16:0/3861396771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:39.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0454f4a640 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f045574b640 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7d9a30 0x56386c7f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 -- 172.21.15.16:0/3861396771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 msgr2=0x56386c7ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7ec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 -- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7d9a30 msgr2=0x56386c7f0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7d9a30 0x56386c7f0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.859+0000 7f0455f4c640 1 -- 172.21.15.16:0/3861396771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386c861dd0 con 0x56386c7dddc0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0455f4c640 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dddc0 0x56386c7eca00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0448011d70 tx=0x7f044800c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7739cfb60c01ef26 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0434ff9640 1 -- 172.21.15.16:0/3861396771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448025070 con 0x56386c7dddc0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0434ff9640 1 -- 172.21.15.16:0/3861396771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f044800f850 con 0x56386c7dddc0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0440036070 con 0x56386c7dddc0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0434ff9640 1 -- 172.21.15.16:0/3861396771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f044800fa50 con 0x56386c7dddc0 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0459d88740 10 client.17520 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:39.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56386c862f30 con 0x56386c7dddc0 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0434ff9640 1 -- 172.21.15.16:0/3861396771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0440036070 con 0x56386c7dddc0 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0434ff9640 1 -- 172.21.15.16:0/3861396771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f04480047a0 con 0x56386c7dddc0 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.860+0000 7f0459d88740 10 client.17520 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0459d88740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0459d88740 20 client.17520 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0459d88740 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7e8680 0x56386c865740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0459d88740 4 client.17520 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56386c74ad60 con 0x56386c7e8680 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f045574b640 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7e8680 0x56386c865740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f045574b640 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7e8680 0x56386c865740 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9348161ff62b27de server_cookie=9937d89b5dfe9828 in_seq=0 out_seq=0 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0434ff9640 10 client.17520 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0434ff9640 1 -- 172.21.15.16:0/3861396771 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56386c74ad60 con 0x56386c7e8680 2024-09-15T22:47:39.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0434ff9640 10 client.17520 handle_command_reply: tid=0 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.861+0000 7f0459d88740 1 client.17520 shutdown 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dddc0 msgr2=0x56386c7eca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dddc0 0x56386c7eca00 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0448011d70 tx=0x7f044800c690 comp rx=0 tx=0).stop 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 shutdown_connections 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56386c7e8680 0x56386c865740 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56386c7dddc0 0x56386c7eca00 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56386c7dd4a0 0x56386c7ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 --2- 172.21.15.16:0/3861396771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56386c7d9a30 0x56386c7f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0434ff9640 0 client.17520 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 >> 172.21.15.16:0/3861396771 conn(0x56386c7e58c0 msgr2=0x56386c7e36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 shutdown_connections 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 1 -- 172.21.15.16:0/3861396771 wait complete. 2024-09-15T22:47:39.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f0459d88740 20 client.17520 trim_cache size 0 max 16384 2024-09-15T22:47:39.870 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:47:39.870 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:47:39.870 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:47:39.870 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f04380a33b0 msgr2=0x7f04380a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f04579af640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f04380a33b0 0x7f04380a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0448005fd0 tx=0x7f0448005850 comp rx=0 tx=0).stop 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 msgr2=0x7f04501a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.862+0000 7f04579af640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f04501a4780 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0440006010 tx=0x7f0440003f10 comp rx=0 tx=0).stop 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 shutdown_connections 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0450105ec0 0x7f045019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f04380a33b0 0x7f04380a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f04501011e0 0x7f04501a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 --2- 172.21.15.16:0/3168289333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04501008c0 0x7f04501a4780 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 >> 172.21.15.16:0/3168289333 conn(0x7f04500fc7d0 msgr2=0x7f0450103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:39.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 shutdown_connections 2024-09-15T22:47:39.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:39.863+0000 7f04579af640 1 -- 172.21.15.16:0/3168289333 wait complete. 2024-09-15T22:47:39.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:39 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:47:39.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:39 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:47:39.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:39 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:47:40.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:39 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:47:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:39 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:47:40.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:39 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:47:40.505 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:47:40.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:47:40.766 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:40.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:40 smithi016 ceph-mon[31492]: pgmap v286: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:40.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:40 smithi059 ceph-mon[34872]: pgmap v286: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:41.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:40 smithi130 ceph-mon[35742]: pgmap v286: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:41.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 -- 172.21.15.16:0/321586185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98108300 msgr2=0x7fbd9810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/321586185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98108300 0x7fbd9810ec70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009cd0 tx=0x7fbd9002a8d0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 -- 172.21.15.16:0/321586185 shutdown_connections 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/321586185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98108300 0x7fbd9810ec70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/321586185 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd98103780 0x7fbd98107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/321586185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd98102dd0 0x7fbd981031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 -- 172.21.15.16:0/321586185 >> 172.21.15.16:0/321586185 conn(0x7fbd980fc7d0 msgr2=0x7fbd980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 -- 172.21.15.16:0/321586185 shutdown_connections 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.259+0000 7fbd9d20d640 1 -- 172.21.15.16:0/321586185 wait complete. 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 Processor -- start 2024-09-15T22:47:41.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 -- start start 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd98102dd0 0x7fbd98100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98103780 0x7fbd98100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd98108300 0x7fbd981014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbd981120d0 con 0x7fbd98103780 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98111f50 con 0x7fbd98102dd0 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd9d20d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98112250 con 0x7fbd98108300 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98103780 0x7fbd98100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:41.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd8ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98103780 0x7fbd98100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38382/0 (socket says 172.21.15.16:38382) 2024-09-15T22:47:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2378396924 learned_addr learned my addr 172.21.15.16:0/2378396924 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.260+0000 7fbd96d9d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd98102dd0 0x7fbd98100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd9759e640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd98108300 0x7fbd981014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2378396924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd98108300 msgr2=0x7fbd981014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd8ffff640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd98108300 0x7fbd981014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2378396924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd98102dd0 msgr2=0x7fbd98100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:41.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd8ffff640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd98102dd0 0x7fbd98100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd8ffff640 1 -- 172.21.15.16:0/2378396924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd981a6a50 con 0x7fbd98103780 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd96d9d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd98102dd0 0x7fbd98100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd8ffff640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98103780 0x7fbd98100f70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9c0 tx=0x7fbd8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd94d99640 1 -- 172.21.15.16:0/2378396924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000ccc0 con 0x7fbd98103780 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd981a6d40 con 0x7fbd98103780 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd94d99640 1 -- 172.21.15.16:0/2378396924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd80005500 con 0x7fbd98103780 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd94d99640 1 -- 172.21.15.16:0/2378396924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80010690 con 0x7fbd98103780 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.261+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd981a71d0 con 0x7fbd98103780 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.262+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd981031b0 con 0x7fbd98103780 2024-09-15T22:47:41.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.264+0000 7fbd94d99640 1 -- 172.21.15.16:0/2378396924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbd80004e80 con 0x7fbd98103780 2024-09-15T22:47:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.264+0000 7fbd94d99640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbd840a33b0 0x7fbd840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.264+0000 7fbd94d99640 1 -- 172.21.15.16:0/2378396924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fbd800d0740 con 0x7fbd98103780 2024-09-15T22:47:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.264+0000 7fbd96d9d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbd840a33b0 0x7fbd840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.265+0000 7fbd96d9d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbd840a33b0 0x7fbd840a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd880058c0 tx=0x7fbd88005ec0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:41.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.268+0000 7fbd94d99640 1 -- 172.21.15.16:0/2378396924 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd800541b0 con 0x7fbd98103780 2024-09-15T22:47:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.481+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd98101f60 con 0x7fbd98103780 2024-09-15T22:47:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.482+0000 7fbd94d99640 1 -- 172.21.15.16:0/2378396924 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fbd8009cd70 con 0x7fbd98103780 2024-09-15T22:47:41.484 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:47:41.485 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:47:41.485 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:47:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.484+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbd840a33b0 msgr2=0x7fbd840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.484+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbd840a33b0 0x7fbd840a5870 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd880058c0 tx=0x7fbd88005ec0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.484+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98103780 msgr2=0x7fbd98100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.484+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98103780 0x7fbd98100f70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000e9c0 tx=0x7fbd8000ee80 comp rx=0 tx=0).stop 2024-09-15T22:47:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 shutdown_connections 2024-09-15T22:47:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbd840a33b0 0x7fbd840a5870 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbd98108300 0x7fbd981014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd98103780 0x7fbd98100f70 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 --2- 172.21.15.16:0/2378396924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd98102dd0 0x7fbd98100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 >> 172.21.15.16:0/2378396924 conn(0x7fbd980fc7d0 msgr2=0x7fbd98105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 shutdown_connections 2024-09-15T22:47:41.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:41.485+0000 7fbd9d20d640 1 -- 172.21.15.16:0/2378396924 wait complete. 2024-09-15T22:47:41.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:41 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2378396924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:41.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:41 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2378396924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:42.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:41 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2378396924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:47:42.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:47:42.417 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:42.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:42 smithi016 ceph-mon[31492]: pgmap v287: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:47:42.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:42 smithi059 ceph-mon[34872]: pgmap v287: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:47:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 -- 172.21.15.16:0/496026531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 msgr2=0x7f299410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 --2- 172.21.15.16:0/496026531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f299410bb50 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f2988009900 tx=0x7f298802a480 comp rx=0 tx=0).stop 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 -- 172.21.15.16:0/496026531 shutdown_connections 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 --2- 172.21.15.16:0/496026531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f299410c170 0x7f299410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 --2- 172.21.15.16:0/496026531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f299410bb50 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 --2- 172.21.15.16:0/496026531 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2994102d40 0x7f2994103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 -- 172.21.15.16:0/496026531 >> 172.21.15.16:0/496026531 conn(0x7f29940fc7a0 msgr2=0x7f29940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 -- 172.21.15.16:0/496026531 shutdown_connections 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.928+0000 7f299a150640 1 -- 172.21.15.16:0/496026531 wait complete. 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 Processor -- start 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 -- start start 2024-09-15T22:47:42.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2994102d40 0x7f2994198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f29941989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f299410c170 0x7f299419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2994111f70 con 0x7f2994103660 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2994111df0 con 0x7f299410c170 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.929+0000 7f299a150640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29941120f0 con 0x7f2994102d40 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f29941989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f29941989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33246/0 (socket says 172.21.15.16:33246) 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 -- 172.21.15.16:0/3091216397 learned_addr learned my addr 172.21.15.16:0/3091216397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2993fff640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f299410c170 0x7f299419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f29937fe640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2994102d40 0x7f2994198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 -- 172.21.15.16:0/3091216397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2994102d40 msgr2=0x7f2994198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2994102d40 0x7f2994198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 -- 172.21.15.16:0/3091216397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f299410c170 msgr2=0x7f299419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f299410c170 0x7f299419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 -- 172.21.15.16:0/3091216397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299419d440 con 0x7f2994103660 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f29937fe640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2994102d40 0x7f2994198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2992ffd640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f29941989d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f29880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2990ff9640 1 -- 172.21.15.16:0/3091216397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298803f030 con 0x7f2994103660 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299419d6d0 con 0x7f2994103660 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2990ff9640 1 -- 172.21.15.16:0/3091216397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2988003ce0 con 0x7f2994103660 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f2990ff9640 1 -- 172.21.15.16:0/3091216397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988035070 con 0x7f2994103660 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.930+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29941a5010 con 0x7f2994103660 2024-09-15T22:47:42.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.933+0000 7f2990ff9640 1 -- 172.21.15.16:0/3091216397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2988042030 con 0x7f2994103660 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.933+0000 7f299c529740 1 Processor -- start 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.933+0000 7f299c529740 1 -- start start 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f299c529740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc1b40 0x55afc7fc0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f299c529740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc05d0 0x55afc7fc0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f299c529740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fd1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f299c529740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55afc7f262f0 con 0x55afc7fc05d0 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f299c529740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55afc7ecdcf0 con 0x55afc7fcf780 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f299c529740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55afc7f222b0 con 0x55afc7fc1b40 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc05d0 0x55afc7fc0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc05d0 0x55afc7fc0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33252/0 (socket says 172.21.15.16:33252) 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 -- 172.21.15.16:0/1771275429 learned_addr learned my addr 172.21.15.16:0/1771275429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:42.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f2993fff640 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc1b40 0x55afc7fc0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f2992ffd640 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fd1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 -- 172.21.15.16:0/1771275429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc1b40 msgr2=0x55afc7fc0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc1b40 0x55afc7fc0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 -- 172.21.15.16:0/1771275429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 msgr2=0x55afc7fd1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fd1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 -- 172.21.15.16:0/1771275429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc7eb23a0 con 0x55afc7fc05d0 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f2992ffd640 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fd1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f2990ff9640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f297c0a33b0 0x7f297c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.934+0000 7f29937fe640 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc05d0 0x55afc7fc0a10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f298400bcc0 tx=0x7f298400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5477666b83edfad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f2993fff640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f297c0a33b0 0x7f297c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f2990ff9640 1 -- 172.21.15.16:0/3091216397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f29880f06b0 con 0x7f2994103660 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f2979ffb640 1 -- 172.21.15.16:0/1771275429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984005020 con 0x55afc7fc05d0 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f2979ffb640 1 -- 172.21.15.16:0/1771275429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29840047a0 con 0x55afc7fc05d0 2024-09-15T22:47:42.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f2979ffb640 1 -- 172.21.15.16:0/1771275429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984011710 con 0x55afc7fc05d0 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f299c529740 1 -- 172.21.15.16:0/1771275429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc05d0 msgr2=0x55afc7fc0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f2993fff640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f297c0a33b0 0x7f297c0a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f298c005fd0 tx=0x7f298c0096a0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f299c529740 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc05d0 0x55afc7fc0a10 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f298400bcc0 tx=0x7f298400f730 comp rx=0 tx=0).stop 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f299c529740 1 -- 172.21.15.16:0/1771275429 shutdown_connections 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f299c529740 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fd1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f299c529740 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc05d0 0x55afc7fc0a10 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f299c529740 1 --2- 172.21.15.16:0/1771275429 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc1b40 0x55afc7fc0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.935+0000 7f299c529740 1 -- 172.21.15.16:0/1771275429 >> 172.21.15.16:0/1771275429 conn(0x55afc7f89310 msgr2=0x55afc7fbcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 -- 172.21.15.16:0/1771275429 shutdown_connections 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 -- 172.21.15.16:0/1771275429 wait complete. 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 -- start start 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 Processor -- start 2024-09-15T22:47:42.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc05d0 0x55afc7fcf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fcc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fcc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.936+0000 7f299c529740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55afc7fd2480 con 0x55afc7fc1b40 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f299c529740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55afc7ee1060 con 0x55afc7fcf780 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f299c529740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55afc7fd2600 con 0x55afc7fc05d0 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f299c529740 10 client.0 _ll_register_callbacks cb 0x55afc7c348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fcc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fcc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33262/0 (socket says 172.21.15.16:33262) 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 -- 172.21.15.16:0/2011018464 learned_addr learned my addr 172.21.15.16:0/2011018464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f29937fe640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc05d0 0x55afc7fcf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2993fff640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fcc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 -- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc05d0 msgr2=0x55afc7fcf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc05d0 0x55afc7fcf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 -- 172.21.15.16:0/2011018464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 msgr2=0x55afc7fcc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fcc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 -- 172.21.15.16:0/2011018464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc7c973c0 con 0x55afc7fc1b40 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f29937fe640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc05d0 0x55afc7fcf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2993fff640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fcc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2992ffd640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fcc3c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f29880b8b00 tx=0x7f29880095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5cba9cd04ea4ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2972ffd640 1 -- 172.21.15.16:0/2011018464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988049070 con 0x55afc7fc1b40 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2972ffd640 1 -- 172.21.15.16:0/2011018464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f29880b8d10 con 0x55afc7fc1b40 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29840047a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.937+0000 7f2972ffd640 1 -- 172.21.15.16:0/2011018464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f29880bc470 con 0x55afc7fc1b40 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 10 client.17538 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afc7fdde20 con 0x55afc7fc1b40 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f2972ffd640 1 -- 172.21.15.16:0/2011018464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29840047a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f2972ffd640 1 -- 172.21.15.16:0/2011018464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f29880f02a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 10 client.17538 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 10 client.17538 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 20 client.17538 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc7fdf170 0x55afc7fe1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 4 client.17538 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afc7f33860 con 0x55afc7fdf170 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.938+0000 7f29937fe640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc7fdf170 0x55afc7fe1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.939+0000 7f29937fe640 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc7fdf170 0x55afc7fe1560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a20a66542976ce3 server_cookie=c8bbf01732328a0a in_seq=0 out_seq=0 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.939+0000 7f2972ffd640 10 client.17538 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f2972ffd640 1 -- 172.21.15.16:0/2011018464 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afc7f33860 con 0x55afc7fdf170 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f2972ffd640 10 client.17538 handle_command_reply: tid=0 2024-09-15T22:47:42.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 client.17538 shutdown 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 msgr2=0x55afc7fcc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fcc3c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f29880b8b00 tx=0x7f29880095e0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 shutdown_connections 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc7fdf170 0x55afc7fe1560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fcf780 0x55afc7fcc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fcc3c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 --2- 172.21.15.16:0/2011018464 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fc05d0 0x55afc7fcf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 >> 172.21.15.16:0/2011018464 conn(0x55afc7f89310 msgr2=0x55afc7fc2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f2972ffd640 0 client.17538 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 shutdown_connections 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 1 -- 172.21.15.16:0/2011018464 wait complete. 2024-09-15T22:47:42.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.940+0000 7f299c529740 20 client.17538 trim_cache size 0 max 16384 2024-09-15T22:47:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 Processor -- start 2024-09-15T22:47:42.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 -- start start 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fbce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fdb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fcbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29941af710 con 0x55afc7fc1b40 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29941acfe0 con 0x55afc7fc05d0 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.953+0000 7f299c529740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29941aae50 con 0x55afc7fcf780 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fdb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fdb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33264/0 (socket says 172.21.15.16:33264) 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 -- 172.21.15.16:0/2499810259 learned_addr learned my addr 172.21.15.16:0/2499810259 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2993fff640 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fcbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f29937fe640 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fbce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 -- 172.21.15.16:0/2499810259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 msgr2=0x55afc7fcbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fcbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 -- 172.21.15.16:0/2499810259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 msgr2=0x55afc7fbce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fbce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 -- 172.21.15.16:0/2499810259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afc7fcb370 con 0x55afc7fc1b40 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f2992ffd640 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fdb720 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2988000c00 tx=0x7f298802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811a900e69671e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f29797fa640 1 -- 172.21.15.16:0/2499810259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988049070 con 0x55afc7fc1b40 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f29797fa640 1 -- 172.21.15.16:0/2499810259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29880bc6a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.954+0000 7f29797fa640 1 -- 172.21.15.16:0/2499810259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29880bc000 con 0x55afc7fc1b40 2024-09-15T22:47:42.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 -- 172.21.15.16:0/2499810259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 msgr2=0x55afc7fdb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fdb720 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2988000c00 tx=0x7f298802a480 comp rx=0 tx=0).stop 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 -- 172.21.15.16:0/2499810259 shutdown_connections 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fcbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fdb720 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 --2- 172.21.15.16:0/2499810259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fbce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 -- 172.21.15.16:0/2499810259 >> 172.21.15.16:0/2499810259 conn(0x55afc7f89310 msgr2=0x55afc7fbc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 -- 172.21.15.16:0/2499810259 shutdown_connections 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 -- 172.21.15.16:0/2499810259 wait complete. 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.955+0000 7f299c529740 1 -- start start 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 1 Processor -- start 2024-09-15T22:47:42.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fd7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fd32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fd3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f29941ab4a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29941ac910 con 0x55afc7fc05d0 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f29941abb70 con 0x55afc7fcf780 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f299c529740 10 client.0 _ll_register_callbacks cb 0x7f2988003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fd32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2992ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fd32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33270/0 (socket says 172.21.15.16:33270) 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2992ffd640 1 -- 172.21.15.16:0/4104546699 learned_addr learned my addr 172.21.15.16:0/4104546699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:42.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f29937fe640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fd7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2993fff640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fd3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2992ffd640 1 -- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 msgr2=0x55afc7fd3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2992ffd640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fd3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2992ffd640 1 -- 172.21.15.16:0/4104546699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 msgr2=0x55afc7fd7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.956+0000 7f2992ffd640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fd7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2992ffd640 1 -- 172.21.15.16:0/4104546699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afc7fd3fe0 con 0x55afc7fc1b40 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2993fff640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fd3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2992ffd640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fd32c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f29880b9070 tx=0x7f29880c5310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221d2e18cb0682df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2972ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2972ffd640 1 -- 172.21.15.16:0/4104546699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988075020 con 0x55afc7fc1b40 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2972ffd640 1 -- 172.21.15.16:0/4104546699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7f29880f02a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29880bc6a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2972ffd640 1 -- 172.21.15.16:0/4104546699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2988049070 con 0x55afc7fc1b40 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f299c529740 10 client.17544 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55afc7fd3fe0 con 0x55afc7fc1b40 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2972ffd640 1 -- 172.21.15.16:0/4104546699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29880bc6a0 con 0x55afc7fc1b40 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f2972ffd640 1 -- 172.21.15.16:0/4104546699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f29880bbaf0 con 0x55afc7fc1b40 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f299c529740 10 client.17544 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f299c529740 10 client.17544 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.957+0000 7f299c529740 20 client.17544 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f299c529740 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc804a090 0x55afc804c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f299c529740 4 client.17544 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afc7f2dd70 con 0x55afc804a090 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f29937fe640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc804a090 0x55afc804c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:42.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f2972ffd640 10 client.17544 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f29937fe640 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc804a090 0x55afc804c4b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb9038a124ae22d5 server_cookie=a852578f4a9e362e in_seq=0 out_seq=0 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f2972ffd640 1 -- 172.21.15.16:0/4104546699 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afc7f2dd70 con 0x55afc804a090 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f2972ffd640 10 client.17544 handle_command_reply: tid=0 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.958+0000 7f299c529740 1 client.17544 shutdown 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 msgr2=0x55afc7fd32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fd32c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f29880b9070 tx=0x7f29880c5310 comp rx=0 tx=0).stop 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 shutdown_connections 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55afc804a090 0x55afc804c4b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55afc7fcf780 0x55afc7fd3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55afc7fc1b40 0x55afc7fd32c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f2972ffd640 0 client.17544 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:47:42.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 --2- 172.21.15.16:0/4104546699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55afc7fc05d0 0x55afc7fd7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 >> 172.21.15.16:0/4104546699 conn(0x55afc7f89310 msgr2=0x55afc7fbc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 shutdown_connections 2024-09-15T22:47:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 1 -- 172.21.15.16:0/4104546699 wait complete. 2024-09-15T22:47:42.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299c529740 20 client.17544 trim_cache size 0 max 16384 2024-09-15T22:47:42.969 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f297c0a33b0 msgr2=0x7f297c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299a150640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f297c0a33b0 0x7f297c0a5870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f298c005fd0 tx=0x7f298c0096a0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 msgr2=0x7f29941989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.959+0000 7f299a150640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f29941989d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2988006010 tx=0x7f29880042f0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 shutdown_connections 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f299410c170 0x7f299419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2994103660 0x7f29941989d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f297c0a33b0 0x7f297c0a5870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 --2- 172.21.15.16:0/3091216397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2994102d40 0x7f2994198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 >> 172.21.15.16:0/3091216397 conn(0x7f29940fc7a0 msgr2=0x7f29940fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 shutdown_connections 2024-09-15T22:47:42.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:42.960+0000 7f299a150640 1 -- 172.21.15.16:0/3091216397 wait complete. 2024-09-15T22:47:43.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:42 smithi130 ceph-mon[35742]: pgmap v287: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:47:43.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:47:43.981 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:44.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 -- 172.21.15.16:0/1587581171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 msgr2=0x7fa15c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 --2- 172.21.15.16:0/1587581171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c1067e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009900 tx=0x7fa14c02a410 comp rx=0 tx=0).stop 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 -- 172.21.15.16:0/1587581171 shutdown_connections 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 --2- 172.21.15.16:0/1587581171 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa15c106f10 0x7fa15c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 --2- 172.21.15.16:0/1587581171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c1067e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 --2- 172.21.15.16:0/1587581171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa15c101de0 0x7fa15c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 -- 172.21.15.16:0/1587581171 >> 172.21.15.16:0/1587581171 conn(0x7fa15c0fd8c0 msgr2=0x7fa15c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 -- 172.21.15.16:0/1587581171 shutdown_connections 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.443+0000 7fa163410640 1 -- 172.21.15.16:0/1587581171 wait complete. 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 Processor -- start 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 -- start start 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa15c101de0 0x7fa15c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa15c106f10 0x7fa15c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c111fb0 con 0x7fa15c102790 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c111e30 con 0x7fa15c106f10 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa163410640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c112130 con 0x7fa15c101de0 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa1609ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa1609ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33294/0 (socket says 172.21.15.16:33294) 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.444+0000 7fa1609ab640 1 -- 172.21.15.16:0/1434883985 learned_addr learned my addr 172.21.15.16:0/1434883985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1619ad640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa15c106f10 0x7fa15c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1611ac640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa15c101de0 0x7fa15c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1609ab640 1 -- 172.21.15.16:0/1434883985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa15c101de0 msgr2=0x7fa15c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1609ab640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa15c101de0 0x7fa15c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1609ab640 1 -- 172.21.15.16:0/1434883985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa15c106f10 msgr2=0x7fa15c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1609ab640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa15c106f10 0x7fa15c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1609ab640 1 -- 172.21.15.16:0/1434883985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c19d4d0 con 0x7fa15c102790 2024-09-15T22:47:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1609ab640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c198a60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa14c006010 tx=0x7fa14c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1527fc640 1 -- 172.21.15.16:0/1434883985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c03f030 con 0x7fa15c102790 2024-09-15T22:47:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1527fc640 1 -- 172.21.15.16:0/1434883985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa14c003ce0 con 0x7fa15c102790 2024-09-15T22:47:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c19d760 con 0x7fa15c102790 2024-09-15T22:47:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa1527fc640 1 -- 172.21.15.16:0/1434883985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c035070 con 0x7fa15c102790 2024-09-15T22:47:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.445+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c074420 con 0x7fa15c102790 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.448+0000 7fa1527fc640 1 -- 172.21.15.16:0/1434883985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa14c042030 con 0x7fa15c102790 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.448+0000 7fa1657e9740 1 Processor -- start 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.448+0000 7fa1657e9740 1 -- start start 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53cbb40 0x55d3a53ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53ca5d0 0x55d3a53caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53d5370 0x55d3a53d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1657e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3a53302f0 con 0x55d3a53cbb40 2024-09-15T22:47:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1657e9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3a52d7cf0 con 0x55d3a53d5370 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1657e9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d3a532c2b0 con 0x55d3a53ca5d0 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53cbb40 0x55d3a53ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53cbb40 0x55d3a53ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33304/0 (socket says 172.21.15.16:33304) 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 -- 172.21.15.16:0/3658386524 learned_addr learned my addr 172.21.15.16:0/3658386524 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1527fc640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1480a77c0 0x7fa1480a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1611ac640 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53ca5d0 0x55d3a53caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1480a77c0 0x7fa1480a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1527fc640 1 -- 172.21.15.16:0/1434883985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fa14c0f0650 con 0x7fa15c102790 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1609ab640 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53d5370 0x55d3a53d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 -- 172.21.15.16:0/3658386524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53ca5d0 msgr2=0x55d3a53caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53ca5d0 0x55d3a53caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 -- 172.21.15.16:0/3658386524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53d5370 msgr2=0x55d3a53d7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53d5370 0x55d3a53d7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.449+0000 7fa1619ad640 1 -- 172.21.15.16:0/3658386524 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a52bc3a0 con 0x55d3a53cbb40 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.450+0000 7fa1619ad640 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53cbb40 0x55d3a53ca090 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa15400cb40 tx=0x7fa1540072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f87d9d981c09d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.450+0000 7fa1477fe640 1 -- 172.21.15.16:0/3658386524 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154015070 con 0x55d3a53cbb40 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.450+0000 7fa1477fe640 1 -- 172.21.15.16:0/3658386524 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa154016040 con 0x55d3a53cbb40 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.450+0000 7fa1477fe640 1 -- 172.21.15.16:0/3658386524 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15401e020 con 0x55d3a53cbb40 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.450+0000 7fa1619ad640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1480a77c0 0x7fa1480a9c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa15401f000 tx=0x7fa15401b000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.450+0000 7fa1657e9740 1 -- 172.21.15.16:0/3658386524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53cbb40 msgr2=0x55d3a53ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.450+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53cbb40 0x55d3a53ca090 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa15400cb40 tx=0x7fa1540072b0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 -- 172.21.15.16:0/3658386524 shutdown_connections 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53d5370 0x55d3a53d7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53ca5d0 0x55d3a53caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3658386524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53cbb40 0x55d3a53ca090 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 -- 172.21.15.16:0/3658386524 >> 172.21.15.16:0/3658386524 conn(0x55d3a5393310 msgr2=0x55d3a53c6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 -- 172.21.15.16:0/3658386524 shutdown_connections 2024-09-15T22:47:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 -- 172.21.15.16:0/3658386524 wait complete. 2024-09-15T22:47:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 -- start start 2024-09-15T22:47:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 Processor -- start 2024-09-15T22:47:44.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53ca5d0 0x55d3a53d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53cbb40 0x55d3a53d2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.451+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53d5370 0x55d3a53d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1657e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3a53d8070 con 0x55d3a53ca5d0 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1657e9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d3a52eb060 con 0x55d3a53cbb40 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1657e9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d3a53d81f0 con 0x55d3a53d5370 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53ca5d0 0x55d3a53d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1657e9740 10 client.0 _ll_register_callbacks cb 0x55d3a503e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53ca5d0 0x55d3a53d4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33316/0 (socket says 172.21.15.16:33316) 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 -- 172.21.15.16:0/2391703632 learned_addr learned my addr 172.21.15.16:0/2391703632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1609ab640 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53cbb40 0x55d3a53d2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1619ad640 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53d5370 0x55d3a53d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 -- 172.21.15.16:0/2391703632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53d5370 msgr2=0x55d3a53d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53d5370 0x55d3a53d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 -- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53cbb40 msgr2=0x55d3a53d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53cbb40 0x55d3a53d2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 -- 172.21.15.16:0/2391703632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a50a13c0 con 0x55d3a53ca5d0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1611ac640 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53ca5d0 0x55d3a53d4d80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa15800bee0 tx=0x7fa158012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37983f387a5cec28 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa144ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa144ff9640 1 -- 172.21.15.16:0/2391703632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158004230 con 0x55d3a53ca5d0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa144ff9640 1 -- 172.21.15.16:0/2391703632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fa1580043d0 con 0x55d3a53ca5d0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.452+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa154016040 con 0x55d3a53ca5d0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa144ff9640 1 -- 172.21.15.16:0/2391703632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa158024050 con 0x55d3a53ca5d0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 10 client.17556 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3a53e3ba0 con 0x55d3a53ca5d0 2024-09-15T22:47:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa144ff9640 1 -- 172.21.15.16:0/2391703632 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa154016040 con 0x55d3a53ca5d0 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa144ff9640 1 -- 172.21.15.16:0/2391703632 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa158003ab0 con 0x55d3a53ca5d0 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 10 client.17556 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 10 client.17556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 20 client.17556 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a53e5060 0x55d3a53e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 4 client.17556 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a533d860 con 0x55d3a53e5060 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.453+0000 7fa1609ab640 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a53e5060 0x55d3a53e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.454+0000 7fa144ff9640 10 client.17556 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.454+0000 7fa1609ab640 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a53e5060 0x55d3a53e7470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e45bd6d5017ed4f2 server_cookie=358e9817f30cf92c in_seq=0 out_seq=0 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa144ff9640 1 -- 172.21.15.16:0/2391703632 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3a533d860 con 0x55d3a53e5060 2024-09-15T22:47:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa144ff9640 10 client.17556 handle_command_reply: tid=0 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 client.17556 shutdown 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53ca5d0 msgr2=0x55d3a53d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53ca5d0 0x55d3a53d4d80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa15800bee0 tx=0x7fa158012820 comp rx=0 tx=0).stop 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 shutdown_connections 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a53e5060 0x55d3a53e7470 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53d5370 0x55d3a53d2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53cbb40 0x55d3a53d2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 --2- 172.21.15.16:0/2391703632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53ca5d0 0x55d3a53d4d80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 >> 172.21.15.16:0/2391703632 conn(0x55d3a5393310 msgr2=0x55d3a53cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa144ff9640 0 client.17556 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 shutdown_connections 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 1 -- 172.21.15.16:0/2391703632 wait complete. 2024-09-15T22:47:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.455+0000 7fa1657e9740 20 client.17556 trim_cache size 0 max 16384 2024-09-15T22:47:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 Processor -- start 2024-09-15T22:47:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 -- start start 2024-09-15T22:47:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a53c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a53e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1af750 con 0x55d3a53d5370 2024-09-15T22:47:44.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1ad020 con 0x55d3a53ca5d0 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1657e9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1aae90 con 0x55d3a53cbb40 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33332/0 (socket says 172.21.15.16:33332) 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 -- 172.21.15.16:0/4168653472 learned_addr learned my addr 172.21.15.16:0/4168653472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1611ac640 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a53c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1609ab640 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a53e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 -- 172.21.15.16:0/4168653472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 msgr2=0x55d3a53e1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a53e1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 -- 172.21.15.16:0/4168653472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 msgr2=0x55d3a53c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a53c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 -- 172.21.15.16:0/4168653472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a53ca2d0 con 0x55d3a53d5370 2024-09-15T22:47:44.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa1619ad640 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53cd120 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c3f0 tx=0x7fa15401aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fee3095ddb7923c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:44.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa146ffd640 1 -- 172.21.15.16:0/4168653472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154030070 con 0x55d3a53d5370 2024-09-15T22:47:44.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa146ffd640 1 -- 172.21.15.16:0/4168653472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa15400c4c0 con 0x55d3a53d5370 2024-09-15T22:47:44.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.469+0000 7fa146ffd640 1 -- 172.21.15.16:0/4168653472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15401e070 con 0x55d3a53d5370 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 -- 172.21.15.16:0/4168653472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 msgr2=0x55d3a53cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53cd120 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c3f0 tx=0x7fa15401aeb0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 -- 172.21.15.16:0/4168653472 shutdown_connections 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53cd120 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a53e1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 --2- 172.21.15.16:0/4168653472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a53c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 -- 172.21.15.16:0/4168653472 >> 172.21.15.16:0/4168653472 conn(0x55d3a5393310 msgr2=0x55d3a53c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 -- 172.21.15.16:0/4168653472 shutdown_connections 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 -- 172.21.15.16:0/4168653472 wait complete. 2024-09-15T22:47:44.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 -- start start 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.470+0000 7fa1657e9740 1 Processor -- start 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a5448000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a5448540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1657e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1657e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1ab4e0 con 0x55d3a53d5370 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1657e9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1ac950 con 0x55d3a53ca5d0 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1657e9740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1abbb0 con 0x55d3a53cbb40 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1657e9740 10 client.0 _ll_register_callbacks cb 0x7fa15800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53d8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33338/0 (socket says 172.21.15.16:33338) 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 -- 172.21.15.16:0/3785455463 learned_addr learned my addr 172.21.15.16:0/3785455463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1609ab640 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a5448540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1611ac640 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a5448000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 -- 172.21.15.16:0/3785455463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 msgr2=0x55d3a5448540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a5448540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 -- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 msgr2=0x55d3a5448000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a5448000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 -- 172.21.15.16:0/3785455463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a53c7960 con 0x55d3a53d5370 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa1619ad640 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53d8a10 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa154009760 tx=0x7fa154009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a82f2954bfe1391 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.471+0000 7fa144ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa144ff9640 1 -- 172.21.15.16:0/3785455463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154030030 con 0x55d3a53d5370 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa144ff9640 1 -- 172.21.15.16:0/3785455463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fa1540223b0 con 0x55d3a53d5370 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa15400c4c0 con 0x55d3a53d5370 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa144ff9640 1 -- 172.21.15.16:0/3785455463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa15403d050 con 0x55d3a53d5370 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 10 client.17562 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3a544f650 con 0x55d3a53d5370 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa144ff9640 1 -- 172.21.15.16:0/3785455463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa15400c4c0 con 0x55d3a53d5370 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa144ff9640 1 -- 172.21.15.16:0/3785455463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa15401aa10 con 0x55d3a53d5370 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 10 client.17562 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 10 client.17562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 20 client.17562 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a544fad0 0x55d3a5451ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:44.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 4 client.17562 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.472+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3a5337d70 con 0x55d3a544fad0 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1611ac640 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a544fad0 0x55d3a5451ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1611ac640 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a544fad0 0x55d3a5451ec0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=664da3e6cd52054f server_cookie=3e74a9dc40a5b29a in_seq=0 out_seq=0 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa144ff9640 10 client.17562 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa144ff9640 1 -- 172.21.15.16:0/3785455463 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3a5337d70 con 0x55d3a544fad0 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa144ff9640 10 client.17562 handle_command_reply: tid=0 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 client.17562 shutdown 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 msgr2=0x55d3a53d8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53d8a10 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa154009760 tx=0x7fa154009ee0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 shutdown_connections 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55d3a544fad0 0x55d3a5451ec0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d3a53d5370 0x55d3a53d8a10 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d3a53cbb40 0x55d3a5448540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa144ff9640 0 client.17562 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:47:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 --2- 172.21.15.16:0/3785455463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d3a53ca5d0 0x55d3a5448000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 >> 172.21.15.16:0/3785455463 conn(0x55d3a5393310 msgr2=0x55d3a53c5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.473+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 shutdown_connections 2024-09-15T22:47:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa1657e9740 1 -- 172.21.15.16:0/3785455463 wait complete. 2024-09-15T22:47:44.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa1657e9740 20 client.17562 trim_cache size 0 max 16384 2024-09-15T22:47:44.482 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1480a77c0 msgr2=0x7fa1480a9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1480a77c0 0x7fa1480a9c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa15401f000 tx=0x7fa15401b000 comp rx=0 tx=0).stop 2024-09-15T22:47:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 msgr2=0x7fa15c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:44.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c198a60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa14c006010 tx=0x7fa14c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 shutdown_connections 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa15c106f10 0x7fa15c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c102790 0x7fa15c198a60 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa1480a77c0 0x7fa1480a9c80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 --2- 172.21.15.16:0/1434883985 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa15c101de0 0x7fa15c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 >> 172.21.15.16:0/1434883985 conn(0x7fa15c0fd8c0 msgr2=0x7fa15c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 shutdown_connections 2024-09-15T22:47:44.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:44.474+0000 7fa163410640 1 -- 172.21.15.16:0/1434883985 wait complete. 2024-09-15T22:47:44.782 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:44 smithi016 ceph-mon[31492]: pgmap v288: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:44.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:44 smithi059 ceph-mon[34872]: pgmap v288: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:45.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:44 smithi130 ceph-mon[35742]: pgmap v288: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:45.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:47:45.472 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 -- 172.21.15.16:0/1718683633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c106f10 msgr2=0x7fdd0c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 --2- 172.21.15.16:0/1718683633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c106f10 0x7fdd0c10ea90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ca0 tx=0x7fdd0402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 -- 172.21.15.16:0/1718683633 shutdown_connections 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 --2- 172.21.15.16:0/1718683633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c106f10 0x7fdd0c10ea90 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 --2- 172.21.15.16:0/1718683633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd0c102790 0x7fdd0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 --2- 172.21.15.16:0/1718683633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c101de0 0x7fdd0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 -- 172.21.15.16:0/1718683633 >> 172.21.15.16:0/1718683633 conn(0x7fdd0c0fd8c0 msgr2=0x7fdd0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 -- 172.21.15.16:0/1718683633 shutdown_connections 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.954+0000 7fdd143ba640 1 -- 172.21.15.16:0/1718683633 wait complete. 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 Processor -- start 2024-09-15T22:47:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 -- start start 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c101de0 0x7fdd0c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd0c102790 0x7fdd0c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c106f10 0x7fdd0c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c112010 con 0x7fdd0c101de0 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c111e90 con 0x7fdd0c106f10 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd143ba640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c112190 con 0x7fdd0c102790 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c101de0 0x7fdd0c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c101de0 0x7fdd0c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33364/0 (socket says 172.21.15.16:33364) 2024-09-15T22:47:45.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 -- 172.21.15.16:0/157595330 learned_addr learned my addr 172.21.15.16:0/157595330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd11955640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd0c102790 0x7fdd0c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 -- 172.21.15.16:0/157595330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd0c102790 msgr2=0x7fdd0c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12957640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c106f10 0x7fdd0c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd0c102790 0x7fdd0c198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 -- 172.21.15.16:0/157595330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c106f10 msgr2=0x7fdd0c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c106f10 0x7fdd0c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12156640 1 -- 172.21.15.16:0/157595330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c19d5a0 con 0x7fdd0c101de0 2024-09-15T22:47:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd11955640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd0c102790 0x7fdd0c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.955+0000 7fdd12957640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c106f10 0x7fdd0c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:45.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.956+0000 7fdd12156640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c101de0 0x7fdd0c1985f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b4a0 tx=0x7fdd0800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.956+0000 7fdd037fe640 1 -- 172.21.15.16:0/157595330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08014070 con 0x7fdd0c101de0 2024-09-15T22:47:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.956+0000 7fdd037fe640 1 -- 172.21.15.16:0/157595330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd08010af0 con 0x7fdd0c101de0 2024-09-15T22:47:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.956+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c19d890 con 0x7fdd0c101de0 2024-09-15T22:47:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.956+0000 7fdd037fe640 1 -- 172.21.15.16:0/157595330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08017d10 con 0x7fdd0c101de0 2024-09-15T22:47:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.956+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c1a5170 con 0x7fdd0c101de0 2024-09-15T22:47:45.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.958+0000 7fdd037fe640 1 -- 172.21.15.16:0/157595330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdd08017070 con 0x7fdd0c101de0 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 Processor -- start 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 -- start start 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d5b40 0x563b861d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d45d0 0x563b861d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b8613a2f0 con 0x563b861d5b40 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b860e1cf0 con 0x563b861df370 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd16793740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563b861362b0 con 0x563b861d45d0 2024-09-15T22:47:45.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd12957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d5b40 0x563b861d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd037fe640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdcf80a33b0 0x7fdcf80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd12957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d5b40 0x563b861d4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33374/0 (socket says 172.21.15.16:33374) 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd12957640 1 -- 172.21.15.16:0/2113743538 learned_addr learned my addr 172.21.15.16:0/2113743538 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.959+0000 7fdd11955640 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd12156640 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d45d0 0x563b861d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd12957640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdcf80a33b0 0x7fdcf80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd037fe640 1 -- 172.21.15.16:0/157595330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fdd080cf440 con 0x7fdd0c101de0 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd11955640 1 -- 172.21.15.16:0/2113743538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d45d0 msgr2=0x563b861d4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:45.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd11955640 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d45d0 0x563b861d4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd11955640 1 -- 172.21.15.16:0/2113743538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d5b40 msgr2=0x563b861d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd11955640 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d5b40 0x563b861d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd11955640 1 -- 172.21.15.16:0/2113743538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b860c63a0 con 0x563b861df370 2024-09-15T22:47:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd12957640 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d5b40 0x563b861d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd11955640 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e1760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00a910 tx=0x7fdcfc00add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de43efae034cc6c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd12957640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdcf80a33b0 0x7fdcf80a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009bf0 tx=0x7fdd0402a8c0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:45.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd00dea640 1 -- 172.21.15.16:0/2113743538 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc015070 con 0x563b861df370 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd00dea640 1 -- 172.21.15.16:0/2113743538 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcfc00cc00 con 0x563b861df370 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.960+0000 7fdd00dea640 1 -- 172.21.15.16:0/2113743538 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0185c0 con 0x563b861df370 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 -- 172.21.15.16:0/2113743538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 msgr2=0x563b861e1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e1760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00a910 tx=0x7fdcfc00add0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 -- 172.21.15.16:0/2113743538 shutdown_connections 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e1760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d45d0 0x563b861d4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 --2- 172.21.15.16:0/2113743538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d5b40 0x563b861d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 -- 172.21.15.16:0/2113743538 >> 172.21.15.16:0/2113743538 conn(0x563b8619d310 msgr2=0x563b861d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 -- 172.21.15.16:0/2113743538 shutdown_connections 2024-09-15T22:47:45.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 -- 172.21.15.16:0/2113743538 wait complete. 2024-09-15T22:47:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.961+0000 7fdd16793740 1 -- start start 2024-09-15T22:47:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 1 Processor -- start 2024-09-15T22:47:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861d5b40 0x563b861dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861df370 0x563b861dc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563b861e2070 con 0x563b861d45d0 2024-09-15T22:47:45.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563b860f5060 con 0x563b861d5b40 2024-09-15T22:47:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x563b861e21f0 con 0x563b861df370 2024-09-15T22:47:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd16793740 10 client.0 _ll_register_callbacks cb 0x563b85e488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861decf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33386/0 (socket says 172.21.15.16:33386) 2024-09-15T22:47:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 -- 172.21.15.16:0/3868355702 learned_addr learned my addr 172.21.15.16:0/3868355702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd11955640 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861d5b40 0x563b861dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 -- 172.21.15.16:0/3868355702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861df370 msgr2=0x563b861dc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861df370 0x563b861dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 -- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861d5b40 msgr2=0x563b861dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861d5b40 0x563b861dbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 -- 172.21.15.16:0/3868355702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b85eab3c0 con 0x563b861d45d0 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdd12156640 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861decf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b470 tx=0x7fdd0800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bf7c56b1fddfd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.962+0000 7fdcf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/3868355702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080b4060 con 0x563b861d45d0 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/3868355702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fdd080cf170 con 0x563b861d45d0 2024-09-15T22:47:45.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcfc00cc00 con 0x563b861d45d0 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/3868355702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd0801e040 con 0x563b861d45d0 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 10 client.17574 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b861edb00 con 0x563b861d45d0 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/3868355702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcfc00cc00 con 0x563b861d45d0 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/3868355702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdd080cca60 con 0x563b861d45d0 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 10 client.17574 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 20 client.17574 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:45.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b861eefc0 0x563b861f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 4 client.17574 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.963+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b86147860 con 0x563b861eefc0 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.964+0000 7fdd11955640 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b861eefc0 0x563b861f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.964+0000 7fdd11955640 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b861eefc0 0x563b861f1430 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620e755abe3ce87c server_cookie=bd973d74b22d727c in_seq=0 out_seq=0 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.964+0000 7fdcf5ffb640 10 client.17574 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/3868355702 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b86147860 con 0x563b861eefc0 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdcf5ffb640 10 client.17574 handle_command_reply: tid=0 2024-09-15T22:47:45.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 client.17574 shutdown 2024-09-15T22:47:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 msgr2=0x563b861decf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861decf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b470 tx=0x7fdd0800c9e0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 shutdown_connections 2024-09-15T22:47:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b861eefc0 0x563b861f1430 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861df370 0x563b861dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdcf5ffb640 0 client.17574 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:45.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861d5b40 0x563b861dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 --2- 172.21.15.16:0/3868355702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861decf0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 >> 172.21.15.16:0/3868355702 conn(0x563b8619d310 msgr2=0x563b861d6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 shutdown_connections 2024-09-15T22:47:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 1 -- 172.21.15.16:0/3868355702 wait complete. 2024-09-15T22:47:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.965+0000 7fdd16793740 20 client.17574 trim_cache size 0 max 16384 2024-09-15T22:47:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.978+0000 7fdd16793740 1 Processor -- start 2024-09-15T22:47:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.978+0000 7fdd16793740 1 -- start start 2024-09-15T22:47:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.978+0000 7fdd16793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.978+0000 7fdd16793740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861eb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.978+0000 7fdd16793740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.978+0000 7fdd16793740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1b1ae0 con 0x563b861d45d0 2024-09-15T22:47:45.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd16793740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1af3b0 con 0x563b861df370 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd16793740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1ad220 con 0x563b861d5b40 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33390/0 (socket says 172.21.15.16:33390) 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 -- 172.21.15.16:0/1137868622 learned_addr learned my addr 172.21.15.16:0/1137868622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12957640 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd11955640 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 -- 172.21.15.16:0/1137868622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 msgr2=0x563b861eb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861eb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 -- 172.21.15.16:0/1137868622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 msgr2=0x563b861f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 -- 172.21.15.16:0/1137868622 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b861da4f0 con 0x563b861d45d0 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12957640 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdd12156640 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861d0e10 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fdd08097e50 tx=0x7fdd08099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6604b9f7e8895a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdcf7fff640 1 -- 172.21.15.16:0/1137868622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0809ade0 con 0x563b861d45d0 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdcf7fff640 1 -- 172.21.15.16:0/1137868622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0801ae50 con 0x563b861d45d0 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.979+0000 7fdcf7fff640 1 -- 172.21.15.16:0/1137868622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd080cf170 con 0x563b861d45d0 2024-09-15T22:47:45.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 -- 172.21.15.16:0/1137868622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 msgr2=0x563b861d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861d0e10 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fdd08097e50 tx=0x7fdd08099990 comp rx=0 tx=0).stop 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 -- 172.21.15.16:0/1137868622 shutdown_connections 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861eb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 --2- 172.21.15.16:0/1137868622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861d45d0 0x563b861d0e10 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 -- 172.21.15.16:0/1137868622 >> 172.21.15.16:0/1137868622 conn(0x563b8619d310 msgr2=0x563b861d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 -- 172.21.15.16:0/1137868622 shutdown_connections 2024-09-15T22:47:45.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.980+0000 7fdd16793740 1 -- 172.21.15.16:0/1137868622 wait complete. 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 -- start start 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 Processor -- start 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861e4450 0x563b861e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1ad870 con 0x563b861e4450 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1aece0 con 0x563b861df370 2024-09-15T22:47:45.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c1adf40 con 0x563b861d5b40 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd16793740 10 client.0 _ll_register_callbacks cb 0x7fdd08009330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd12957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861e4450 0x563b861e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd12156640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd12957640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861e4450 0x563b861e48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33406/0 (socket says 172.21.15.16:33406) 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd12957640 1 -- 172.21.15.16:0/2546647718 learned_addr learned my addr 172.21.15.16:0/2546647718 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.981+0000 7fdd11955640 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd12957640 1 -- 172.21.15.16:0/2546647718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 msgr2=0x563b861e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:46.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd12957640 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861e7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd12957640 1 -- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 msgr2=0x563b861e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd12957640 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd12957640 1 -- 172.21.15.16:0/2546647718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b861e4e10 con 0x563b861e4450 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd12156640 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd12957640 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861e4450 0x563b861e48d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009dc0 tx=0x7fdd04009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59abec42e5c79c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdcf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/2546647718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd04044070 con 0x563b861e4450 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/2546647718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fdd04004240 con 0x563b861e4450 2024-09-15T22:47:46.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd0801ae50 con 0x563b861e4450 2024-09-15T22:47:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/2546647718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdd04004b40 con 0x563b861e4450 2024-09-15T22:47:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd16793740 10 client.17580 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563b8625a800 con 0x563b861e4450 2024-09-15T22:47:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/2546647718 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0801ae50 con 0x563b861e4450 2024-09-15T22:47:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/2546647718 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdd04047070 con 0x563b861e4450 2024-09-15T22:47:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd16793740 10 client.17580 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:46.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.982+0000 7fdd16793740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdd16793740 20 client.17580 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdd16793740 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b8625af30 0x563b8625d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdd16793740 4 client.17580 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b86141d70 con 0x563b8625af30 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdd12156640 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b8625af30 0x563b8625d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdd12156640 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b8625af30 0x563b8625d350 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12461bfaa8959a4e server_cookie=3340e3ea05635783 in_seq=0 out_seq=0 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdcf5ffb640 10 client.17580 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:46.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdcf5ffb640 1 -- 172.21.15.16:0/2546647718 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b86141d70 con 0x563b8625af30 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.983+0000 7fdcf5ffb640 10 client.17580 handle_command_reply: tid=0 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 client.17580 shutdown 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861e4450 msgr2=0x563b861e48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861e4450 0x563b861e48d0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009dc0 tx=0x7fdd04009880 comp rx=0 tx=0).stop 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 shutdown_connections 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x563b8625af30 0x563b8625d350 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563b861e4450 0x563b861e48d0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563b861df370 0x563b861e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 --2- 172.21.15.16:0/2546647718 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x563b861d5b40 0x563b861e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdcf5ffb640 0 client.17580 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 >> 172.21.15.16:0/2546647718 conn(0x563b8619d310 msgr2=0x563b861cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 shutdown_connections 2024-09-15T22:47:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 1 -- 172.21.15.16:0/2546647718 wait complete. 2024-09-15T22:47:46.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd16793740 20 client.17580 trim_cache size 0 max 16384 2024-09-15T22:47:46.006 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:47:46.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdcf80a33b0 msgr2=0x7fdcf80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdcf80a33b0 0x7fdcf80a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009bf0 tx=0x7fdd0402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c101de0 msgr2=0x7fdd0c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c101de0 0x7fdd0c1985f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800b4a0 tx=0x7fdd0800b960 comp rx=0 tx=0).stop 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 shutdown_connections 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd0c106f10 0x7fdd0c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fdcf80a33b0 0x7fdcf80a5870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fdd0c102790 0x7fdd0c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.984+0000 7fdd143ba640 1 --2- 172.21.15.16:0/157595330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0c101de0 0x7fdd0c1985f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:46.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.985+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 >> 172.21.15.16:0/157595330 conn(0x7fdd0c0fd8c0 msgr2=0x7fdd0c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:46.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.985+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 shutdown_connections 2024-09-15T22:47:46.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:45.985+0000 7fdd143ba640 1 -- 172.21.15.16:0/157595330 wait complete. 2024-09-15T22:47:46.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:46 smithi059 ceph-mon[34872]: pgmap v289: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:47.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:46 smithi130 ceph-mon[35742]: pgmap v289: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:47.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:46 smithi016 ceph-mon[31492]: pgmap v289: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:47.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:47:47.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:47:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1481982315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688100e20 msgr2=0x7fd688101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1481982315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688100e20 0x7fd688101200 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009900 tx=0x7fd67c02a480 comp rx=0 tx=0).stop 2024-09-15T22:47:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1481982315 shutdown_connections 2024-09-15T22:47:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1481982315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd688102ef0 0x7fd68810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1481982315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd688102570 0x7fd6881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1481982315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688100e20 0x7fd688101200 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1481982315 >> 172.21.15.16:0/1481982315 conn(0x7fd6880fc7d0 msgr2=0x7fd6880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1481982315 shutdown_connections 2024-09-15T22:47:48.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.391+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1481982315 wait complete. 2024-09-15T22:47:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 Processor -- start 2024-09-15T22:47:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 -- start start 2024-09-15T22:47:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd688100e20 0x7fd6881973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688102570 0x7fd6881978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd688102ef0 0x7fd68819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd688110f80 con 0x7fd688102570 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd688110e00 con 0x7fd688102ef0 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68eeb3640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd688111100 con 0x7fd688100e20 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688102570 0x7fd6881978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68cc4f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd688100e20 0x7fd6881973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd687fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688102570 0x7fd6881978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33418/0 (socket says 172.21.15.16:33418) 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd687fff640 1 -- 172.21.15.16:0/1841061144 learned_addr learned my addr 172.21.15.16:0/1841061144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.392+0000 7fd68d450640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd688102ef0 0x7fd68819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd687fff640 1 -- 172.21.15.16:0/1841061144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd688100e20 msgr2=0x7fd6881973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd687fff640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd688100e20 0x7fd6881973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd687fff640 1 -- 172.21.15.16:0/1841061144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd688102ef0 msgr2=0x7fd68819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd687fff640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd688102ef0 0x7fd68819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd687fff640 1 -- 172.21.15.16:0/1841061144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68819c360 con 0x7fd688102570 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd688100e20 0x7fd6881973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd68d450640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd688102ef0 0x7fd68819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd687fff640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688102570 0x7fd6881978f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd674009970 tx=0x7fd674009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd685ffb640 1 -- 172.21.15.16:0/1841061144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67400d020 con 0x7fd688102570 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd685ffb640 1 -- 172.21.15.16:0/1841061144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd674007d40 con 0x7fd688102570 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68819c650 con 0x7fd688102570 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd685ffb640 1 -- 172.21.15.16:0/1841061144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67400ccc0 con 0x7fd688102570 2024-09-15T22:47:48.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.393+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd688069f70 con 0x7fd688102570 2024-09-15T22:47:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd685ffb640 1 -- 172.21.15.16:0/1841061144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd67400c460 con 0x7fd688102570 2024-09-15T22:47:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 Processor -- start 2024-09-15T22:47:48.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 -- start start 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x560070290e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x56007029fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600701f6280 con 0x560070290a20 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56007019db10 con 0x560070291340 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd69128c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5600701f2240 con 0x5600702a0150 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd68d450640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x560070290e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd68d450640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x560070290e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33422/0 (socket says 172.21.15.16:33422) 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd68d450640 1 -- 172.21.15.16:0/1404775315 learned_addr learned my addr 172.21.15.16:0/1404775315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.396+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x56007029fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd687fff640 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd68d450640 1 -- 172.21.15.16:0/1404775315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 msgr2=0x5600702a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd68d450640 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd68d450640 1 -- 172.21.15.16:0/1404775315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 msgr2=0x56007029fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd68d450640 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x56007029fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd68d450640 1 -- 172.21.15.16:0/1404775315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600701aa1f0 con 0x560070290a20 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd687fff640 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd68d450640 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x560070290e00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd680010750 tx=0x7fd680010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862d7b4c0f3fdd25 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd685ffb640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd6780a33b0 0x7fd6780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd672ffd640 1 -- 172.21.15.16:0/1404775315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68000e8d0 con 0x560070290a20 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd6780a33b0 0x7fd6780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd685ffb640 1 -- 172.21.15.16:0/1841061144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fd6740d1510 con 0x7fd688102570 2024-09-15T22:47:48.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd672ffd640 1 -- 172.21.15.16:0/1404775315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680012450 con 0x560070290a20 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.397+0000 7fd672ffd640 1 -- 172.21.15.16:0/1404775315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6800115e0 con 0x560070290a20 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 -- 172.21.15.16:0/1404775315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 msgr2=0x560070290e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd6780a33b0 0x7fd6780a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009230 tx=0x7fd67c0058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x560070290e00 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd680010750 tx=0x7fd680010b50 comp rx=0 tx=0).stop 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 -- 172.21.15.16:0/1404775315 shutdown_connections 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x56007029fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 --2- 172.21.15.16:0/1404775315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x560070290e00 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 -- 172.21.15.16:0/1404775315 >> 172.21.15.16:0/1404775315 conn(0x56007028d0f0 msgr2=0x56007028d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:48.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 -- 172.21.15.16:0/1404775315 shutdown_connections 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.398+0000 7fd69128c740 1 -- 172.21.15.16:0/1404775315 wait complete. 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 -- start start 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 Processor -- start 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x56007029d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x56007029db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5600702a2d20 con 0x560070290a20 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5600701b0f10 con 0x5600702a0150 2024-09-15T22:47:48.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5600702a2ea0 con 0x560070291340 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd69128c740 10 client.0 _ll_register_callbacks cb 0x560070179a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd68cc4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd68cc4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33426/0 (socket says 172.21.15.16:33426) 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd68cc4f640 1 -- 172.21.15.16:0/409193889 learned_addr learned my addr 172.21.15.16:0/409193889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd687fff640 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x56007029d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.399+0000 7fd68d450640 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x56007029db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd68cc4f640 1 -- 172.21.15.16:0/409193889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 msgr2=0x56007029d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x56007029d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd68cc4f640 1 -- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 msgr2=0x56007029db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x56007029db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd68cc4f640 1 -- 172.21.15.16:0/409193889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56006ff676a0 con 0x560070290a20 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d080 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e141baddbfa5a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd667fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd667fff640 1 -- 172.21.15.16:0/409193889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c035050 con 0x560070290a20 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd667fff640 1 -- 172.21.15.16:0/409193889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fd67c031750 con 0x560070290a20 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd680012450 con 0x560070290a20 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd667fff640 1 -- 172.21.15.16:0/409193889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd67c031950 con 0x560070290a20 2024-09-15T22:47:48.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd69128c740 10 client.17592 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5600702ae960 con 0x560070290a20 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.400+0000 7fd667fff640 1 -- 172.21.15.16:0/409193889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd680012450 con 0x560070290a20 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd667fff640 1 -- 172.21.15.16:0/409193889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd67c053cb0 con 0x560070290a20 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd69128c740 10 client.17592 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd69128c740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd69128c740 20 client.17592 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd69128c740 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5600702afe20 0x5600702b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd69128c740 4 client.17592 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600702037f0 con 0x5600702afe20 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd687fff640 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5600702afe20 0x5600702b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd687fff640 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5600702afe20 0x5600702b2210 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a000408e646a38 server_cookie=feda4d17e091687d in_seq=0 out_seq=0 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.401+0000 7fd667fff640 10 client.17592 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:48.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.402+0000 7fd667fff640 1 -- 172.21.15.16:0/409193889 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600702037f0 con 0x5600702afe20 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.402+0000 7fd667fff640 10 client.17592 handle_command_reply: tid=0 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.402+0000 7fd69128c740 1 client.17592 shutdown 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 msgr2=0x56007029d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d080 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd67c000c00 tx=0x7fd67c02a480 comp rx=0 tx=0).stop 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 shutdown_connections 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x5600702afe20 0x5600702b2210 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x56007029db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x56007029d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd667fff640 0 client.17592 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 --2- 172.21.15.16:0/409193889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d080 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 >> 172.21.15.16:0/409193889 conn(0x56007028d0f0 msgr2=0x56007028c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:48.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 shutdown_connections 2024-09-15T22:47:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 1 -- 172.21.15.16:0/409193889 wait complete. 2024-09-15T22:47:48.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.403+0000 7fd69128c740 20 client.17592 trim_cache size 0 max 16384 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 Processor -- start 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 -- start start 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x5600702ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x5600702abc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881af590 con 0x560070290a20 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ace60 con 0x5600702a0150 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.420+0000 7fd69128c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881aacd0 con 0x560070291340 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33430/0 (socket says 172.21.15.16:33430) 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 -- 172.21.15.16:0/3302047671 learned_addr learned my addr 172.21.15.16:0/3302047671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:48.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68d450640 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x5600702abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd687fff640 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x5600702ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 -- 172.21.15.16:0/3302047671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 msgr2=0x5600702ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x5600702ab6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 -- 172.21.15.16:0/3302047671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 msgr2=0x5600702abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x5600702abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 -- 172.21.15.16:0/3302047671 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600702b2870 con 0x560070290a20 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68d450640 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x5600702abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd687fff640 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x5600702ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d950 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009900 tx=0x7fd67c003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d893cd483cc8df1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd6727fc640 1 -- 172.21.15.16:0/3302047671 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c035030 con 0x560070290a20 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd6727fc640 1 -- 172.21.15.16:0/3302047671 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c04e070 con 0x560070290a20 2024-09-15T22:47:48.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.421+0000 7fd6727fc640 1 -- 172.21.15.16:0/3302047671 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c049770 con 0x560070290a20 2024-09-15T22:47:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 -- 172.21.15.16:0/3302047671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 msgr2=0x56007029d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d950 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009900 tx=0x7fd67c003d20 comp rx=0 tx=0).stop 2024-09-15T22:47:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 -- 172.21.15.16:0/3302047671 shutdown_connections 2024-09-15T22:47:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600702a0150 0x5600702abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x560070291340 0x5600702ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 --2- 172.21.15.16:0/3302047671 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x56007029d950 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 -- 172.21.15.16:0/3302047671 >> 172.21.15.16:0/3302047671 conn(0x56007028d0f0 msgr2=0x5600702aafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 -- 172.21.15.16:0/3302047671 shutdown_connections 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 -- 172.21.15.16:0/3302047671 wait complete. 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.422+0000 7fd69128c740 1 -- start start 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 1 Processor -- start 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x5600702a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x5600702a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ab320 con 0x560070290a20 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ac790 con 0x560070291340 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ab9f0 con 0x5600702a0150 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd69128c740 10 client.0 _ll_register_callbacks cb 0x7fd67c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:48.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x5600702a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x5600702a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33432/0 (socket says 172.21.15.16:33432) 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 -- 172.21.15.16:0/2180198948 learned_addr learned my addr 172.21.15.16:0/2180198948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68d450640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd687fff640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x5600702a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 -- 172.21.15.16:0/2180198948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 msgr2=0x5600702a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 -- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 msgr2=0x5600702a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x5600702a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68cc4f640 1 -- 172.21.15.16:0/2180198948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560070287680 con 0x560070290a20 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd68d450640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.423+0000 7fd687fff640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x5600702a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd68cc4f640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x5600702a47f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd67c049580 tx=0x7fd67c0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5737c3a8bf3f69ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd667fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:48.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd667fff640 1 -- 172.21.15.16:0/2180198948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c064030 con 0x560070290a20 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd667fff640 1 -- 172.21.15.16:0/2180198948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fd67c042040 con 0x560070290a20 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67c04e070 con 0x560070290a20 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd667fff640 1 -- 172.21.15.16:0/2180198948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd67c071050 con 0x560070290a20 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd69128c740 10 client.17598 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56007031b330 con 0x560070290a20 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd667fff640 1 -- 172.21.15.16:0/2180198948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c04e070 con 0x560070290a20 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.424+0000 7fd667fff640 1 -- 172.21.15.16:0/2180198948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd67c053800 con 0x560070290a20 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd69128c740 10 client.17598 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd69128c740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd69128c740 20 client.17598 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd69128c740 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56007031b970 0x56007031dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd69128c740 4 client.17598 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5600701fdd00 con 0x56007031b970 2024-09-15T22:47:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd687fff640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56007031b970 0x56007031dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd667fff640 10 client.17598 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.425+0000 7fd687fff640 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56007031b970 0x56007031dd90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b09d5fbd9582c3b server_cookie=63831e15dc4d6108 in_seq=0 out_seq=0 2024-09-15T22:47:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd667fff640 1 -- 172.21.15.16:0/2180198948 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5600701fdd00 con 0x56007031b970 2024-09-15T22:47:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd667fff640 10 client.17598 handle_command_reply: tid=0 2024-09-15T22:47:48.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 client.17598 shutdown 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 msgr2=0x5600702a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x5600702a47f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd67c049580 tx=0x7fd67c0092e0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 shutdown_connections 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56007031b970 0x56007031dd90 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5600702a0150 0x5600702a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x560070291340 0x5600702a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 --2- 172.21.15.16:0/2180198948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560070290a20 0x5600702a47f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.426+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 >> 172.21.15.16:0/2180198948 conn(0x56007028d0f0 msgr2=0x5600702ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd667fff640 0 client.17598 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 shutdown_connections 2024-09-15T22:47:48.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd69128c740 1 -- 172.21.15.16:0/2180198948 wait complete. 2024-09-15T22:47:48.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd69128c740 20 client.17598 trim_cache size 0 max 16384 2024-09-15T22:47:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:47:48.433 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:47:48.433 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "1cf874ff-550e-457d-9b61-9cea3f13b28f", 2024-09-15T22:47:48.433 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:47:48.433 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd6780a33b0 msgr2=0x7fd6780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd6780a33b0 0x7fd6780a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009230 tx=0x7fd67c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688102570 msgr2=0x7fd6881978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688102570 0x7fd6881978f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fd674009970 tx=0x7fd674009e30 comp rx=0 tx=0).stop 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 shutdown_connections 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd688102ef0 0x7fd68819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd688102570 0x7fd6881978f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd6780a33b0 0x7fd6780a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 --2- 172.21.15.16:0/1841061144 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd688100e20 0x7fd6881973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.427+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 >> 172.21.15.16:0/1841061144 conn(0x7fd6880fc7d0 msgr2=0x7fd6880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:48.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.428+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 shutdown_connections 2024-09-15T22:47:48.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:48.428+0000 7fd68eeb3640 1 -- 172.21.15.16:0/1841061144 wait complete. 2024-09-15T22:47:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:48 smithi016 ceph-mon[31492]: pgmap v290: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:48 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:47:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:48 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:48 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:47:48.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:48 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:48.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:48 smithi059 ceph-mon[34872]: pgmap v290: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:48.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:48 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:47:48.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:48 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:48.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:48 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:47:48.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:48 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:48 smithi130 ceph-mon[35742]: pgmap v290: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:47:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:48 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:47:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:48 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:48 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:47:49.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:48 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:47:49.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:47:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:47:49.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 -- 172.21.15.16:0/2015810509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4101420 msgr2=0x7fb9f410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 --2- 172.21.15.16:0/2015810509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4101420 0x7fb9f410d950 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009cd0 tx=0x7fb9ec02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 -- 172.21.15.16:0/2015810509 shutdown_connections 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 --2- 172.21.15.16:0/2015810509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4101420 0x7fb9f410d950 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 --2- 172.21.15.16:0/2015810509 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9f4100a80 0x7fb9f4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 --2- 172.21.15.16:0/2015810509 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9f4106a80 0x7fb9f4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 -- 172.21.15.16:0/2015810509 >> 172.21.15.16:0/2015810509 conn(0x7fb9f40fc7a0 msgr2=0x7fb9f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 -- 172.21.15.16:0/2015810509 shutdown_connections 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.829+0000 7fb9fc672640 1 -- 172.21.15.16:0/2015810509 wait complete. 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 Processor -- start 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 -- start start 2024-09-15T22:47:49.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4100a80 0x7fb9f4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9f4101420 0x7fb9f4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9f4106a80 0x7fb9f419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110e80 con 0x7fb9f4100a80 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110d00 con 0x7fb9f4106a80 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fc672640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4111000 con 0x7fb9f4101420 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fa40e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4100a80 0x7fb9f4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fa40e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4100a80 0x7fb9f4198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33448/0 (socket says 172.21.15.16:33448) 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fa40e640 1 -- 172.21.15.16:0/3920060432 learned_addr learned my addr 172.21.15.16:0/3920060432 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9f4101420 0x7fb9f4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.830+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9f4106a80 0x7fb9f419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fa40e640 1 -- 172.21.15.16:0/3920060432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9f4101420 msgr2=0x7fb9f4198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fa40e640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9f4101420 0x7fb9f4198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fa40e640 1 -- 172.21.15.16:0/3920060432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9f4106a80 msgr2=0x7fb9f419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fa40e640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9f4106a80 0x7fb9f419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fa40e640 1 -- 172.21.15.16:0/3920060432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f419d3d0 con 0x7fb9f4100a80 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fa40e640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4100a80 0x7fb9f4198420 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c920 tx=0x7fb9f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9eb7fe640 1 -- 172.21.15.16:0/3920060432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0007c70 con 0x7fb9f4100a80 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9eb7fe640 1 -- 172.21.15.16:0/3920060432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f0007e10 con 0x7fb9f4100a80 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f419d6c0 con 0x7fb9f4100a80 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9eb7fe640 1 -- 172.21.15.16:0/3920060432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0018890 con 0x7fb9f4100a80 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.831+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f41a4fa0 con 0x7fb9f4100a80 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.833+0000 7fb9eb7fe640 1 -- 172.21.15.16:0/3920060432 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb9f0018a70 con 0x7fb9f4100a80 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.833+0000 7fb9fea4b740 1 Processor -- start 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fea4b740 1 -- start start 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19380 0x55f2a5a19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a28550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f2a597f2e0 con 0x55f2a5a19380 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f2a5926ce0 con 0x55f2a5a19ca0 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f2a597b2a0 con 0x55f2a5a28a90 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19380 0x55f2a5a19760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19380 0x55f2a5a19760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33460/0 (socket says 172.21.15.16:33460) 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 -- 172.21.15.16:0/432982813 learned_addr learned my addr 172.21.15.16:0/432982813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fa40e640 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a28550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 -- 172.21.15.16:0/432982813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a28a90 msgr2=0x55f2a5a2ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 -- 172.21.15.16:0/432982813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 msgr2=0x55f2a5a28550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a28550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9fac0f640 1 -- 172.21.15.16:0/432982813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a590b440 con 0x55f2a5a19380 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.834+0000 7fb9eb7fe640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb9e00a33b0 0x7fb9e00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.835+0000 7fb9fa40e640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb9e00a33b0 0x7fb9e00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.835+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19380 0x55f2a5a19760 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec005cc0 tx=0x7fb9ec009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906d2f9b9eeeb523 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.835+0000 7fb9eb7fe640 1 -- 172.21.15.16:0/3920060432 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fb9f00d01a0 con 0x7fb9f4100a80 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.835+0000 7fb9e8ff9640 1 -- 172.21.15.16:0/432982813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec031ac0 con 0x55f2a5a19380 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.835+0000 7fb9e8ff9640 1 -- 172.21.15.16:0/432982813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec036040 con 0x55f2a5a19380 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.835+0000 7fb9e8ff9640 1 -- 172.21.15.16:0/432982813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec004740 con 0x55f2a5a19380 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 -- 172.21.15.16:0/432982813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19380 msgr2=0x55f2a5a19760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fa40e640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb9e00a33b0 0x7fb9e00a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00cd5f0 tx=0x7fb9f00d5040 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19380 0x55f2a5a19760 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec005cc0 tx=0x7fb9ec009a20 comp rx=0 tx=0).stop 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 -- 172.21.15.16:0/432982813 shutdown_connections 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a28550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/432982813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19380 0x55f2a5a19760 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 -- 172.21.15.16:0/432982813 >> 172.21.15.16:0/432982813 conn(0x55f2a5a15a30 msgr2=0x55f2a5a15e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 -- 172.21.15.16:0/432982813 shutdown_connections 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 -- 172.21.15.16:0/432982813 wait complete. 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 -- start start 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 Processor -- start 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a28440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.836+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a256f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a28a90 0x55f2a5a25c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f2a5a2b660 con 0x55f2a5a19ca0 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f2a593a050 con 0x55f2a5a28a90 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55f2a5a2b7e0 con 0x55f2a5a19380 2024-09-15T22:47:49.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fea4b740 10 client.0 _ll_register_callbacks cb 0x55f2a597a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a256f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a256f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33462/0 (socket says 172.21.15.16:33462) 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fac0f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a28440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 -- 172.21.15.16:0/2456184887 learned_addr learned my addr 172.21.15.16:0/2456184887 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a28a90 0x55f2a5a25c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 -- 172.21.15.16:0/2456184887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 msgr2=0x55f2a5a28440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a28440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 -- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a28a90 msgr2=0x55f2a5a25c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a28a90 0x55f2a5a25c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 -- 172.21.15.16:0/2456184887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2a56f03c0 con 0x55f2a5a19ca0 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a28a90 0x55f2a5a25c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a256f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400b920 tx=0x7fb9e400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33aaf5c96b66f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:49.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.837+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2456184887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4005020 con 0x55f2a5a19ca0 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2456184887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fb9e40051c0 con 0x55f2a5a19ca0 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ec036040 con 0x55f2a5a19ca0 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2456184887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9e4024050 con 0x55f2a5a19ca0 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 10 client.17610 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2a5a37110 con 0x55f2a5a19ca0 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2456184887 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec036040 con 0x55f2a5a19ca0 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2456184887 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb9e401b470 con 0x55f2a5a19ca0 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 10 client.17610 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 20 client.17610 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:49.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a38620 0x55f2a5a3aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 4 client.17610 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2a598c850 con 0x55f2a5a38620 2024-09-15T22:47:49.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.838+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a38620 0x55f2a5a3aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.839+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a38620 0x55f2a5a3aa40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3134389b7210a538 server_cookie=40b5c4be383961e5 in_seq=0 out_seq=0 2024-09-15T22:47:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.839+0000 7fb9ddffb640 10 client.17610 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2456184887 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2a598c850 con 0x55f2a5a38620 2024-09-15T22:47:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9ddffb640 10 client.17610 handle_command_reply: tid=0 2024-09-15T22:47:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 client.17610 shutdown 2024-09-15T22:47:49.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19ca0 msgr2=0x55f2a5a256f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a256f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400b920 tx=0x7fb9e400bde0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 shutdown_connections 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a38620 0x55f2a5a3aa40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a28a90 0x55f2a5a25c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a256f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2456184887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a28440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 >> 172.21.15.16:0/2456184887 conn(0x55f2a5a15a30 msgr2=0x55f2a5a1b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9ddffb640 0 client.17610 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 shutdown_connections 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2456184887 wait complete. 2024-09-15T22:47:49.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.840+0000 7fb9fea4b740 20 client.17610 trim_cache size 0 max 16384 2024-09-15T22:47:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 Processor -- start 2024-09-15T22:47:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 -- start start 2024-09-15T22:47:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19380 0x55f2a5a34900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a24fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a20c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41af620 con 0x55f2a5a28a90 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41acef0 con 0x55f2a5a19380 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.857+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41aad60 con 0x55f2a5a19ca0 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a20c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33464/0 (socket says 172.21.15.16:33464) 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 -- 172.21.15.16:0/776894533 learned_addr learned my addr 172.21.15.16:0/776894533 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19380 0x55f2a5a34900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a24fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 -- 172.21.15.16:0/776894533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19ca0 msgr2=0x55f2a5a24fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a24fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 -- 172.21.15.16:0/776894533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19380 msgr2=0x55f2a5a34900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19380 0x55f2a5a34900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 -- 172.21.15.16:0/776894533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2a5a25690 con 0x55f2a5a28a90 2024-09-15T22:47:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a20c00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009960 tx=0x7fb9ec009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28099dcbfe87ad56 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9dffff640 1 -- 172.21.15.16:0/776894533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0315e0 con 0x55f2a5a28a90 2024-09-15T22:47:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9dffff640 1 -- 172.21.15.16:0/776894533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec036040 con 0x55f2a5a28a90 2024-09-15T22:47:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.858+0000 7fb9dffff640 1 -- 172.21.15.16:0/776894533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec003cd0 con 0x55f2a5a28a90 2024-09-15T22:47:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 -- 172.21.15.16:0/776894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 msgr2=0x55f2a5a20c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a20c00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009960 tx=0x7fb9ec009a20 comp rx=0 tx=0).stop 2024-09-15T22:47:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 -- 172.21.15.16:0/776894533 shutdown_connections 2024-09-15T22:47:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a20c00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a24fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/776894533 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19380 0x55f2a5a34900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 -- 172.21.15.16:0/776894533 >> 172.21.15.16:0/776894533 conn(0x55f2a5a15a30 msgr2=0x55f2a5a204f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 -- 172.21.15.16:0/776894533 shutdown_connections 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 -- 172.21.15.16:0/776894533 wait complete. 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 -- start start 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.859+0000 7fb9fea4b740 1 Processor -- start 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a2c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a2cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fea4b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41ab3b0 con 0x55f2a5a28a90 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41ac820 con 0x55f2a5a19ca0 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fea4b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41aba80 con 0x55f2a5a19380 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fea4b740 10 client.0 _ll_register_callbacks cb 0x7fb9e40042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:49.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33468/0 (socket says 172.21.15.16:33468) 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 -- 172.21.15.16:0/2975108689 learned_addr learned my addr 172.21.15.16:0/2975108689 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a2cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a2c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 -- 172.21.15.16:0/2975108689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 msgr2=0x55f2a5a2c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a2c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 -- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 msgr2=0x55f2a5a2cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a2cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 -- 172.21.15.16:0/2975108689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2a5a26000 con 0x55f2a5a28a90 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a2cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9fac0f640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2d280 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec048040 tx=0x7fb9ec003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14676b0ed1df742 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:47:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.860+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2975108689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec0315e0 con 0x55f2a5a28a90 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2975108689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9467+0+0 (secure 0 0 0) 0x7fb9ec064020 con 0x55f2a5a28a90 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ec036040 con 0x55f2a5a28a90 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2975108689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9ec0044b0 con 0x55f2a5a28a90 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 10 client.17616 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f2a5a1c0e0 con 0x55f2a5a28a90 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2975108689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec036040 con 0x55f2a5a28a90 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2975108689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb9ec030070 con 0x55f2a5a28a90 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 10 client.17616 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 20 client.17616 populate_metadata read hostname 'smithi016' 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a1c370 0x55f2a5a1e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 4 client.17616 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:47:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f2a5986d60 con 0x55f2a5a1c370 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.861+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a1c370 0x55f2a5a1e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9f9c0d640 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a1c370 0x55f2a5a1e790 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1701558dd52abe4c server_cookie=9069f58cf74a816f in_seq=0 out_seq=0 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9ddffb640 10 client.17616 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9ddffb640 1 -- 172.21.15.16:0/2975108689 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f2a5986d60 con 0x55f2a5a1c370 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9ddffb640 10 client.17616 handle_command_reply: tid=0 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 client.17616 shutdown 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 msgr2=0x55f2a5a2d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2d280 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec048040 tx=0x7fb9ec003ab0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 shutdown_connections 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55f2a5a1c370 0x55f2a5a1e790 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f2a5a28a90 0x55f2a5a2d280 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f2a5a19ca0 0x55f2a5a2cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 --2- 172.21.15.16:0/2975108689 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55f2a5a19380 0x55f2a5a2c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 >> 172.21.15.16:0/2975108689 conn(0x55f2a5a15a30 msgr2=0x55f2a5a20d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9ddffb640 0 client.17616 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 shutdown_connections 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 1 -- 172.21.15.16:0/2975108689 wait complete. 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.862+0000 7fb9fea4b740 20 client.17616 trim_cache size 0 max 16384 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:47:49.869 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:47:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "1cf874ff-550e-457d-9b61-9cea3f13b28f": { 2024-09-15T22:47:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:47:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "1cf874ff-550e-457d-9b61-9cea3f13b28f", 2024-09-15T22:47:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:47:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:47:49.870 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:47:49.870 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb9e00a33b0 msgr2=0x7fb9e00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb9e00a33b0 0x7fb9e00a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00cd5f0 tx=0x7fb9f00d5040 comp rx=0 tx=0).stop 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4100a80 msgr2=0x7fb9f4198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4100a80 0x7fb9f4198420 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c920 tx=0x7fb9f000cde0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 shutdown_connections 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9f4106a80 0x7fb9f419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fb9e00a33b0 0x7fb9e00a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fb9f4101420 0x7fb9f4198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 --2- 172.21.15.16:0/3920060432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9f4100a80 0x7fb9f4198420 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 >> 172.21.15.16:0/3920060432 conn(0x7fb9f40fc7a0 msgr2=0x7fb9f406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 shutdown_connections 2024-09-15T22:47:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:47:49.863+0000 7fb9fc672640 1 -- 172.21.15.16:0/3920060432 wait complete. 2024-09-15T22:47:50.583 INFO:tasks.cephfs.filesystem:scrub status for tag:1cf874ff-550e-457d-9b61-9cea3f13b28f - {'path': '/', 'tag': '1cf874ff-550e-457d-9b61-9cea3f13b28f', 'options': 'recursive,force'} 2024-09-15T22:47:50.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:50 smithi016 ceph-mon[31492]: pgmap v291: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:50.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:50 smithi059 ceph-mon[34872]: pgmap v291: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:51.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:50 smithi130 ceph-mon[35742]: pgmap v291: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:52.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:52 smithi016 ceph-mon[31492]: pgmap v292: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:47:52.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:52 smithi059 ceph-mon[34872]: pgmap v292: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:47:53.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:52 smithi130 ceph-mon[35742]: pgmap v292: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-09-15T22:47:53.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:53 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:47:54.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:53 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:47:54.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:53 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:47:54.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:54 smithi059 ceph-mon[34872]: pgmap v293: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:55.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:54 smithi130 ceph-mon[35742]: pgmap v293: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:55.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:54 smithi016 ceph-mon[31492]: pgmap v293: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:56.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:56 smithi059 ceph-mon[34872]: pgmap v294: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:57.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:56 smithi130 ceph-mon[35742]: pgmap v294: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:57.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:56 smithi016 ceph-mon[31492]: pgmap v294: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:47:58.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:47:58 smithi059 ceph-mon[34872]: pgmap v295: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-09-15T22:47:59.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:47:58 smithi130 ceph-mon[35742]: pgmap v295: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-09-15T22:47:59.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:47:58 smithi016 ceph-mon[31492]: pgmap v295: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-09-15T22:48:00.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:00 smithi059 ceph-mon[34872]: pgmap v296: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-15T22:48:01.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:00 smithi130 ceph-mon[35742]: pgmap v296: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-15T22:48:01.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:00 smithi016 ceph-mon[31492]: pgmap v296: 129 pgs: 129 active+clean; 17 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-15T22:48:02.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:02 smithi059 ceph-mon[34872]: pgmap v297: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:48:03.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:02 smithi130 ceph-mon[35742]: pgmap v297: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:48:03.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:02 smithi016 ceph-mon[31492]: pgmap v297: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:48:04.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:04 smithi059 ceph-mon[34872]: pgmap v298: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-09-15T22:48:05.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:04 smithi130 ceph-mon[35742]: pgmap v298: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-09-15T22:48:05.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:04 smithi016 ceph-mon[31492]: pgmap v298: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-09-15T22:48:06.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:06 smithi059 ceph-mon[34872]: pgmap v299: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-09-15T22:48:07.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:06 smithi130 ceph-mon[35742]: pgmap v299: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-09-15T22:48:07.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:06 smithi016 ceph-mon[31492]: pgmap v299: 129 pgs: 129 active+clean; 17 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-09-15T22:48:08.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:08 smithi059 ceph-mon[34872]: pgmap v300: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:48:09.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:08 smithi130 ceph-mon[35742]: pgmap v300: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:48:09.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:08 smithi016 ceph-mon[31492]: pgmap v300: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:48:09.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:09 smithi059 ceph-mon[34872]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:48:10.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:09 smithi130 ceph-mon[35742]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:48:10.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:09 smithi016 ceph-mon[31492]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:48:10.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:10 smithi059 ceph-mon[34872]: pgmap v302: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:48:11.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:10 smithi130 ceph-mon[35742]: pgmap v302: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:48:11.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:10 smithi016 ceph-mon[31492]: pgmap v302: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:48:12.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:12 smithi059 ceph-mon[34872]: pgmap v303: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:13.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:12 smithi130 ceph-mon[35742]: pgmap v303: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:13.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:12 smithi016 ceph-mon[31492]: pgmap v303: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:14 smithi059 ceph-mon[34872]: pgmap v304: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:14.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:14 smithi059 ceph-mon[34872]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:48:15.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:14 smithi130 ceph-mon[35742]: pgmap v304: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:15.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:14 smithi130 ceph-mon[35742]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:48:15.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:14 smithi016 ceph-mon[31492]: pgmap v304: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:15.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:14 smithi016 ceph-mon[31492]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:48:16.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:16 smithi059 ceph-mon[34872]: pgmap v306: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:48:17.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:16 smithi130 ceph-mon[35742]: pgmap v306: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:48:17.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:16 smithi016 ceph-mon[31492]: pgmap v306: 129 pgs: 129 active+clean; 17 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:48:18.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:18 smithi059 ceph-mon[34872]: pgmap v307: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 239 B/s wr, 7 op/s 2024-09-15T22:48:19.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:18 smithi130 ceph-mon[35742]: pgmap v307: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 239 B/s wr, 7 op/s 2024-09-15T22:48:19.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:18 smithi016 ceph-mon[31492]: pgmap v307: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 239 B/s wr, 7 op/s 2024-09-15T22:48:20.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:48:20.857 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:48:20.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:20 smithi059 ceph-mon[34872]: pgmap v308: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:20.977 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:20 smithi016 ceph-mon[31492]: pgmap v308: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:21.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:20 smithi130 ceph-mon[35742]: pgmap v308: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:48:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.297+0000 7ff74f351640 1 -- 172.21.15.16:0/2072661498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff748103660 msgr2=0x7ff74810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.297+0000 7ff74f351640 1 --2- 172.21.15.16:0/2072661498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff748103660 0x7ff74810bb50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff738009900 tx=0x7ff73802a480 comp rx=0 tx=0).stop 2024-09-15T22:48:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.297+0000 7ff74f351640 1 -- 172.21.15.16:0/2072661498 shutdown_connections 2024-09-15T22:48:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.297+0000 7ff74f351640 1 --2- 172.21.15.16:0/2072661498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff74810c170 0x7ff74810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.297+0000 7ff74f351640 1 --2- 172.21.15.16:0/2072661498 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff748103660 0x7ff74810bb50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.297+0000 7ff74f351640 1 --2- 172.21.15.16:0/2072661498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748102d40 0x7ff748103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.298+0000 7ff74f351640 1 -- 172.21.15.16:0/2072661498 >> 172.21.15.16:0/2072661498 conn(0x7ff7480fc7a0 msgr2=0x7ff7480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.298+0000 7ff74f351640 1 -- 172.21.15.16:0/2072661498 shutdown_connections 2024-09-15T22:48:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.298+0000 7ff74f351640 1 -- 172.21.15.16:0/2072661498 wait complete. 2024-09-15T22:48:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.298+0000 7ff74f351640 1 Processor -- start 2024-09-15T22:48:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.298+0000 7ff74f351640 1 -- start start 2024-09-15T22:48:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74f351640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff748102d40 0x7ff748198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74f351640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748103660 0x7ff7481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74f351640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff74810c170 0x7ff74819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74f351640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff748111f70 con 0x7ff748103660 2024-09-15T22:48:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74f351640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff748111df0 con 0x7ff748102d40 2024-09-15T22:48:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74f351640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481120f0 con 0x7ff74810c170 2024-09-15T22:48:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74c8ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748103660 0x7ff7481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74c8ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748103660 0x7ff7481989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40730/0 (socket says 172.21.15.16:40730) 2024-09-15T22:48:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74c8ec640 1 -- 172.21.15.16:0/948064633 learned_addr learned my addr 172.21.15.16:0/948064633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d8ee640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff74810c170 0x7ff74819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff748102d40 0x7ff748198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d8ee640 1 -- 172.21.15.16:0/948064633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff748102d40 msgr2=0x7ff748198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d8ee640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff748102d40 0x7ff748198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d8ee640 1 -- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748103660 msgr2=0x7ff7481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d8ee640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748103660 0x7ff7481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d8ee640 1 -- 172.21.15.16:0/948064633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74819d420 con 0x7ff74810c170 2024-09-15T22:48:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff748102d40 0x7ff748198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.299+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748103660 0x7ff7481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.300+0000 7ff74d8ee640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff74810c170 0x7ff74819cd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff74000e940 tx=0x7ff74000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.300+0000 7ff73e7fc640 1 -- 172.21.15.16:0/948064633 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740004d20 con 0x7ff74810c170 2024-09-15T22:48:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.300+0000 7ff73e7fc640 1 -- 172.21.15.16:0/948064633 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff740005020 con 0x7ff74810c170 2024-09-15T22:48:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.300+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74819d710 con 0x7ff74810c170 2024-09-15T22:48:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.300+0000 7ff73e7fc640 1 -- 172.21.15.16:0/948064633 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7400116b0 con 0x7ff74810c170 2024-09-15T22:48:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.300+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7481a4ff0 con 0x7ff74810c170 2024-09-15T22:48:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.302+0000 7ff75172a740 1 Processor -- start 2024-09-15T22:48:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff73e7fc640 1 -- 172.21.15.16:0/948064633 <== mon.1 v2:172.21.15.130:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff7400054d0 con 0x7ff74810c170 2024-09-15T22:48:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff75172a740 1 -- start start 2024-09-15T22:48:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff75172a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843670760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff75172a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff75172a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843681e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff75172a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c8435d62e0 con 0x55c843670ca0 2024-09-15T22:48:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff75172a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c84357dce0 con 0x55c843670380 2024-09-15T22:48:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff75172a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c8435d22a0 con 0x55c84367fa90 2024-09-15T22:48:21.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40738/0 (socket says 172.21.15.16:40738) 2024-09-15T22:48:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 -- 172.21.15.16:0/3576652073 learned_addr learned my addr 172.21.15.16:0/3576652073 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843681e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843670760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 -- 172.21.15.16:0/3576652073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 msgr2=0x55c843681e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff73e7fc640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff7340a33b0 0x7ff7340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843681e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 -- 172.21.15.16:0/3576652073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 msgr2=0x55c843670760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843670760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74d0ed640 1 -- 172.21.15.16:0/3576652073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c843562440 con 0x55c843670ca0 2024-09-15T22:48:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.303+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843681e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:21.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.304+0000 7ff74d8ee640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff7340a33b0 0x7ff7340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.304+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367f550 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff744009d60 tx=0x7ff7440117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a7dfc8540c5d7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.304+0000 7ff73e7fc640 1 -- 172.21.15.16:0/948064633 <== mon.1 v2:172.21.15.130:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7ff7400d04c0 con 0x7ff74810c170 2024-09-15T22:48:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.304+0000 7ff7337fe640 1 -- 172.21.15.16:0/3576652073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744012650 con 0x55c843670ca0 2024-09-15T22:48:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.304+0000 7ff7337fe640 1 -- 172.21.15.16:0/3576652073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff744009e50 con 0x55c843670ca0 2024-09-15T22:48:21.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.304+0000 7ff7337fe640 1 -- 172.21.15.16:0/3576652073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74400fcc0 con 0x55c843670ca0 2024-09-15T22:48:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 -- 172.21.15.16:0/3576652073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 msgr2=0x55c84367f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367f550 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ff744009d60 tx=0x7ff7440117b0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff74d8ee640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff7340a33b0 0x7ff7340a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7400a4b30 tx=0x7ff7400855e0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 -- 172.21.15.16:0/3576652073 shutdown_connections 2024-09-15T22:48:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843681e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367f550 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 --2- 172.21.15.16:0/3576652073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843670760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 -- 172.21.15.16:0/3576652073 >> 172.21.15.16:0/3576652073 conn(0x55c84366ca30 msgr2=0x55c84366ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 -- 172.21.15.16:0/3576652073 shutdown_connections 2024-09-15T22:48:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 -- 172.21.15.16:0/3576652073 wait complete. 2024-09-15T22:48:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 -- start start 2024-09-15T22:48:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.305+0000 7ff75172a740 1 Processor -- start 2024-09-15T22:48:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff75172a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84367f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff75172a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff75172a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c84367cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff75172a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c843682660 con 0x55c843670ca0 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff75172a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c843591050 con 0x55c843670380 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff75172a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55c8436827e0 con 0x55c84367fa90 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff75172a740 10 client.0 _ll_register_callbacks cb 0x55c8435d1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40748/0 (socket says 172.21.15.16:40748) 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 -- 172.21.15.16:0/1523556439 learned_addr learned my addr 172.21.15.16:0/1523556439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c84367cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84367f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 -- 172.21.15.16:0/1523556439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 msgr2=0x55c84367cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c84367cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 -- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 msgr2=0x55c84367f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84367f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 -- 172.21.15.16:0/1523556439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8433473c0 con 0x55c843670ca0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367c640 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff74400f860 tx=0x7ff74400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fb10f534419433 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff730ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.306+0000 7ff730ff9640 1 -- 172.21.15.16:0/1523556439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744028030 con 0x55c843670ca0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff730ff9640 1 -- 172.21.15.16:0/1523556439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7ff744005250 con 0x55c843670ca0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff744009e50 con 0x55c843670ca0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff730ff9640 1 -- 172.21.15.16:0/1523556439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff744032050 con 0x55c843670ca0 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 10 client.17628 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c84368e010 con 0x55c843670ca0 2024-09-15T22:48:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff730ff9640 1 -- 172.21.15.16:0/1523556439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff744009e50 con 0x55c843670ca0 2024-09-15T22:48:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff730ff9640 1 -- 172.21.15.16:0/1523556439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff7440124e0 con 0x55c843670ca0 2024-09-15T22:48:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 10 client.17628 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 20 client.17628 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c84368f470 0x55c843691860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 4 client.17628 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8435e3850 con 0x55c84368f470 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.307+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c84368f470 0x55c843691860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.308+0000 7ff730ff9640 10 client.17628 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.308+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c84368f470 0x55c843691860 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6973b4f7b10eaa74 server_cookie=4238702785b5ec71 in_seq=0 out_seq=0 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.308+0000 7ff730ff9640 1 -- 172.21.15.16:0/1523556439 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8435e3850 con 0x55c84368f470 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.308+0000 7ff730ff9640 10 client.17628 handle_command_reply: tid=0 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.308+0000 7ff75172a740 1 client.17628 shutdown 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 msgr2=0x55c84367c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367c640 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff74400f860 tx=0x7ff74400f6a0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 shutdown_connections 2024-09-15T22:48:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c84368f470 0x55c843691860 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c84367cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c84367c640 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff730ff9640 0 client.17628 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 --2- 172.21.15.16:0/1523556439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84367f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 >> 172.21.15.16:0/1523556439 conn(0x55c84366ca30 msgr2=0x55c843672790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 shutdown_connections 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 1 -- 172.21.15.16:0/1523556439 wait complete. 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.309+0000 7ff75172a740 20 client.17628 trim_cache size 0 max 16384 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 Processor -- start 2024-09-15T22:48:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 -- start start 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84368b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c843696ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843697400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481af790 con 0x55c843670ca0 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481ad060 con 0x55c843670380 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.326+0000 7ff75172a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481aaed0 con 0x55c84367fa90 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c843696ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c843696ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40756/0 (socket says 172.21.15.16:40756) 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 -- 172.21.15.16:0/261086834 learned_addr learned my addr 172.21.15.16:0/261086834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84368b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 -- 172.21.15.16:0/261086834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 msgr2=0x55c843697400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843697400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843697400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 -- 172.21.15.16:0/261086834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 msgr2=0x55c84368b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84368b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 -- 172.21.15.16:0/261086834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c84367ceb0 con 0x55c843670ca0 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843697400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c843696ec0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff744012ac0 tx=0x7ff7440117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b4420e9be939f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff732ffd640 1 -- 172.21.15.16:0/261086834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74403a020 con 0x55c843670ca0 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff732ffd640 1 -- 172.21.15.16:0/261086834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff744009e50 con 0x55c843670ca0 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.327+0000 7ff732ffd640 1 -- 172.21.15.16:0/261086834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744028070 con 0x55c843670ca0 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 -- 172.21.15.16:0/261086834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 msgr2=0x55c843696ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c843696ec0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff744012ac0 tx=0x7ff7440117b0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 -- 172.21.15.16:0/261086834 shutdown_connections 2024-09-15T22:48:21.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c84367fa90 0x55c843697400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c843670ca0 0x55c843696ec0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 --2- 172.21.15.16:0/261086834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c84368b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 -- 172.21.15.16:0/261086834 >> 172.21.15.16:0/261086834 conn(0x55c84366ca30 msgr2=0x55c84367b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 -- 172.21.15.16:0/261086834 shutdown_connections 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 -- 172.21.15.16:0/261086834 wait complete. 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 -- start start 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.328+0000 7ff75172a740 1 Processor -- start 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff75172a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843679750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff75172a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c843670ca0 0x55c843679c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff75172a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c84367fa90 0x55c84367a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff75172a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481ab520 con 0x55c84367fa90 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff75172a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481ac990 con 0x55c843670380 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff75172a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481abbf0 con 0x55c843670ca0 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff75172a740 10 client.0 _ll_register_callbacks cb 0x7ff744009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c84367fa90 0x55c84367a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c84367fa90 0x55c84367a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40760/0 (socket says 172.21.15.16:40760) 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 -- 172.21.15.16:0/2034090587 learned_addr learned my addr 172.21.15.16:0/2034090587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c843670ca0 0x55c843679c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843679750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 -- 172.21.15.16:0/2034090587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c843670ca0 msgr2=0x55c843679c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c843670ca0 0x55c843679c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 -- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 msgr2=0x55c843679750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843679750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.329+0000 7ff74c8ec640 1 -- 172.21.15.16:0/2034090587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8436f8f40 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c843670ca0 0x55c843679c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff74c8ec640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c84367fa90 0x55c84367a1d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff738009850 tx=0x7ff738040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5962c8dee3598fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff730ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff730ff9640 1 -- 172.21.15.16:0/2034090587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738035040 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff730ff9640 1 -- 172.21.15.16:0/2034090587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7ff738004250 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff744009e50 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff730ff9640 1 -- 172.21.15.16:0/2034090587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff73805b050 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff75172a740 10 client.17634 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c8436fa1a0 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.330+0000 7ff730ff9640 1 -- 172.21.15.16:0/2034090587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff744009e50 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff730ff9640 1 -- 172.21.15.16:0/2034090587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff738004a60 con 0x55c84367fa90 2024-09-15T22:48:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff75172a740 10 client.17634 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff75172a740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff75172a740 20 client.17634 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff75172a740 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c8436fa900 0x55c8436fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff75172a740 4 client.17634 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8435ddd60 con 0x55c8436fa900 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c8436fa900 0x55c8436fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff74d0ed640 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c8436fa900 0x55c8436fcd20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd16d811f946227 server_cookie=3ce6cf67b6daedd5 in_seq=0 out_seq=0 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.331+0000 7ff730ff9640 10 client.17634 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff730ff9640 1 -- 172.21.15.16:0/2034090587 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c8435ddd60 con 0x55c8436fa900 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff730ff9640 10 client.17634 handle_command_reply: tid=0 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 client.17634 shutdown 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c84367fa90 msgr2=0x55c84367a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c84367fa90 0x55c84367a1d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff738009850 tx=0x7ff738040040 comp rx=0 tx=0).stop 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 shutdown_connections 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55c8436fa900 0x55c8436fcd20 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c84367fa90 0x55c84367a1d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55c843670ca0 0x55c843679c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff730ff9640 0 client.17634 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 --2- 172.21.15.16:0/2034090587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c843670380 0x55c843679750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 >> 172.21.15.16:0/2034090587 conn(0x55c84366ca30 msgr2=0x55c84366b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 shutdown_connections 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 1 -- 172.21.15.16:0/2034090587 wait complete. 2024-09-15T22:48:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.332+0000 7ff75172a740 20 client.17634 trim_cache size 0 max 16384 2024-09-15T22:48:21.362 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:48:21.362 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:48:21.362 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:48:21.362 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:48:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff7340a33b0 msgr2=0x7ff7340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff7340a33b0 0x7ff7340a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7400a4b30 tx=0x7ff7400855e0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff74810c170 msgr2=0x7ff74819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff74810c170 0x7ff74819cd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff74000e940 tx=0x7ff74000ee00 comp rx=0 tx=0).stop 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 shutdown_connections 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff7340a33b0 0x7ff7340a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff74810c170 0x7ff74819cd40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff748103660 0x7ff7481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 --2- 172.21.15.16:0/948064633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff748102d40 0x7ff748198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 >> 172.21.15.16:0/948064633 conn(0x7ff7480fc7a0 msgr2=0x7ff7480fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 shutdown_connections 2024-09-15T22:48:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:21.333+0000 7ff74f351640 1 -- 172.21.15.16:0/948064633 wait complete. 2024-09-15T22:48:22.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:48:22.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:48:22.365 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:48:22.771 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:22 smithi016 ceph-mon[31492]: pgmap v309: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:48:22.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 -- 172.21.15.16:0/2303198141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0102d30 msgr2=0x7fa2c0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 --2- 172.21.15.16:0/2303198141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0102d30 0x7fa2c0103110 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009930 tx=0x7fa2b002a440 comp rx=0 tx=0).stop 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 -- 172.21.15.16:0/2303198141 shutdown_connections 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 --2- 172.21.15.16:0/2303198141 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2c01082a0 0x7fa2c010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 --2- 172.21.15.16:0/2303198141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2c01036e0 0x7fa2c0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 --2- 172.21.15.16:0/2303198141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c0102d30 0x7fa2c0103110 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 -- 172.21.15.16:0/2303198141 >> 172.21.15.16:0/2303198141 conn(0x7fa2c00fc7d0 msgr2=0x7fa2c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 -- 172.21.15.16:0/2303198141 shutdown_connections 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 -- 172.21.15.16:0/2303198141 wait complete. 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.852+0000 7fa2c5830640 1 Processor -- start 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2c5830640 1 -- start start 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2c5830640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2c0102d30 0x7fa2c0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2c5830640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c01036e0 0x7fa2c0198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:22.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2c5830640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2c01082a0 0x7fa2c019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2c5830640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0112100 con 0x7fa2c01036e0 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2c5830640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0111f80 con 0x7fa2c0102d30 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2c5830640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0112280 con 0x7fa2c01082a0 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2be7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c01036e0 0x7fa2c0198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2be7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c01036e0 0x7fa2c0198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54138/0 (socket says 172.21.15.16:54138) 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2be7fc640 1 -- 172.21.15.16:0/1928646316 learned_addr learned my addr 172.21.15.16:0/1928646316 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2bf7fe640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2c01082a0 0x7fa2c019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.853+0000 7fa2beffd640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2c0102d30 0x7fa2c0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2be7fc640 1 -- 172.21.15.16:0/1928646316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2c01082a0 msgr2=0x7fa2c019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2be7fc640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2c01082a0 0x7fa2c019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2be7fc640 1 -- 172.21.15.16:0/1928646316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2c0102d30 msgr2=0x7fa2c0198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2be7fc640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2c0102d30 0x7fa2c0198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2be7fc640 1 -- 172.21.15.16:0/1928646316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c019d5d0 con 0x7fa2c01036e0 2024-09-15T22:48:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2be7fc640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c01036e0 0x7fa2c0198b60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006fd0 tx=0x7fa2a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2c482e640 1 -- 172.21.15.16:0/1928646316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8011830 con 0x7fa2c01036e0 2024-09-15T22:48:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2c482e640 1 -- 172.21.15.16:0/1928646316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a8016070 con 0x7fa2c01036e0 2024-09-15T22:48:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c019d8c0 con 0x7fa2c01036e0 2024-09-15T22:48:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2c482e640 1 -- 172.21.15.16:0/1928646316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800f2e0 con 0x7fa2c01036e0 2024-09-15T22:48:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.854+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c01a51a0 con 0x7fa2c01036e0 2024-09-15T22:48:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.855+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c006a850 con 0x7fa2c01036e0 2024-09-15T22:48:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.856+0000 7fa2c482e640 1 -- 172.21.15.16:0/1928646316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa2a801b460 con 0x7fa2c01036e0 2024-09-15T22:48:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.857+0000 7fa2c482e640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2ac0a3430 0x7fa2ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.857+0000 7fa2c482e640 1 -- 172.21.15.16:0/1928646316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7fa2a80cfbe0 con 0x7fa2c01036e0 2024-09-15T22:48:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.857+0000 7fa2beffd640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2ac0a3430 0x7fa2ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.858+0000 7fa2beffd640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2ac0a3430 0x7fa2ac0a58f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009900 tx=0x7fa2b00058d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:22.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:22.860+0000 7fa2c482e640 1 -- 172.21.15.16:0/1928646316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2a809c320 con 0x7fa2c01036e0 2024-09-15T22:48:22.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:22 smithi059 ceph-mon[34872]: pgmap v309: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:48:23.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:22 smithi130 ceph-mon[35742]: pgmap v309: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:48:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.070+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2c0103110 con 0x7fa2c01036e0 2024-09-15T22:48:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.071+0000 7fa2c482e640 1 -- 172.21.15.16:0/1928646316 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fa2c0103110 con 0x7fa2c01036e0 2024-09-15T22:48:23.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:48:23.072 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:48:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:48:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2ac0a3430 msgr2=0x7fa2ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2ac0a3430 0x7fa2ac0a58f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009900 tx=0x7fa2b00058d0 comp rx=0 tx=0).stop 2024-09-15T22:48:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c01036e0 msgr2=0x7fa2c0198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c01036e0 0x7fa2c0198b60 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8006fd0 tx=0x7fa2a800b8e0 comp rx=0 tx=0).stop 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 shutdown_connections 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa2ac0a3430 0x7fa2ac0a58f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa2c01082a0 0x7fa2c019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2c01036e0 0x7fa2c0198b60 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 --2- 172.21.15.16:0/1928646316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2c0102d30 0x7fa2c0198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 >> 172.21.15.16:0/1928646316 conn(0x7fa2c00fc7d0 msgr2=0x7fa2c0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 shutdown_connections 2024-09-15T22:48:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:23.073+0000 7fa2c5830640 1 -- 172.21.15.16:0/1928646316 wait complete. 2024-09-15T22:48:23.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:48:23.996 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:48:24.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:23 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1928646316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:48:24.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:23 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1928646316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:48:24.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:23 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1928646316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:48:24.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 -- 172.21.15.16:0/3297151792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308106a80 msgr2=0x7f4308106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 --2- 172.21.15.16:0/3297151792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308106a80 0x7f4308106e60 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009930 tx=0x7f42f802a420 comp rx=0 tx=0).stop 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 -- 172.21.15.16:0/3297151792 shutdown_connections 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 --2- 172.21.15.16:0/3297151792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4308101420 0x7f430810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 --2- 172.21.15.16:0/3297151792 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100a80 0x7f4308100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 --2- 172.21.15.16:0/3297151792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308106a80 0x7f4308106e60 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 -- 172.21.15.16:0/3297151792 >> 172.21.15.16:0/3297151792 conn(0x7f43080fc7a0 msgr2=0x7f43080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 -- 172.21.15.16:0/3297151792 shutdown_connections 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.503+0000 7f430def5640 1 -- 172.21.15.16:0/3297151792 wait complete. 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 Processor -- start 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 -- start start 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100a80 0x7f43081984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308101420 0x7f43081989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4308106a80 0x7f430819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4308111fa0 con 0x7f4308101420 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4308111e20 con 0x7f4308106a80 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f430def5640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4308112120 con 0x7f4308100a80 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308101420 0x7f43081989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308101420 0x7f43081989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54160/0 (socket says 172.21.15.16:54160) 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 -- 172.21.15.16:0/3310596586 learned_addr learned my addr 172.21.15.16:0/3310596586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f43077fe640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100a80 0x7f43081984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4307fff640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4308106a80 0x7f430819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 -- 172.21.15.16:0/3310596586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100a80 msgr2=0x7f43081984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100a80 0x7f43081984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 -- 172.21.15.16:0/3310596586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4308106a80 msgr2=0x7f430819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4308106a80 0x7f430819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f4306ffd640 1 -- 172.21.15.16:0/3310596586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f430819d450 con 0x7f4308101420 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.504+0000 7f43077fe640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100a80 0x7f43081984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.505+0000 7f4306ffd640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308101420 0x7f43081989e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00d950 tx=0x7f42fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.505+0000 7f4304ff9640 1 -- 172.21.15.16:0/3310596586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc017070 con 0x7f4308101420 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.505+0000 7f4304ff9640 1 -- 172.21.15.16:0/3310596586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42fc009c30 con 0x7f4308101420 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.505+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f430819d740 con 0x7f4308101420 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.505+0000 7f4304ff9640 1 -- 172.21.15.16:0/3310596586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc00f6b0 con 0x7f4308101420 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.505+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43081a5020 con 0x7f4308101420 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f4304ff9640 1 -- 172.21.15.16:0/3310596586 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f42fc0109d0 con 0x7f4308101420 2024-09-15T22:48:24.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 Processor -- start 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 -- start start 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55632c96b2f0 con 0x55632ca06b40 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55632c912cf0 con 0x55632ca14780 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f43102ce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55632c9672b0 con 0x55632ca055d0 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f4307fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f4307fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca05090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54174/0 (socket says 172.21.15.16:54174) 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.508+0000 7f4307fff640 1 -- 172.21.15.16:0/1348589064 learned_addr learned my addr 172.21.15.16:0/1348589064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4306ffd640 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f43077fe640 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4307fff640 1 -- 172.21.15.16:0/1348589064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 msgr2=0x55632ca05a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4307fff640 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca05a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4307fff640 1 -- 172.21.15.16:0/1348589064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 msgr2=0x55632ca16b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4307fff640 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca16b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4304ff9640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f42f00a33b0 0x7f42f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4307fff640 1 -- 172.21.15.16:0/1348589064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632c8f73a0 con 0x55632ca06b40 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4304ff9640 1 -- 172.21.15.16:0/3310596586 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f42fc0cfe10 con 0x7f4308101420 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4307fff640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f42f00a33b0 0x7f42f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f4307fff640 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca05090 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4300008d40 tx=0x7f430000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eece1c86804a7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f42edffb640 1 -- 172.21.15.16:0/1348589064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300020070 con 0x55632ca06b40 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f42edffb640 1 -- 172.21.15.16:0/1348589064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430000f3c0 con 0x55632ca06b40 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.509+0000 7f42edffb640 1 -- 172.21.15.16:0/1348589064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4300014070 con 0x55632ca06b40 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 -- 172.21.15.16:0/1348589064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 msgr2=0x55632ca05090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f4307fff640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f42f00a33b0 0x7f42f00a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f430001d000 tx=0x7f4300020000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca05090 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f4300008d40 tx=0x7f430000e720 comp rx=0 tx=0).stop 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 -- 172.21.15.16:0/1348589064 shutdown_connections 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca05a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 --2- 172.21.15.16:0/1348589064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca05090 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 -- 172.21.15.16:0/1348589064 >> 172.21.15.16:0/1348589064 conn(0x55632c9ce310 msgr2=0x55632ca01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 -- 172.21.15.16:0/1348589064 shutdown_connections 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.510+0000 7f43102ce740 1 -- 172.21.15.16:0/1348589064 wait complete. 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 -- start start 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 Processor -- start 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca055d0 0x55632ca14170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca11420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca14780 0x55632ca11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55632ca17480 con 0x55632ca06b40 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55632c926060 con 0x55632ca055d0 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55632ca17600 con 0x55632ca14780 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43102ce740 10 client.0 _ll_register_callbacks cb 0x55632c6798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca11420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca11420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54190/0 (socket says 172.21.15.16:54190) 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f4306ffd640 1 -- 172.21.15.16:0/4081850472 learned_addr learned my addr 172.21.15.16:0/4081850472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f43077fe640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca055d0 0x55632ca14170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.511+0000 7f4307fff640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca14780 0x55632ca11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f4306ffd640 1 -- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca14780 msgr2=0x55632ca11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f4306ffd640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca14780 0x55632ca11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f4306ffd640 1 -- 172.21.15.16:0/4081850472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca055d0 msgr2=0x55632ca14170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f4306ffd640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca055d0 0x55632ca14170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f4306ffd640 1 -- 172.21.15.16:0/4081850472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632c6dc3c0 con 0x55632ca06b40 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f4307fff640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca14780 0x55632ca11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f4306ffd640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca11420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f42fc09ba40 tx=0x7f42fc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3096fc284eae5259 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f42e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f42e6ffd640 1 -- 172.21.15.16:0/4081850472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc020040 con 0x55632ca06b40 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f42e6ffd640 1 -- 172.21.15.16:0/4081850472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f42fc09b190 con 0x55632ca06b40 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430000f3c0 con 0x55632ca06b40 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f42e6ffd640 1 -- 172.21.15.16:0/4081850472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42fc009de0 con 0x55632ca06b40 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f43102ce740 10 client.17652 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55632ca22f70 con 0x55632ca06b40 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f42e6ffd640 1 -- 172.21.15.16:0/4081850472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430000f3c0 con 0x55632ca06b40 2024-09-15T22:48:24.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f42e6ffd640 1 -- 172.21.15.16:0/4081850472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f42fc09ac00 con 0x55632ca06b40 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f43102ce740 10 client.17652 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f43102ce740 10 client.17652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f43102ce740 20 client.17652 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.512+0000 7f43102ce740 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca24380 0x55632ca26790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.513+0000 7f43102ce740 4 client.17652 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.513+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55632c978860 con 0x55632ca24380 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.513+0000 7f43077fe640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca24380 0x55632ca26790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.513+0000 7f43077fe640 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca24380 0x55632ca26790 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf50bc4d1db4e856 server_cookie=79e74a4c17e22341 in_seq=0 out_seq=0 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.513+0000 7f42e6ffd640 10 client.17652 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f42e6ffd640 1 -- 172.21.15.16:0/4081850472 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55632c978860 con 0x55632ca24380 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f42e6ffd640 10 client.17652 handle_command_reply: tid=0 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 client.17652 shutdown 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 msgr2=0x55632ca11420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca11420 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f42fc09ba40 tx=0x7f42fc010d70 comp rx=0 tx=0).stop 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 shutdown_connections 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca24380 0x55632ca26790 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca14780 0x55632ca11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca11420 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f42e6ffd640 0 client.17652 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 --2- 172.21.15.16:0/4081850472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca055d0 0x55632ca14170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 >> 172.21.15.16:0/4081850472 conn(0x55632c9ce310 msgr2=0x55632ca07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 shutdown_connections 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.514+0000 7f43102ce740 1 -- 172.21.15.16:0/4081850472 wait complete. 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.515+0000 7f43102ce740 20 client.17652 trim_cache size 0 max 16384 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.527+0000 7f43102ce740 1 Processor -- start 2024-09-15T22:48:24.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.527+0000 7f43102ce740 1 -- start start 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f43102ce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca20720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f43102ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca2c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f43102ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca2c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f43102ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43081af7b0 con 0x55632ca06b40 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f43102ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43081ad080 con 0x55632ca14780 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f43102ce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f43081aaef0 con 0x55632ca055d0 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca2c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca2c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54204/0 (socket says 172.21.15.16:54204) 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 -- 172.21.15.16:0/1580225658 learned_addr learned my addr 172.21.15.16:0/1580225658 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f43077fe640 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca20720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 -- 172.21.15.16:0/1580225658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 msgr2=0x55632ca20720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca20720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 -- 172.21.15.16:0/1580225658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 msgr2=0x55632ca2c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca2c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 -- 172.21.15.16:0/1580225658 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55632ca06780 con 0x55632ca06b40 2024-09-15T22:48:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.528+0000 7f4306ffd640 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca2c060 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0cd260 tx=0x7f42fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c789fc1b4d96ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.529+0000 7f42ed7fa640 1 -- 172.21.15.16:0/1580225658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc020070 con 0x55632ca06b40 2024-09-15T22:48:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.529+0000 7f42ed7fa640 1 -- 172.21.15.16:0/1580225658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42fc0a49b0 con 0x55632ca06b40 2024-09-15T22:48:24.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.529+0000 7f42ed7fa640 1 -- 172.21.15.16:0/1580225658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0983b0 con 0x55632ca06b40 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 -- 172.21.15.16:0/1580225658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 msgr2=0x55632ca2c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca2c060 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0cd260 tx=0x7f42fc005880 comp rx=0 tx=0).stop 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 -- 172.21.15.16:0/1580225658 shutdown_connections 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca2c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca2c060 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 --2- 172.21.15.16:0/1580225658 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca20720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 -- 172.21.15.16:0/1580225658 >> 172.21.15.16:0/1580225658 conn(0x55632c9ce310 msgr2=0x55632ca10530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 -- 172.21.15.16:0/1580225658 shutdown_connections 2024-09-15T22:48:24.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 -- 172.21.15.16:0/1580225658 wait complete. 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 -- start start 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.530+0000 7f43102ce740 1 Processor -- start 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43102ce740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca0c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43102ce740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca1c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43102ce740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca1ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43102ce740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43081ab540 con 0x55632ca06b40 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43102ce740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43081ac9b0 con 0x55632ca14780 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43102ce740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f43081abc10 con 0x55632ca055d0 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43102ce740 10 client.0 _ll_register_callbacks cb 0x7f42fc00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca1c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43077fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca0c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43077fe640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca0c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.130:3300/0 says I am v2:172.21.15.16:33622/0 (socket says 172.21.15.16:33622) 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4306ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca1c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54212/0 (socket says 172.21.15.16:54212) 2024-09-15T22:48:24.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43077fe640 1 -- 172.21.15.16:0/3550865260 learned_addr learned my addr 172.21.15.16:0/3550865260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4307fff640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca1ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4306ffd640 1 -- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 msgr2=0x55632ca0c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4306ffd640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca0c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4306ffd640 1 -- 172.21.15.16:0/3550865260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 msgr2=0x55632ca1ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4306ffd640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca1ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4306ffd640 1 -- 172.21.15.16:0/3550865260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632ca09530 con 0x55632ca06b40 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f43077fe640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca0c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.531+0000 7f4307fff640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca1ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f4306ffd640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca1c890 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x55632ca11250 tx=0x7f42fc09ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9470a2e88c21be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f42e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:24.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f42e6ffd640 1 -- 172.21.15.16:0/3550865260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0b8020 con 0x55632ca06b40 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f42e6ffd640 1 -- 172.21.15.16:0/3550865260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f42fc09b3a0 con 0x55632ca06b40 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42fc0a49b0 con 0x55632ca06b40 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f42e6ffd640 1 -- 172.21.15.16:0/3550865260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42fc020070 con 0x55632ca06b40 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 10 client.17658 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55632ca8b220 con 0x55632ca06b40 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f42e6ffd640 1 -- 172.21.15.16:0/3550865260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42fc0a49b0 con 0x55632ca06b40 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f42e6ffd640 1 -- 172.21.15.16:0/3550865260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f42fc097c90 con 0x55632ca06b40 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 10 client.17658 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 10 client.17658 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 20 client.17658 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca8b7d0 0x55632ca8dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 4 client.17658 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:48:24.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.532+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55632c972d70 con 0x55632ca8b7d0 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43077fe640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca8b7d0 0x55632ca8dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43077fe640 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca8b7d0 0x55632ca8dbf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4081ca429005c77d server_cookie=918a92217ad7c304 in_seq=0 out_seq=0 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f42e6ffd640 10 client.17658 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f42e6ffd640 1 -- 172.21.15.16:0/3550865260 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55632c972d70 con 0x55632ca8b7d0 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f42e6ffd640 10 client.17658 handle_command_reply: tid=0 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 client.17658 shutdown 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 msgr2=0x55632ca1c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca1c890 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x55632ca11250 tx=0x7f42fc09ae30 comp rx=0 tx=0).stop 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 shutdown_connections 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x55632ca8b7d0 0x55632ca8dbf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55632ca14780 0x55632ca1ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55632ca06b40 0x55632ca1c890 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f42e6ffd640 0 client.17658 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:48:24.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 --2- 172.21.15.16:0/3550865260 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55632ca055d0 0x55632ca0c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.533+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 >> 172.21.15.16:0/3550865260 conn(0x55632c9ce310 msgr2=0x55632ca12c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:24.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 shutdown_connections 2024-09-15T22:48:24.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f43102ce740 1 -- 172.21.15.16:0/3550865260 wait complete. 2024-09-15T22:48:24.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f43102ce740 20 client.17658 trim_cache size 0 max 16384 2024-09-15T22:48:24.551 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f42f00a33b0 msgr2=0x7f42f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f42f00a33b0 0x7f42f00a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f430001d000 tx=0x7f4300020000 comp rx=0 tx=0).stop 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308101420 msgr2=0x7f43081989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308101420 0x7f43081989e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00d950 tx=0x7f42fc00de10 comp rx=0 tx=0).stop 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 shutdown_connections 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4308106a80 0x7f430819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4308101420 0x7f43081989e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f42f00a33b0 0x7f42f00a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 --2- 172.21.15.16:0/3310596586 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4308100a80 0x7f43081984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 >> 172.21.15.16:0/3310596586 conn(0x7f43080fc7a0 msgr2=0x7f43080fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:24.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 shutdown_connections 2024-09-15T22:48:24.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:24.534+0000 7f430def5640 1 -- 172.21.15.16:0/3310596586 wait complete. 2024-09-15T22:48:25.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:24 smithi130 ceph-mon[35742]: pgmap v310: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:48:25.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:24 smithi016 ceph-mon[31492]: pgmap v310: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:48:25.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:24 smithi059 ceph-mon[34872]: pgmap v310: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:48:25.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:48:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 -- 172.21.15.16:0/4137043079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404100a80 msgr2=0x7f7404100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 --2- 172.21.15.16:0/4137043079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404100a80 0x7f7404100ee0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009930 tx=0x7f73f402a420 comp rx=0 tx=0).stop 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 -- 172.21.15.16:0/4137043079 shutdown_connections 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 --2- 172.21.15.16:0/4137043079 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7404101420 0x7f740410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 --2- 172.21.15.16:0/4137043079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404100a80 0x7f7404100ee0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 --2- 172.21.15.16:0/4137043079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404106a80 0x7f7404106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 -- 172.21.15.16:0/4137043079 >> 172.21.15.16:0/4137043079 conn(0x7f74040fc7a0 msgr2=0x7f74040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 -- 172.21.15.16:0/4137043079 shutdown_connections 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 -- 172.21.15.16:0/4137043079 wait complete. 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 Processor -- start 2024-09-15T22:48:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.978+0000 7f740c2a2640 1 -- start start 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740c2a2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7404100a80 0x7f740419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740c2a2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404101420 0x7f740419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740c2a2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404106a80 0x7f740419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740c2a2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7404110e80 con 0x7f7404106a80 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740c2a2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7404110d00 con 0x7f7404101420 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740c2a2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7404111000 con 0x7f7404100a80 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404106a80 0x7f740419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404106a80 0x7f740419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54228/0 (socket says 172.21.15.16:54228) 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 -- 172.21.15.16:0/806833956 learned_addr learned my addr 172.21.15.16:0/806833956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a03e640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7404100a80 0x7f740419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740983d640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404101420 0x7f740419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 -- 172.21.15.16:0/806833956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7404100a80 msgr2=0x7f740419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7404100a80 0x7f740419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 -- 172.21.15.16:0/806833956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404101420 msgr2=0x7f740419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404101420 0x7f740419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 -- 172.21.15.16:0/806833956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740419f5a0 con 0x7f7404106a80 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740983d640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404101420 0x7f740419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f740a83f640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404106a80 0x7f740419eec0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00eb10 tx=0x7f73fc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.979+0000 7f73fb7fe640 1 -- 172.21.15.16:0/806833956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00ce20 con 0x7f7404106a80 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.980+0000 7f73fb7fe640 1 -- 172.21.15.16:0/806833956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73fc005020 con 0x7f7404106a80 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.980+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740419f890 con 0x7f7404106a80 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.980+0000 7f73fb7fe640 1 -- 172.21.15.16:0/806833956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc010690 con 0x7f7404106a80 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.980+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74041a7170 con 0x7f7404106a80 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.982+0000 7f73fb7fe640 1 -- 172.21.15.16:0/806833956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f73fc005540 con 0x7f7404106a80 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.982+0000 7f740e67b740 1 Processor -- start 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.982+0000 7f740e67b740 1 -- start start 2024-09-15T22:48:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740e67b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740e67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740e67b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740e67b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc7e53c2f0 con 0x55fc7e5d7b40 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740e67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fc7e4e3cf0 con 0x55fc7e5d65d0 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740e67b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55fc7e5382b0 con 0x55fc7e5e1370 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f73fb7fe640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f73f00a33b0 0x7f73f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54236/0 (socket says 172.21.15.16:54236) 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 -- 172.21.15.16:0/3409074272 learned_addr learned my addr 172.21.15.16:0/3409074272 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740a03e640 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740a83f640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f73f00a33b0 0x7f73f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f73fb7fe640 1 -- 172.21.15.16:0/806833956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f73fc0d0840 con 0x7f7404106a80 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 -- 172.21.15.16:0/3409074272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 msgr2=0x55fc7e5e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 -- 172.21.15.16:0/3409074272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 msgr2=0x55fc7e5d6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5d6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.983+0000 7f740983d640 1 -- 172.21.15.16:0/3409074272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc7e4c83a0 con 0x55fc7e5d7b40 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.984+0000 7f740a83f640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f73f00a33b0 0x7f73f00a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73fc09c470 tx=0x7f73fc091c00 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.984+0000 7f740983d640 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d6090 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f73f4031550 tx=0x7f73f4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715f2a2f781a3cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.984+0000 7f73f8dea640 1 -- 172.21.15.16:0/3409074272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035050 con 0x55fc7e5d7b40 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.984+0000 7f73f8dea640 1 -- 172.21.15.16:0/3409074272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f4031980 con 0x55fc7e5d7b40 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.984+0000 7f73f8dea640 1 -- 172.21.15.16:0/3409074272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4005650 con 0x55fc7e5d7b40 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.984+0000 7f740e67b740 1 -- 172.21.15.16:0/3409074272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d7b40 msgr2=0x55fc7e5d6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.984+0000 7f740e67b740 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d6090 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f73f4031550 tx=0x7f73f4005bb0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- 172.21.15.16:0/3409074272 shutdown_connections 2024-09-15T22:48:26.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5d6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 --2- 172.21.15.16:0/3409074272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d6090 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- 172.21.15.16:0/3409074272 >> 172.21.15.16:0/3409074272 conn(0x55fc7e59f310 msgr2=0x55fc7e5d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- 172.21.15.16:0/3409074272 shutdown_connections 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- 172.21.15.16:0/3409074272 wait complete. 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- start start 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 Processor -- start 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55fc7e5e4010 con 0x55fc7e5e1370 2024-09-15T22:48:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fc7e4f7060 con 0x55fc7e5d65d0 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.985+0000 7f740e67b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55fc7e5e4190 con 0x55fc7e5d7b40 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740e67b740 10 client.0 _ll_register_callbacks cb 0x55fc7e24a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5de5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54244/0 (socket says 172.21.15.16:54244) 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 -- 172.21.15.16:0/3366396724 learned_addr learned my addr 172.21.15.16:0/3366396724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740a03e640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 -- 172.21.15.16:0/3366396724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d7b40 msgr2=0x55fc7e5ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 -- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 msgr2=0x55fc7e5e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 -- 172.21.15.16:0/3366396724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc7e2ad3c0 con 0x55fc7e5e1370 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740a03e640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740983d640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5de5d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f73f4032eb0 tx=0x7f73f4041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648380245897eac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f73edffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f73edffb640 1 -- 172.21.15.16:0/3366396724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f404e020 con 0x55fc7e5e1370 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f73edffb640 1 -- 172.21.15.16:0/3366396724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f73f4004b80 con 0x55fc7e5e1370 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f4031980 con 0x55fc7e5e1370 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f73edffb640 1 -- 172.21.15.16:0/3366396724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73f4059050 con 0x55fc7e5e1370 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740e67b740 10 client.17670 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.986+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc7e5efa70 con 0x55fc7e5e1370 2024-09-15T22:48:26.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f73edffb640 1 -- 172.21.15.16:0/3366396724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f4031980 con 0x55fc7e5e1370 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f73edffb640 1 -- 172.21.15.16:0/3366396724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73f4003820 con 0x55fc7e5e1370 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f740e67b740 10 client.17670 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f740e67b740 10 client.17670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f740e67b740 20 client.17670 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f740e67b740 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e5f0dc0 0x55fc7e5f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f740e67b740 4 client.17670 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc7e549860 con 0x55fc7e5f0dc0 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.987+0000 7f740a03e640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e5f0dc0 0x55fc7e5f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.988+0000 7f740a03e640 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e5f0dc0 0x55fc7e5f31b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11c9109f07a07edb server_cookie=bf8c6d43ba843354 in_seq=0 out_seq=0 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.988+0000 7f73edffb640 10 client.17670 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.988+0000 7f73edffb640 1 -- 172.21.15.16:0/3366396724 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc7e549860 con 0x55fc7e5f0dc0 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.988+0000 7f73edffb640 10 client.17670 handle_command_reply: tid=0 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.988+0000 7f740e67b740 1 client.17670 shutdown 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 msgr2=0x55fc7e5de5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5de5d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f73f4032eb0 tx=0x7f73f4041040 comp rx=0 tx=0).stop 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 shutdown_connections 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e5f0dc0 0x55fc7e5f31b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5de5d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f73edffb640 0 client.17670 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 --2- 172.21.15.16:0/3366396724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 >> 172.21.15.16:0/3366396724 conn(0x55fc7e59f310 msgr2=0x55fc7e5d8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 shutdown_connections 2024-09-15T22:48:26.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 1 -- 172.21.15.16:0/3366396724 wait complete. 2024-09-15T22:48:26.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:25.989+0000 7f740e67b740 20 client.17670 trim_cache size 0 max 16384 2024-09-15T22:48:26.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 Processor -- start 2024-09-15T22:48:26.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 -- start start 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5dff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 0x55fc7e5d9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74041b3a70 con 0x55fc7e5d65d0 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74041b1340 con 0x55fc7e5d7b40 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740e67b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f74041af1b0 con 0x55fc7e5e1370 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5dff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5dff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54256/0 (socket says 172.21.15.16:54256) 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.002+0000 7f740983d640 1 -- 172.21.15.16:0/4166643376 learned_addr learned my addr 172.21.15.16:0/4166643376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f740a03e640 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f740983d640 1 -- 172.21.15.16:0/4166643376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 msgr2=0x55fc7e5d9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f740983d640 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 0x55fc7e5d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f740983d640 1 -- 172.21.15.16:0/4166643376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 msgr2=0x55fc7e5d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f740983d640 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f740983d640 1 -- 172.21.15.16:0/4166643376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc7e5d2ed0 con 0x55fc7e5d65d0 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f740983d640 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5dff50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f73f4057040 tx=0x7f73f40097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1fec4ced267534 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f73effff640 1 -- 172.21.15.16:0/4166643376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f404e040 con 0x55fc7e5d65d0 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f73effff640 1 -- 172.21.15.16:0/4166643376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f4005350 con 0x55fc7e5d65d0 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.003+0000 7f73effff640 1 -- 172.21.15.16:0/4166643376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f4035070 con 0x55fc7e5d65d0 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 -- 172.21.15.16:0/4166643376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d65d0 msgr2=0x55fc7e5dff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5dff50 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f73f4057040 tx=0x7f73f40097e0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 -- 172.21.15.16:0/4166643376 shutdown_connections 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5e1370 0x55fc7e5d9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 --2- 172.21.15.16:0/4166643376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5dff50 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 -- 172.21.15.16:0/4166643376 >> 172.21.15.16:0/4166643376 conn(0x55fc7e59f310 msgr2=0x55fc7e5df840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 -- 172.21.15.16:0/4166643376 shutdown_connections 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 -- 172.21.15.16:0/4166643376 wait complete. 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 -- start start 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.004+0000 7f740e67b740 1 Processor -- start 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5e5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74041af800 con 0x55fc7e5e1370 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74041b0c70 con 0x55fc7e5d7b40 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f74041afed0 con 0x55fc7e5d65d0 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 10 client.0 _ll_register_callbacks cb 0x7f740000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54272/0 (socket says 172.21.15.16:54272) 2024-09-15T22:48:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 -- 172.21.15.16:0/558777701 learned_addr learned my addr 172.21.15.16:0/558777701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740a03e640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 -- 172.21.15.16:0/558777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d65d0 msgr2=0x55fc7e5e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 -- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 msgr2=0x55fc7e5e5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5e5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 -- 172.21.15.16:0/558777701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc7e65a880 con 0x55fc7e5e1370 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740a03e640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5e5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740983d640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e5d50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f73f40096c0 tx=0x7f73f4004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef12f271cdcaf90 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f73edffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f73edffb640 1 -- 172.21.15.16:0/558777701 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f404e070 con 0x55fc7e5e1370 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f73edffb640 1 -- 172.21.15.16:0/558777701 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f73f40315b0 con 0x55fc7e5e1370 2024-09-15T22:48:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.005+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f4005350 con 0x55fc7e5e1370 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f73edffb640 1 -- 172.21.15.16:0/558777701 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73f4063040 con 0x55fc7e5e1370 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 10 client.17676 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc7e65bb40 con 0x55fc7e5e1370 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f73edffb640 1 -- 172.21.15.16:0/558777701 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f4005350 con 0x55fc7e5e1370 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f73edffb640 1 -- 172.21.15.16:0/558777701 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73f404a6b0 con 0x55fc7e5e1370 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 10 client.17676 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 10 client.17676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 20 client.17676 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e65bfc0 0x55fc7e65e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 4 client.17676 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc7e543d70 con 0x55fc7e65bfc0 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.006+0000 7f740a03e640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e65bfc0 0x55fc7e65e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740a03e640 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e65bfc0 0x55fc7e65e3b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea0f131ddf27a2e5 server_cookie=66a93169c132f68b in_seq=0 out_seq=0 2024-09-15T22:48:26.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f73edffb640 10 client.17676 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f73edffb640 1 -- 172.21.15.16:0/558777701 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc7e543d70 con 0x55fc7e65bfc0 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f73edffb640 10 client.17676 handle_command_reply: tid=0 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 client.17676 shutdown 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 msgr2=0x55fc7e5e5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e5d50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f73f40096c0 tx=0x7f73f4004e70 comp rx=0 tx=0).stop 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 shutdown_connections 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x55fc7e65bfc0 0x55fc7e65e3b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55fc7e5e1370 0x55fc7e5e5d50 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc7e5d7b40 0x55fc7e5e5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 --2- 172.21.15.16:0/558777701 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55fc7e5d65d0 0x55fc7e5e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 >> 172.21.15.16:0/558777701 conn(0x55fc7e59f310 msgr2=0x55fc7e5d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f73edffb640 0 client.17676 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 shutdown_connections 2024-09-15T22:48:26.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.007+0000 7f740e67b740 1 -- 172.21.15.16:0/558777701 wait complete. 2024-09-15T22:48:26.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740e67b740 20 client.17676 trim_cache size 0 max 16384 2024-09-15T22:48:26.018 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:48:26.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f73f00a33b0 msgr2=0x7f73f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f73f00a33b0 0x7f73f00a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f73fc09c470 tx=0x7f73fc091c00 comp rx=0 tx=0).stop 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404106a80 msgr2=0x7f740419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404106a80 0x7f740419eec0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00eb10 tx=0x7f73fc00ef10 comp rx=0 tx=0).stop 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 shutdown_connections 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7404106a80 0x7f740419eec0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7404101420 0x7f740419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f73f00a33b0 0x7f73f00a5870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 --2- 172.21.15.16:0/806833956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7404100a80 0x7f740419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 >> 172.21.15.16:0/806833956 conn(0x7f74040fc7a0 msgr2=0x7f740406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 shutdown_connections 2024-09-15T22:48:26.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:26.008+0000 7f740c2a2640 1 -- 172.21.15.16:0/806833956 wait complete. 2024-09-15T22:48:26.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:48:26.974 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:48:26.996 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:26 smithi059 ceph-mon[34872]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:48:27.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:26 smithi130 ceph-mon[35742]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:48:27.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:26 smithi016 ceph-mon[31492]: pgmap v311: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 -- 172.21.15.16:0/2146877680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc101420 msgr2=0x7f4acc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/2146877680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc101420 0x7f4acc10d950 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009cd0 tx=0x7f4ac402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 -- 172.21.15.16:0/2146877680 shutdown_connections 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/2146877680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc101420 0x7f4acc10d950 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/2146877680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4acc100a80 0x7f4acc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/2146877680 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4acc106a80 0x7f4acc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 -- 172.21.15.16:0/2146877680 >> 172.21.15.16:0/2146877680 conn(0x7f4acc0fc7a0 msgr2=0x7f4acc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 -- 172.21.15.16:0/2146877680 shutdown_connections 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.472+0000 7f4ad34b1640 1 -- 172.21.15.16:0/2146877680 wait complete. 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 Processor -- start 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 -- start start 2024-09-15T22:48:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc100a80 0x7f4acc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4acc101420 0x7f4acc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4acc106a80 0x7f4acc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc110ec0 con 0x7f4acc100a80 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc110d40 con 0x7f4acc106a80 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad34b1640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc111040 con 0x7f4acc101420 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc100a80 0x7f4acc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc100a80 0x7f4acc19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54294/0 (socket says 172.21.15.16:54294) 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 -- 172.21.15.16:0/1140852691 learned_addr learned my addr 172.21.15.16:0/1140852691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4acc101420 0x7f4acc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4acc106a80 0x7f4acc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 -- 172.21.15.16:0/1140852691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4acc101420 msgr2=0x7f4acc19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4acc101420 0x7f4acc19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 -- 172.21.15.16:0/1140852691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4acc106a80 msgr2=0x7f4acc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4acc106a80 0x7f4acc19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.473+0000 7f4ad124d640 1 -- 172.21.15.16:0/1140852691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc19f640 con 0x7f4acc100a80 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.474+0000 7f4ad124d640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc100a80 0x7f4acc19a690 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c970 tx=0x7f4ac800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.474+0000 7f4ac27fc640 1 -- 172.21.15.16:0/1140852691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8007bb0 con 0x7f4acc100a80 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.474+0000 7f4ac27fc640 1 -- 172.21.15.16:0/1140852691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac8007d50 con 0x7f4acc100a80 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.474+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc19f930 con 0x7f4acc100a80 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.474+0000 7f4ac27fc640 1 -- 172.21.15.16:0/1140852691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac800f450 con 0x7f4acc100a80 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.474+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a7210 con 0x7f4acc100a80 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.476+0000 7f4ac27fc640 1 -- 172.21.15.16:0/1140852691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4ac8005020 con 0x7f4acc100a80 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.476+0000 7f4ad588a740 1 Processor -- start 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad588a740 1 -- start start 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a55d0 0x55d5c02a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad588a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5c020b2f0 con 0x55d5c02a6b40 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad588a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d5c01b2cf0 con 0x55d5c02b0370 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad588a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d5c02072b0 con 0x55d5c02a55d0 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54302/0 (socket says 172.21.15.16:54302) 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 -- 172.21.15.16:0/2361514782 learned_addr learned my addr 172.21.15.16:0/2361514782 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad124d640 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a55d0 0x55d5c02a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 -- 172.21.15.16:0/2361514782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a55d0 msgr2=0x55d5c02a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a55d0 0x55d5c02a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ac27fc640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4ab80a33b0 0x7f4ab80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 -- 172.21.15.16:0/2361514782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 msgr2=0x55d5c02b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad1a4e640 1 -- 172.21.15.16:0/2361514782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c01973a0 con 0x55d5c02a6b40 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.477+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.478+0000 7f4ac27fc640 1 -- 172.21.15.16:0/1140852691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f4ac80cff60 con 0x7f4acc100a80 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.478+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4ab80a33b0 0x7f4ab80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.478+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a5090 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4005cc0 tx=0x7f4ac4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a5014cbb609664 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.478+0000 7f4ab77fe640 1 -- 172.21.15.16:0/2361514782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4003970 con 0x55d5c02a6b40 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.478+0000 7f4ab77fe640 1 -- 172.21.15.16:0/2361514782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac4036040 con 0x55d5c02a6b40 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.478+0000 7f4ab77fe640 1 -- 172.21.15.16:0/2361514782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4004e60 con 0x55d5c02a6b40 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 -- 172.21.15.16:0/2361514782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 msgr2=0x55d5c02a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a5090 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4005cc0 tx=0x7f4ac4009a20 comp rx=0 tx=0).stop 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4ab80a33b0 0x7f4ab80a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4048000 tx=0x7f4ac404b000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 -- 172.21.15.16:0/2361514782 shutdown_connections 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a55d0 0x55d5c02a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2361514782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a5090 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 -- 172.21.15.16:0/2361514782 >> 172.21.15.16:0/2361514782 conn(0x55d5c026e310 msgr2=0x55d5c02a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 -- 172.21.15.16:0/2361514782 shutdown_connections 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 -- 172.21.15.16:0/2361514782 wait complete. 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 -- start start 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.479+0000 7f4ad588a740 1 Processor -- start 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a6b40 0x55d5c02ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02ad640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad588a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d5c02b3010 con 0x55d5c02a55d0 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad588a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d5c01c6060 con 0x55d5c02b0370 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad588a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55d5c02b3190 con 0x55d5c02a6b40 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad588a740 10 client.0 _ll_register_callbacks cb 0x55d5bff198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54312/0 (socket says 172.21.15.16:54312) 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 -- 172.21.15.16:0/451471953 learned_addr learned my addr 172.21.15.16:0/451471953 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a6b40 0x55d5c02ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 -- 172.21.15.16:0/451471953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a6b40 msgr2=0x55d5c02ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a6b40 0x55d5c02ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 -- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 msgr2=0x55d5c02ad640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02ad640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 -- 172.21.15.16:0/451471953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bff7c3c0 con 0x55d5c02a55d0 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ad124d640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02afd80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c940 tx=0x7f4ac809b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2688ea2534f96f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ab4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.480+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/451471953 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac801d030 con 0x55d5c02a55d0 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/451471953 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f4ac80d20e0 con 0x55d5c02a55d0 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac4036040 con 0x55d5c02a55d0 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/451471953 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ac809be90 con 0x55d5c02a55d0 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 10 client.17688 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:27.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5c02beae0 con 0x55d5c02a55d0 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/451471953 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac4036040 con 0x55d5c02a55d0 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/451471953 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac809b2e0 con 0x55d5c02a55d0 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 10 client.17688 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 20 client.17688 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c02bfef0 0x55d5c02c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 4 client.17688 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5c0218860 con 0x55d5c02bfef0 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.481+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c02bfef0 0x55d5c02c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.482+0000 7f4ab4ff9640 10 client.17688 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.482+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c02bfef0 0x55d5c02c2320 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d26a651946345f server_cookie=8513dbc5a135eba3 in_seq=0 out_seq=0 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/451471953 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5c0218860 con 0x55d5c02bfef0 2024-09-15T22:48:27.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ab4ff9640 10 client.17688 handle_command_reply: tid=0 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 client.17688 shutdown 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 msgr2=0x55d5c02afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02afd80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c940 tx=0x7f4ac809b9d0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 shutdown_connections 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c02bfef0 0x55d5c02c2320 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02b0370 0x55d5c02ad640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ab4ff9640 0 client.17688 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02a6b40 0x55d5c02ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 --2- 172.21.15.16:0/451471953 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02afd80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 >> 172.21.15.16:0/451471953 conn(0x55d5c026e310 msgr2=0x55d5c02a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 shutdown_connections 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 1 -- 172.21.15.16:0/451471953 wait complete. 2024-09-15T22:48:27.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.483+0000 7f4ad588a740 20 client.17688 trim_cache size 0 max 16384 2024-09-15T22:48:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 Processor -- start 2024-09-15T22:48:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 -- start start 2024-09-15T22:48:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a55d0 0x55d5c02a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1b3b60 con 0x55d5c02a6b40 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1b1430 con 0x55d5c02a55d0 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad588a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1af2a0 con 0x55d5c02b0370 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.496+0000 7f4ad0a4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54324/0 (socket says 172.21.15.16:54324) 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 -- 172.21.15.16:0/3716347772 learned_addr learned my addr 172.21.15.16:0/3716347772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad124d640 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a55d0 0x55d5c02a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 -- 172.21.15.16:0/3716347772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 msgr2=0x55d5c02c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 -- 172.21.15.16:0/3716347772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a55d0 msgr2=0x55d5c02a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a55d0 0x55d5c02a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 -- 172.21.15.16:0/3716347772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c02a0310 con 0x55d5c02a6b40 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a8660 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4abc000c00 tx=0x7f4abc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde0c37d67048abc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ab6ffd640 1 -- 172.21.15.16:0/3716347772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc016b80 con 0x55d5c02a6b40 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ab6ffd640 1 -- 172.21.15.16:0/3716347772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4abc016d20 con 0x55d5c02a6b40 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.497+0000 7f4ab6ffd640 1 -- 172.21.15.16:0/3716347772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc01c3d0 con 0x55d5c02a6b40 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 -- 172.21.15.16:0/3716347772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 msgr2=0x55d5c02a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a8660 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4abc000c00 tx=0x7f4abc008040 comp rx=0 tx=0).stop 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 -- 172.21.15.16:0/3716347772 shutdown_connections 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a6b40 0x55d5c02a8660 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 --2- 172.21.15.16:0/3716347772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a55d0 0x55d5c02a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 -- 172.21.15.16:0/3716347772 >> 172.21.15.16:0/3716347772 conn(0x55d5c026e310 msgr2=0x55d5c02a0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 -- 172.21.15.16:0/3716347772 shutdown_connections 2024-09-15T22:48:27.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 -- 172.21.15.16:0/3716347772 wait complete. 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 -- start start 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.498+0000 7f4ad588a740 1 Processor -- start 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a6b40 0x55d5c02b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad588a740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad588a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1af8f0 con 0x55d5c02a55d0 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad588a740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1b0d60 con 0x55d5c02a6b40 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad588a740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1affc0 con 0x55d5c02b0370 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad588a740 10 client.0 _ll_register_callbacks cb 0x7f4ac8007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54338/0 (socket says 172.21.15.16:54338) 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 -- 172.21.15.16:0/2476066733 learned_addr learned my addr 172.21.15.16:0/2476066733 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:27.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad1a4e640 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 -- 172.21.15.16:0/2476066733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 msgr2=0x55d5c02b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 -- 172.21.15.16:0/2476066733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a6b40 msgr2=0x55d5c02b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a6b40 0x55d5c02b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 -- 172.21.15.16:0/2476066733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c02adfd0 con 0x55d5c02a55d0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ad124d640 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02b8600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4ac809a140 tx=0x7f4ac809b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df05a7325c34a15a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.499+0000 7f4ab4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/2476066733 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac801d040 con 0x55d5c02a55d0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/2476066733 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f4ac809ad50 con 0x55d5c02a55d0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc016d20 con 0x55d5c02a55d0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/2476066733 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ac809b7f0 con 0x55d5c02a55d0 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 10 client.17694 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:27.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5c032b390 con 0x55d5c02a55d0 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/2476066733 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4abc016d20 con 0x55d5c02a55d0 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/2476066733 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac809af50 con 0x55d5c02a55d0 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 10 client.17694 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 20 client.17694 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c032bac0 0x55d5c032df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 4 client.17694 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5c0212d70 con 0x55d5c032bac0 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.500+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c032bac0 0x55d5c032df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad0a4c640 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c032bac0 0x55d5c032df40 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17ec452651af0ad2 server_cookie=86bd344cc476995d in_seq=0 out_seq=0 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ab4ff9640 10 client.17694 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ab4ff9640 1 -- 172.21.15.16:0/2476066733 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5c0212d70 con 0x55d5c032bac0 2024-09-15T22:48:27.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ab4ff9640 10 client.17694 handle_command_reply: tid=0 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 client.17694 shutdown 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 msgr2=0x55d5c02b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02b8600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4ac809a140 tx=0x7f4ac809b570 comp rx=0 tx=0).stop 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 shutdown_connections 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55d5c032bac0 0x55d5c032df40 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55d5c02b0370 0x55d5c02b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d5c02a6b40 0x55d5c02b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 --2- 172.21.15.16:0/2476066733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d5c02a55d0 0x55d5c02b8600 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 >> 172.21.15.16:0/2476066733 conn(0x55d5c026e310 msgr2=0x55d5c02ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ab4ff9640 0 client.17694 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 shutdown_connections 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 1 -- 172.21.15.16:0/2476066733 wait complete. 2024-09-15T22:48:27.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.501+0000 7f4ad588a740 20 client.17694 trim_cache size 0 max 16384 2024-09-15T22:48:27.510 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:48:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4ab80a33b0 msgr2=0x7f4ab80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4ab80a33b0 0x7f4ab80a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4048000 tx=0x7f4ac404b000 comp rx=0 tx=0).stop 2024-09-15T22:48:27.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc100a80 msgr2=0x7f4acc19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc100a80 0x7f4acc19a690 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c970 tx=0x7f4ac800ce30 comp rx=0 tx=0).stop 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 shutdown_connections 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4acc106a80 0x7f4acc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f4ab80a33b0 0x7f4ab80a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4acc101420 0x7f4acc19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 --2- 172.21.15.16:0/1140852691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4acc100a80 0x7f4acc19a690 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 >> 172.21.15.16:0/1140852691 conn(0x7f4acc0fc7a0 msgr2=0x7f4acc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 shutdown_connections 2024-09-15T22:48:27.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:27.502+0000 7f4ad34b1640 1 -- 172.21.15.16:0/1140852691 wait complete. 2024-09-15T22:48:29.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:28 smithi130 ceph-mon[35742]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:48:29.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:28 smithi016 ceph-mon[31492]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:48:29.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:28 smithi059 ceph-mon[34872]: pgmap v312: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:48:29.294 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:48:29.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:48:29.565 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:48:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 -- 172.21.15.16:0/2391059163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a8106f50 msgr2=0x7f46a810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/2391059163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a8106f50 0x7f46a810eaf0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009cd0 tx=0x7f46a002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 -- 172.21.15.16:0/2391059163 shutdown_connections 2024-09-15T22:48:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/2391059163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a8106f50 0x7f46a810eaf0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/2391059163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a8100ca0 0x7f46a8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/2391059163 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a81002f0 0x7f46a81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 -- 172.21.15.16:0/2391059163 >> 172.21.15.16:0/2391059163 conn(0x7f46a8077210 msgr2=0x7f46a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 -- 172.21.15.16:0/2391059163 shutdown_connections 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.024+0000 7f46ad5a4640 1 -- 172.21.15.16:0/2391059163 wait complete. 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 Processor -- start 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 -- start start 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a81002f0 0x7f46a8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a8100ca0 0x7f46a81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a8106f50 0x7f46a819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8111fa0 con 0x7f46a81002f0 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8111e20 con 0x7f46a8106f50 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46ad5a4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8112120 con 0x7f46a8100ca0 2024-09-15T22:48:30.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.025+0000 7f46a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a81002f0 0x7f46a8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a81002f0 0x7f46a8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54354/0 (socket says 172.21.15.16:54354) 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1492563732 learned_addr learned my addr 172.21.15.16:0/1492563732 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a8106f50 0x7f46a819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a8100ca0 0x7f46a81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1492563732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a8100ca0 msgr2=0x7f46a81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a8100ca0 0x7f46a81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1492563732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a8106f50 msgr2=0x7f46a819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a8106f50 0x7f46a819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 -- 172.21.15.16:0/1492563732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a819d410 con 0x7f46a81002f0 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a8106f50 0x7f46a819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46a6ffd640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a81002f0 0x7f46a8198460 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f469800c890 tx=0x7f469800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f468ffff640 1 -- 172.21.15.16:0/1492563732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698004220 con 0x7f46a81002f0 2024-09-15T22:48:30.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f468ffff640 1 -- 172.21.15.16:0/1492563732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46980043c0 con 0x7f46a81002f0 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a819d700 con 0x7f46a81002f0 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f468ffff640 1 -- 172.21.15.16:0/1492563732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46980049e0 con 0x7f46a81002f0 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.026+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a81a4fe0 con 0x7f46a81002f0 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f468ffff640 1 -- 172.21.15.16:0/1492563732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f46980079b0 con 0x7f46a81002f0 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 Processor -- start 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 -- start start 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61de800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d61e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6139280 con 0x55e5d61d3b60 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e5d60e0b10 con 0x55e5d61d4480 2024-09-15T22:48:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46af97d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e5d6135240 con 0x55e5d61ded40 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61d3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54364/0 (socket says 172.21.15.16:54364) 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46a77fe640 1 -- 172.21.15.16:0/1355864332 learned_addr learned my addr 172.21.15.16:0/1355864332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.029+0000 7f46a6ffd640 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d61e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a77fe640 1 -- 172.21.15.16:0/1355864332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 msgr2=0x55e5d61e1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d61e1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a77fe640 1 -- 172.21.15.16:0/1355864332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 msgr2=0x55e5d61de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61de800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a77fe640 1 -- 172.21.15.16:0/1355864332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d60ed1f0 con 0x55e5d61d3b60 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d61e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f468ffff640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46900a33b0 0x7f46900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a6ffd640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46900a33b0 0x7f46900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61d3f40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f46a00315f0 tx=0x7f46a0009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d1bedd814de63b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f468ffff640 1 -- 172.21.15.16:0/1492563732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f46980d0240 con 0x7f46a81002f0 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f468d7fa640 1 -- 172.21.15.16:0/1355864332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0031ac0 con 0x55e5d61d3b60 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f468d7fa640 1 -- 172.21.15.16:0/1355864332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a0036040 con 0x55e5d61d3b60 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.030+0000 7f468d7fa640 1 -- 172.21.15.16:0/1355864332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00305a0 con 0x55e5d61d3b60 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 -- 172.21.15.16:0/1355864332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 msgr2=0x55e5d61d3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61d3f40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f46a00315f0 tx=0x7f46a0009a20 comp rx=0 tx=0).stop 2024-09-15T22:48:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46a6ffd640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46900a33b0 0x7f46900a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55e5d61d0f50 tx=0x7f4698005300 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 -- 172.21.15.16:0/1355864332 shutdown_connections 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d61e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61de800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 --2- 172.21.15.16:0/1355864332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61d3f40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 -- 172.21.15.16:0/1355864332 >> 172.21.15.16:0/1355864332 conn(0x55e5d61d00f0 msgr2=0x55e5d61d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 -- 172.21.15.16:0/1355864332 shutdown_connections 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 -- 172.21.15.16:0/1355864332 wait complete. 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 -- start start 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.031+0000 7f46af97d740 1 Processor -- start 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46af97d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46af97d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d4480 0x55e5d61dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46af97d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 0x55e5d61dc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46af97d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e5d61e1910 con 0x55e5d61d3b60 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46af97d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e5d60f3f10 con 0x55e5d61ded40 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46af97d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55e5d61e1a90 con 0x55e5d61d4480 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46af97d740 10 client.0 _ll_register_callbacks cb 0x55e5d60bca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61dbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54380/0 (socket says 172.21.15.16:54380) 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 -- 172.21.15.16:0/1980845056 learned_addr learned my addr 172.21.15.16:0/1980845056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d4480 0x55e5d61dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 -- 172.21.15.16:0/1980845056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d4480 msgr2=0x55e5d61dc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d4480 0x55e5d61dc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 -- 172.21.15.16:0/1980845056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 msgr2=0x55e5d61dc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 0x55e5d61dc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 -- 172.21.15.16:0/1980845056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d5eaa6a0 con 0x55e5d61d3b60 2024-09-15T22:48:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61dbc20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f46a0004480 tx=0x7f46a0031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a97ad44adf1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.032+0000 7f468aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f468aff5640 1 -- 172.21.15.16:0/1980845056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0050070 con 0x55e5d61d3b60 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a0036040 con 0x55e5d61d3b60 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f468aff5640 1 -- 172.21.15.16:0/1980845056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f46a0003b80 con 0x55e5d61d3b60 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f468aff5640 1 -- 172.21.15.16:0/1980845056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46a005c050 con 0x55e5d61d3b60 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 10 client.17706 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5d61f1890 con 0x55e5d61d3b60 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f468aff5640 1 -- 172.21.15.16:0/1980845056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a0036040 con 0x55e5d61d3b60 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f468aff5640 1 -- 172.21.15.16:0/1980845056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f46a0030070 con 0x55e5d61d3b60 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 10 client.17706 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 20 client.17706 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d61f2be0 0x55e5d61f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 4 client.17706 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d61467f0 con 0x55e5d61f2be0 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.033+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d61f2be0 0x55e5d61f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.034+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d61f2be0 0x55e5d61f4fd0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c3bb08674bc0c3 server_cookie=2086577f259bec5f in_seq=0 out_seq=0 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.034+0000 7f468aff5640 10 client.17706 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f468aff5640 1 -- 172.21.15.16:0/1980845056 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5d61467f0 con 0x55e5d61f2be0 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f468aff5640 10 client.17706 handle_command_reply: tid=0 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 client.17706 shutdown 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 msgr2=0x55e5d61dbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61dbc20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f46a0004480 tx=0x7f46a0031af0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 shutdown_connections 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d61f2be0 0x55e5d61f4fd0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 0x55e5d61dc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d4480 0x55e5d61dc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f468aff5640 0 client.17706 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 --2- 172.21.15.16:0/1980845056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61dbc20 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 >> 172.21.15.16:0/1980845056 conn(0x55e5d61d00f0 msgr2=0x55e5d61d5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 shutdown_connections 2024-09-15T22:48:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 1 -- 172.21.15.16:0/1980845056 wait complete. 2024-09-15T22:48:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.035+0000 7f46af97d740 20 client.17706 trim_cache size 0 max 16384 2024-09-15T22:48:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.052+0000 7f46af97d740 1 Processor -- start 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46af97d740 1 -- start start 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46af97d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d3b60 0x55e5d61ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46af97d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d4480 0x55e5d61f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46af97d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 0x55e5d61f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46af97d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81af810 con 0x55e5d61d4480 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46af97d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81ad0e0 con 0x55e5d61ded40 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46af97d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81aaf50 con 0x55e5d61d3b60 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d4480 0x55e5d61f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d4480 0x55e5d61f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54390/0 (socket says 172.21.15.16:54390) 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 -- 172.21.15.16:0/1005197120 learned_addr learned my addr 172.21.15.16:0/1005197120 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d3b60 0x55e5d61ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a77fe640 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 0x55e5d61f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 -- 172.21.15.16:0/1005197120 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d3b60 msgr2=0x55e5d61ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d3b60 0x55e5d61ee880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 -- 172.21.15.16:0/1005197120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 msgr2=0x55e5d61f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 0x55e5d61f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 -- 172.21.15.16:0/1005197120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d61d42b0 con 0x55e5d61d4480 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f46a67fc640 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d4480 0x55e5d61f7ff0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f46a8104850 tx=0x7f469c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e346f8070395a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f468cff9640 1 -- 172.21.15.16:0/1005197120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c004220 con 0x55e5d61d4480 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.053+0000 7f468cff9640 1 -- 172.21.15.16:0/1005197120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469c0043c0 con 0x55e5d61d4480 2024-09-15T22:48:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f468cff9640 1 -- 172.21.15.16:0/1005197120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0035f0 con 0x55e5d61d4480 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 -- 172.21.15.16:0/1005197120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d4480 msgr2=0x55e5d61f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d4480 0x55e5d61f7ff0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f46a8104850 tx=0x7f469c00d340 comp rx=0 tx=0).stop 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 -- 172.21.15.16:0/1005197120 shutdown_connections 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61ded40 0x55e5d61f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d4480 0x55e5d61f7ff0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 --2- 172.21.15.16:0/1005197120 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61d3b60 0x55e5d61ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 -- 172.21.15.16:0/1005197120 >> 172.21.15.16:0/1005197120 conn(0x55e5d61d00f0 msgr2=0x55e5d61ee170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 -- 172.21.15.16:0/1005197120 shutdown_connections 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 -- 172.21.15.16:0/1005197120 wait complete. 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.054+0000 7f46af97d740 1 -- start start 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 1 Processor -- start 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d625c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81ab5a0 con 0x55e5d61d3b60 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81aca10 con 0x55e5d61d4480 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81abc70 con 0x55e5d61ded40 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46af97d740 10 client.0 _ll_register_callbacks cb 0x7f46a00314c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54392/0 (socket says 172.21.15.16:54392) 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 -- 172.21.15.16:0/2514845650 learned_addr learned my addr 172.21.15.16:0/2514845650 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a67fc640 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 -- 172.21.15.16:0/2514845650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 msgr2=0x55e5d625c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d625c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 -- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 msgr2=0x55e5d61f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 -- 172.21.15.16:0/2514845650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d625ca60 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a67fc640 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f46a77fe640 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61f6420 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b3a4a751321991 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.055+0000 7f468aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f468aff5640 1 -- 172.21.15.16:0/2514845650 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0050030 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f468aff5640 1 -- 172.21.15.16:0/2514845650 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f46a00305a0 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f469c0043c0 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f468aff5640 1 -- 172.21.15.16:0/2514845650 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46a0067050 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 10 client.17712 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5d625dd00 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f468aff5640 1 -- 172.21.15.16:0/2514845650 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469c0043c0 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f468aff5640 1 -- 172.21.15.16:0/2514845650 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f46a00464d0 con 0x55e5d61d3b60 2024-09-15T22:48:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 10 client.17712 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 20 client.17712 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d625e3d0 0x55e5d62607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 4 client.17712 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e5d6140d00 con 0x55e5d625e3d0 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.056+0000 7f46a67fc640 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d625e3d0 0x55e5d62607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.057+0000 7f46a67fc640 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d625e3d0 0x55e5d62607f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=282e998576085d9b server_cookie=5d98ef87d0c93beb in_seq=0 out_seq=0 2024-09-15T22:48:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.057+0000 7f468aff5640 10 client.17712 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f468aff5640 1 -- 172.21.15.16:0/2514845650 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e5d6140d00 con 0x55e5d625e3d0 2024-09-15T22:48:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f468aff5640 10 client.17712 handle_command_reply: tid=0 2024-09-15T22:48:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 client.17712 shutdown 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 msgr2=0x55e5d61f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61f6420 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a00095c0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 shutdown_connections 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55e5d625e3d0 0x55e5d62607f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55e5d61ded40 0x55e5d625c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f468aff5640 0 client.17712 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e5d61d4480 0x55e5d61f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 --2- 172.21.15.16:0/2514845650 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e5d61d3b60 0x55e5d61f6420 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 >> 172.21.15.16:0/2514845650 conn(0x55e5d61d00f0 msgr2=0x55e5d61ee990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 shutdown_connections 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 1 -- 172.21.15.16:0/2514845650 wait complete. 2024-09-15T22:48:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.058+0000 7f46af97d740 20 client.17712 trim_cache size 0 max 16384 2024-09-15T22:48:30.065 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:48:30.065 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-15T22:48:30.065 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "be196785-583c-40c9-8a77-38ccfd863919", 2024-09-15T22:48:30.065 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-15T22:48:30.065 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46900a33b0 msgr2=0x7f46900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46900a33b0 0x7f46900a5870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55e5d61d0f50 tx=0x7f4698005300 comp rx=0 tx=0).stop 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a81002f0 msgr2=0x7f46a8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a81002f0 0x7f46a8198460 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f469800c890 tx=0x7f469800cd50 comp rx=0 tx=0).stop 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 shutdown_connections 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a8106f50 0x7f46a819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46900a33b0 0x7f46900a5870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46a8100ca0 0x7f46a81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 --2- 172.21.15.16:0/1492563732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46a81002f0 0x7f46a8198460 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 >> 172.21.15.16:0/1492563732 conn(0x7f46a8077210 msgr2=0x7f46a8105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 shutdown_connections 2024-09-15T22:48:30.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:30.059+0000 7f46ad5a4640 1 -- 172.21.15.16:0/1492563732 wait complete. 2024-09-15T22:48:30.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:48:30.971 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:48:31.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:30 smithi130 ceph-mon[35742]: pgmap v313: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:30 smithi130 ceph-mon[35742]: scrub queued for path: / 2024-09-15T22:48:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:30 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:48:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:30 smithi130 ceph-mon[35742]: scrub summary: active paths [/] 2024-09-15T22:48:31.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:30 smithi130 ceph-mon[35742]: scrub summary: idle+waiting paths [/] 2024-09-15T22:48:31.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:30 smithi016 ceph-mon[31492]: pgmap v313: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:31.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:30 smithi016 ceph-mon[31492]: scrub queued for path: / 2024-09-15T22:48:31.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:30 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:48:31.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:30 smithi016 ceph-mon[31492]: scrub summary: active paths [/] 2024-09-15T22:48:31.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:30 smithi016 ceph-mon[31492]: scrub summary: idle+waiting paths [/] 2024-09-15T22:48:31.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:30 smithi059 ceph-mon[34872]: pgmap v313: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:31.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:30 smithi059 ceph-mon[34872]: scrub queued for path: / 2024-09-15T22:48:31.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:30 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:48:31.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:30 smithi059 ceph-mon[34872]: scrub summary: active paths [/] 2024-09-15T22:48:31.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:30 smithi059 ceph-mon[34872]: scrub summary: idle+waiting paths [/] 2024-09-15T22:48:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 -- 172.21.15.16:0/1618508870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 msgr2=0x7f8c3810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 --2- 172.21.15.16:0/1618508870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c3810ec40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009cd0 tx=0x7f8c3002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 -- 172.21.15.16:0/1618508870 shutdown_connections 2024-09-15T22:48:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 --2- 172.21.15.16:0/1618508870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c3810ec40 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 --2- 172.21.15.16:0/1618508870 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c381036e0 0x7f8c38107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 --2- 172.21.15.16:0/1618508870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c38102d30 0x7f8c38103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 -- 172.21.15.16:0/1618508870 >> 172.21.15.16:0/1618508870 conn(0x7f8c380fc7d0 msgr2=0x7f8c380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 -- 172.21.15.16:0/1618508870 shutdown_connections 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.471+0000 7f8c3c992640 1 -- 172.21.15.16:0/1618508870 wait complete. 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 Processor -- start 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 -- start start 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c38102d30 0x7f8c38100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c381036e0 0x7f8c38100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c38101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381120a0 con 0x7f8c381082a0 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38111f20 con 0x7f8c381036e0 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3c992640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8c38112220 con 0x7f8c38102d30 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c36d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c38101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c3659c640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c38102d30 0x7f8c38100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c36d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c38101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54424/0 (socket says 172.21.15.16:54424) 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c36d9d640 1 -- 172.21.15.16:0/888720095 learned_addr learned my addr 172.21.15.16:0/888720095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.472+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c381036e0 0x7f8c38100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c36d9d640 1 -- 172.21.15.16:0/888720095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c38102d30 msgr2=0x7f8c38100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c36d9d640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c38102d30 0x7f8c38100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c36d9d640 1 -- 172.21.15.16:0/888720095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c381036e0 msgr2=0x7f8c38100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c36d9d640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c381036e0 0x7f8c38100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c36d9d640 1 -- 172.21.15.16:0/888720095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c381a6a00 con 0x7f8c381082a0 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c3659c640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c38102d30 0x7f8c38100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c381036e0 0x7f8c38100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c36d9d640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c38101410 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005cc0 tx=0x7f8c30003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c277fe640 1 -- 172.21.15.16:0/888720095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30004040 con 0x7f8c381082a0 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c381a6c90 con 0x7f8c381082a0 2024-09-15T22:48:31.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c277fe640 1 -- 172.21.15.16:0/888720095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c300046c0 con 0x7f8c381082a0 2024-09-15T22:48:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c277fe640 1 -- 172.21.15.16:0/888720095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30046480 con 0x7f8c381082a0 2024-09-15T22:48:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.473+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c381a71d0 con 0x7f8c381082a0 2024-09-15T22:48:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.475+0000 7f8c277fe640 1 -- 172.21.15.16:0/888720095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8c300305a0 con 0x7f8c381082a0 2024-09-15T22:48:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.475+0000 7f8c3ed6b740 1 Processor -- start 2024-09-15T22:48:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3ed6b740 1 -- start start 2024-09-15T22:48:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b2d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2ddc0 0x55af29b38140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b38680 0x55af29b3aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af29a932e0 con 0x55af29b2d4a0 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55af29a3ace0 con 0x55af29b2ddc0 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55af29a8f2a0 con 0x55af29b38680 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b2d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b2d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54426/0 (socket says 172.21.15.16:54426) 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 -- 172.21.15.16:0/818472189 learned_addr learned my addr 172.21.15.16:0/818472189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c277fe640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8c200a33b0 0x7f8c200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c3659c640 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2ddc0 0x55af29b38140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b38680 0x55af29b3aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 -- 172.21.15.16:0/818472189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b38680 msgr2=0x55af29b3aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b38680 0x55af29b3aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 -- 172.21.15.16:0/818472189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2ddc0 msgr2=0x55af29b38140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2ddc0 0x55af29b38140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c35d9b640 1 -- 172.21.15.16:0/818472189 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af29a1f440 con 0x55af29b2d4a0 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.476+0000 7f8c36d9d640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8c200a33b0 0x7f8c200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.477+0000 7f8c277fe640 1 -- 172.21.15.16:0/888720095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f8c3002e030 con 0x7f8c381082a0 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.477+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b2d880 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x55af29b2adb0 tx=0x7f8c2c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d104a201cd9b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.477+0000 7f8c24ff9640 1 -- 172.21.15.16:0/818472189 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c004020 con 0x55af29b2d4a0 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.477+0000 7f8c24ff9640 1 -- 172.21.15.16:0/818472189 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c00edb0 con 0x55af29b2d4a0 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.477+0000 7f8c24ff9640 1 -- 172.21.15.16:0/818472189 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c005560 con 0x55af29b2d4a0 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/818472189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 msgr2=0x55af29b2d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b2d880 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x55af29b2adb0 tx=0x7f8c2c00bb10 comp rx=0 tx=0).stop 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c36d9d640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8c200a33b0 0x7f8c200a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c300bd4c0 tx=0x7f8c30043000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/818472189 shutdown_connections 2024-09-15T22:48:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b38680 0x55af29b3aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2ddc0 0x55af29b38140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/818472189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b2d880 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/818472189 >> 172.21.15.16:0/818472189 conn(0x55af29b29a30 msgr2=0x55af29b29e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/818472189 shutdown_connections 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/818472189 wait complete. 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- start start 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 Processor -- start 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b38040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b38680 0x55af29b358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55af29b3b250 con 0x55af29b2d4a0 2024-09-15T22:48:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55af29a4e050 con 0x55af29b38680 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.478+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x55af29b3b3d0 con 0x55af29b2ddc0 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c3ed6b740 10 client.0 _ll_register_callbacks cb 0x55af29a8e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b38040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b38040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54442/0 (socket says 172.21.15.16:54442) 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 -- 172.21.15.16:0/1166915123 learned_addr learned my addr 172.21.15.16:0/1166915123 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c3659c640 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b38680 0x55af29b358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 -- 172.21.15.16:0/1166915123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 msgr2=0x55af29b352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 -- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b38680 msgr2=0x55af29b358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b38680 0x55af29b358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 -- 172.21.15.16:0/1166915123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af298043c0 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b38040 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00b3d0 tx=0x7f8c2c005200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d245283a2badcb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c1dffb640 1 -- 172.21.15.16:0/1166915123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0054e0 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c1dffb640 1 -- 172.21.15.16:0/1166915123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f8c2c0046c0 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c2c00edb0 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c1dffb640 1 -- 172.21.15.16:0/1166915123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c0048c0 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.479+0000 7f8c3ed6b740 10 client.17724 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af29b46ca0 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c1dffb640 1 -- 172.21.15.16:0/1166915123 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c00edb0 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c1dffb640 1 -- 172.21.15.16:0/1166915123 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8c2c02c020 con 0x55af29b2d4a0 2024-09-15T22:48:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3ed6b740 10 client.17724 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3ed6b740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3ed6b740 20 client.17724 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29b47ff0 0x55af29b4a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3ed6b740 4 client.17724 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af29aa0850 con 0x55af29b47ff0 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.480+0000 7f8c3659c640 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29b47ff0 0x55af29b4a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.481+0000 7f8c3659c640 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29b47ff0 0x55af29b4a3e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ed5d6a056c8f75 server_cookie=d9f2c6a5bbd7ce09 in_seq=0 out_seq=0 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.481+0000 7f8c1dffb640 10 client.17724 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.481+0000 7f8c1dffb640 1 -- 172.21.15.16:0/1166915123 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af29aa0850 con 0x55af29b47ff0 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.481+0000 7f8c1dffb640 10 client.17724 handle_command_reply: tid=0 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.481+0000 7f8c3ed6b740 1 client.17724 shutdown 2024-09-15T22:48:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.481+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 msgr2=0x55af29b38040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.481+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b38040 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00b3d0 tx=0x7f8c2c005200 comp rx=0 tx=0).stop 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 shutdown_connections 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29b47ff0 0x55af29b4a3e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b38680 0x55af29b358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/1166915123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b2d4a0 0x55af29b38040 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 >> 172.21.15.16:0/1166915123 conn(0x55af29b29a30 msgr2=0x55af29b2f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c1dffb640 0 client.17724 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 shutdown_connections 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/1166915123 wait complete. 2024-09-15T22:48:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.482+0000 7f8c3ed6b740 20 client.17724 trim_cache size 0 max 16384 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.499+0000 7f8c3ed6b740 1 Processor -- start 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.499+0000 7f8c3ed6b740 1 -- start start 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b307f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29b4fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381b3a80 con 0x55af29b38680 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381b1350 con 0x55af29b2d4a0 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381af1c0 con 0x55af29b2ddc0 2024-09-15T22:48:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29b4fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29b4fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58680/0 (socket says 172.21.15.16:58680) 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 -- 172.21.15.16:0/476431848 learned_addr learned my addr 172.21.15.16:0/476431848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c3659c640 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 -- 172.21.15.16:0/476431848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 msgr2=0x55af29b307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 -- 172.21.15.16:0/476431848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 msgr2=0x55af29b444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 -- 172.21.15.16:0/476431848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af29b33840 con 0x55af29b38680 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29b4fb00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c001eb0 tx=0x7f8c2c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b05efc7d975058 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c1ffff640 1 -- 172.21.15.16:0/476431848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c038030 con 0x55af29b38680 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c1ffff640 1 -- 172.21.15.16:0/476431848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c039040 con 0x55af29b38680 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.500+0000 7f8c1ffff640 1 -- 172.21.15.16:0/476431848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c005560 con 0x55af29b38680 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/476431848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 msgr2=0x55af29b4fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29b4fb00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c001eb0 tx=0x7f8c2c012670 comp rx=0 tx=0).stop 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/476431848 shutdown_connections 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29b4fb00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/476431848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/476431848 >> 172.21.15.16:0/476431848 conn(0x55af29b29a30 msgr2=0x55af29b300e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/476431848 shutdown_connections 2024-09-15T22:48:31.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/476431848 wait complete. 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.501+0000 7f8c3ed6b740 1 -- start start 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 1 Processor -- start 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b3bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b3c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29bb0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381af810 con 0x55af29b38680 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381b0c80 con 0x55af29b2d4a0 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f8c381afee0 con 0x55af29b2ddc0 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3ed6b740 10 client.0 _ll_register_callbacks cb 0x7f8c2c012700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29bb0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29bb0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58682/0 (socket says 172.21.15.16:58682) 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 -- 172.21.15.16:0/2553671403 learned_addr learned my addr 172.21.15.16:0/2553671403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c3659c640 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b3c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b3bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 -- 172.21.15.16:0/2553671403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 msgr2=0x55af29b3c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b3c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 -- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 msgr2=0x55af29b3bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b3bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.502+0000 7f8c35d9b640 1 -- 172.21.15.16:0/2553671403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af29bb1100 con 0x55af29b38680 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c35d9b640 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29bb0ac0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00b4e0 tx=0x7f8c2c0122b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dff77ab26161cf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c1dffb640 1 -- 172.21.15.16:0/2553671403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c00e720 con 0x55af29b38680 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c1dffb640 1 -- 172.21.15.16:0/2553671403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f8c2c004a60 con 0x55af29b38680 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c2c039040 con 0x55af29b38680 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c1dffb640 1 -- 172.21.15.16:0/2553671403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c00ed90 con 0x55af29b38680 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 10 client.17730 fetch_fsmap learned FSMap version 26 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af29bb23f0 con 0x55af29b38680 2024-09-15T22:48:31.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c1dffb640 1 -- 172.21.15.16:0/2553671403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c2c039040 con 0x55af29b38680 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c1dffb640 1 -- 172.21.15.16:0/2553671403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8c2c03e020 con 0x55af29b38680 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 10 client.17730 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 20 client.17730 populate_metadata read hostname 'smithi016' 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29bb2a00 0x55af29bb4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 4 client.17730 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af29a9ad60 con 0x55af29bb2a00 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.503+0000 7f8c3659c640 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29bb2a00 0x55af29bb4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3659c640 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29bb2a00 0x55af29bb4df0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3254a77fde33391 server_cookie=65311f59633f4ac4 in_seq=0 out_seq=0 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c1dffb640 10 client.17730 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c1dffb640 1 -- 172.21.15.16:0/2553671403 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55af29a9ad60 con 0x55af29bb2a00 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c1dffb640 10 client.17730 handle_command_reply: tid=0 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 client.17730 shutdown 2024-09-15T22:48:31.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 msgr2=0x55af29bb0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29bb0ac0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00b4e0 tx=0x7f8c2c0122b0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 shutdown_connections 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x55af29bb2a00 0x55af29bb4df0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55af29b38680 0x55af29bb0ac0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x55af29b2ddc0 0x55af29b3c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 --2- 172.21.15.16:0/2553671403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55af29b2d4a0 0x55af29b3bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c1dffb640 0 client.17730 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 >> 172.21.15.16:0/2553671403 conn(0x55af29b29a30 msgr2=0x55af29b28b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 shutdown_connections 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 1 -- 172.21.15.16:0/2553671403 wait complete. 2024-09-15T22:48:31.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.504+0000 7f8c3ed6b740 20 client.17730 trim_cache size 0 max 16384 2024-09-15T22:48:31.522 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:48:31.522 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:48:31.522 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-15T22:48:31.522 INFO:teuthology.orchestra.run.smithi016.stdout: "be196785-583c-40c9-8a77-38ccfd863919": { 2024-09-15T22:48:31.522 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-15T22:48:31.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "be196785-583c-40c9-8a77-38ccfd863919", 2024-09-15T22:48:31.522 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-15T22:48:31.523 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:48:31.523 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:48:31.523 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:48:31.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8c200a33b0 msgr2=0x7f8c200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8c200a33b0 0x7f8c200a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c300bd4c0 tx=0x7f8c30043000 comp rx=0 tx=0).stop 2024-09-15T22:48:31.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 msgr2=0x7f8c38101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:31.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c38101410 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005cc0 tx=0x7f8c30003970 comp rx=0 tx=0).stop 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 shutdown_connections 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c381082a0 0x7f8c38101410 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c381036e0 0x7f8c38100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f8c200a33b0 0x7f8c200a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 --2- 172.21.15.16:0/888720095 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8c38102d30 0x7f8c38100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 >> 172.21.15.16:0/888720095 conn(0x7f8c380fc7d0 msgr2=0x7f8c38105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 shutdown_connections 2024-09-15T22:48:31.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:31.505+0000 7f8c3c992640 1 -- 172.21.15.16:0/888720095 wait complete. 2024-09-15T22:48:32.192 INFO:tasks.cephfs.filesystem:scrub status for tag:be196785-583c-40c9-8a77-38ccfd863919 - {'path': '/', 'tag': 'be196785-583c-40c9-8a77-38ccfd863919', 'options': 'recursive,force'} 2024-09-15T22:48:33.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:32 smithi130 ceph-mon[35742]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:48:33.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:32 smithi016 ceph-mon[31492]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:48:33.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:32 smithi059 ceph-mon[34872]: pgmap v314: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-09-15T22:48:34.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:33 smithi130 ceph-mon[35742]: scrub summary: idle 2024-09-15T22:48:34.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:33 smithi016 ceph-mon[31492]: scrub summary: idle 2024-09-15T22:48:34.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:33 smithi059 ceph-mon[34872]: scrub summary: idle 2024-09-15T22:48:34.995 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:34 smithi016 ceph-mon[31492]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:48:34.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-15T22:48:35.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:34 smithi130 ceph-mon[35742]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:48:35.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 -- 172.21.15.16:0/2860980357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401035b0 msgr2=0x7ff14010ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 --2- 172.21.15.16:0/2860980357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401035b0 0x7ff14010ba90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff134009900 tx=0x7ff13402a480 comp rx=0 tx=0).stop 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 -- 172.21.15.16:0/2860980357 shutdown_connections 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 --2- 172.21.15.16:0/2860980357 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff14010c0e0 0x7ff14010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 --2- 172.21.15.16:0/2860980357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401035b0 0x7ff14010ba90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 --2- 172.21.15.16:0/2860980357 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff140102c90 0x7ff140103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 -- 172.21.15.16:0/2860980357 >> 172.21.15.16:0/2860980357 conn(0x7ff1400fc700 msgr2=0x7ff1400feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 -- 172.21.15.16:0/2860980357 shutdown_connections 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 -- 172.21.15.16:0/2860980357 wait complete. 2024-09-15T22:48:35.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.114+0000 7ff145c1e640 1 Processor -- start 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff145c1e640 1 -- start start 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff145c1e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff140102c90 0x7ff140198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff145c1e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1401035b0 0x7ff1401988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff145c1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14010c0e0 0x7ff14019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff145c1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff140111f10 con 0x7ff14010c0e0 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff145c1e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff140111d90 con 0x7ff1401035b0 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff145c1e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff140112090 con 0x7ff140102c90 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14010c0e0 0x7ff14019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14010c0e0 0x7ff14019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58698/0 (socket says 172.21.15.16:58698) 2024-09-15T22:48:35.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 -- 172.21.15.16:0/4103871872 learned_addr learned my addr 172.21.15.16:0/4103871872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13f7fe640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff140102c90 0x7ff140198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13effd640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1401035b0 0x7ff1401988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 -- 172.21.15.16:0/4103871872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff140102c90 msgr2=0x7ff140198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff140102c90 0x7ff140198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 -- 172.21.15.16:0/4103871872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1401035b0 msgr2=0x7ff1401988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1401035b0 0x7ff1401988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.115+0000 7ff13ffff640 1 -- 172.21.15.16:0/4103871872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14019d370 con 0x7ff14010c0e0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.116+0000 7ff13ffff640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14010c0e0 0x7ff14019cc40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff13800eb10 tx=0x7ff13800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.116+0000 7ff13cff9640 1 -- 172.21.15.16:0/4103871872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138004b20 con 0x7ff14010c0e0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.116+0000 7ff13cff9640 1 -- 172.21.15.16:0/4103871872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff138004cc0 con 0x7ff14010c0e0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.116+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14019d660 con 0x7ff14010c0e0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.116+0000 7ff13cff9640 1 -- 172.21.15.16:0/4103871872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138010760 con 0x7ff14010c0e0 2024-09-15T22:48:35.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.116+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1401a4ef0 con 0x7ff14010c0e0 2024-09-15T22:48:35.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.117+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff140103070 con 0x7ff14010c0e0 2024-09-15T22:48:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.118+0000 7ff13cff9640 1 -- 172.21.15.16:0/4103871872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff138005020 con 0x7ff14010c0e0 2024-09-15T22:48:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.119+0000 7ff13cff9640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff1280a3460 0x7ff1280a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.119+0000 7ff13f7fe640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff1280a3460 0x7ff1280a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.119+0000 7ff13cff9640 1 -- 172.21.15.16:0/4103871872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7ff1380d0620 con 0x7ff14010c0e0 2024-09-15T22:48:35.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.119+0000 7ff13f7fe640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff1280a3460 0x7ff1280a5920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff130005fd0 tx=0x7ff13000b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:35.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.123+0000 7ff13cff9640 1 -- 172.21.15.16:0/4103871872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff13809cbf0 con 0x7ff14010c0e0 2024-09-15T22:48:35.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:34 smithi059 ceph-mon[34872]: pgmap v315: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:48:35.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.321+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff1400fea20 con 0x7ff1280a3460 2024-09-15T22:48:35.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.329+0000 7ff13cff9640 1 -- 172.21.15.16:0/4103871872 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7ff1400fea20 con 0x7ff1280a3460 2024-09-15T22:48:35.330 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-15T00:00:00", "created": "2024-09-15T22:38:33", "first": "2024-09-15T22:39:00", "last": "2024-09-15T22:48:00", "last_pruned": "2024-09-15T22:48:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-15T22:48:35.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff1280a3460 msgr2=0x7ff1280a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff1280a3460 0x7ff1280a5920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff130005fd0 tx=0x7ff13000b390 comp rx=0 tx=0).stop 2024-09-15T22:48:35.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14010c0e0 msgr2=0x7ff14019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14010c0e0 0x7ff14019cc40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff13800eb10 tx=0x7ff13800c2d0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 shutdown_connections 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff14010c0e0 0x7ff14019cc40 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1401035b0 0x7ff1401988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff1280a3460 0x7ff1280a5920 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 --2- 172.21.15.16:0/4103871872 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff140102c90 0x7ff140198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 >> 172.21.15.16:0/4103871872 conn(0x7ff1400fc700 msgr2=0x7ff1400fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 shutdown_connections 2024-09-15T22:48:35.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.331+0000 7ff145c1e640 1 -- 172.21.15.16:0/4103871872 wait complete. 2024-09-15T22:48:35.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-15T22:48:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 -- 172.21.15.16:0/1726294539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578102480 msgr2=0x7f857810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 --2- 172.21.15.16:0/1726294539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578102480 0x7f857810e9b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8570009cd0 tx=0x7f857002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 -- 172.21.15.16:0/1726294539 shutdown_connections 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 --2- 172.21.15.16:0/1726294539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578102480 0x7f857810e9b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 --2- 172.21.15.16:0/1726294539 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8578101ae0 0x7f8578101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 --2- 172.21.15.16:0/1726294539 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8578107b80 0x7f8578107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 -- 172.21.15.16:0/1726294539 >> 172.21.15.16:0/1726294539 conn(0x7f85780fd7e0 msgr2=0x7f85780ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 -- 172.21.15.16:0/1726294539 shutdown_connections 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.447+0000 7f857ced2640 1 -- 172.21.15.16:0/1726294539 wait complete. 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 Processor -- start 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 -- start start 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8578101ae0 0x7f8578198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8578102480 0x7f85781987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578107b80 0x7f857819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8578111f30 con 0x7f8578107b80 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8578111db0 con 0x7f8578101ae0 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857ced2640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f85781120b0 con 0x7f8578102480 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578107b80 0x7f857819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f8576d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578107b80 0x7f857819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58716/0 (socket says 172.21.15.16:58716) 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f8576d9d640 1 -- 172.21.15.16:0/1693437180 learned_addr learned my addr 172.21.15.16:0/1693437180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.448+0000 7f857659c640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8578101ae0 0x7f8578198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8575d9b640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8578102480 0x7f85781987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8576d9d640 1 -- 172.21.15.16:0/1693437180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8578102480 msgr2=0x7f85781987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8576d9d640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8578102480 0x7f85781987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8576d9d640 1 -- 172.21.15.16:0/1693437180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8578101ae0 msgr2=0x7f8578198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8576d9d640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8578101ae0 0x7f8578198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8576d9d640 1 -- 172.21.15.16:0/1693437180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857819d280 con 0x7f8578107b80 2024-09-15T22:48:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8575d9b640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8578102480 0x7f85781987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f8576d9d640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578107b80 0x7f857819cb50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8570006010 tx=0x7f8570003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f856f7fe640 1 -- 172.21.15.16:0/1693437180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570004450 con 0x7f8578107b80 2024-09-15T22:48:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857819d510 con 0x7f8578107b80 2024-09-15T22:48:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f856f7fe640 1 -- 172.21.15.16:0/1693437180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8570003e10 con 0x7f8578107b80 2024-09-15T22:48:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f856f7fe640 1 -- 172.21.15.16:0/1693437180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570046480 con 0x7f8578107b80 2024-09-15T22:48:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.449+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85781a4e00 con 0x7f8578107b80 2024-09-15T22:48:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.450+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8578101f40 con 0x7f8578107b80 2024-09-15T22:48:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.451+0000 7f856f7fe640 1 -- 172.21.15.16:0/1693437180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f85700045f0 con 0x7f8578107b80 2024-09-15T22:48:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.452+0000 7f856f7fe640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f85640a3460 0x7f85640a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.452+0000 7f857659c640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f85640a3460 0x7f85640a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.452+0000 7f856f7fe640 1 -- 172.21.15.16:0/1693437180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f857002e030 con 0x7f8578107b80 2024-09-15T22:48:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.453+0000 7f857659c640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f85640a3460 0x7f85640a5920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f856800ab60 tx=0x7f856800a2d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.456+0000 7f856f7fe640 1 -- 172.21.15.16:0/1693437180 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f857004b060 con 0x7f8578107b80 2024-09-15T22:48:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.651+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 --> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f85780ff4d0 con 0x7f85640a3460 2024-09-15T22:48:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.658+0000 7f856f7fe640 1 -- 172.21.15.16:0/1693437180 <== mgr.15234 v2:172.21.15.16:6800/1157292381 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f85780ff4d0 con 0x7f85640a3460 2024-09-15T22:48:35.659 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-09-15T22:48:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f85640a3460 msgr2=0x7f85640a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f85640a3460 0x7f85640a5920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f856800ab60 tx=0x7f856800a2d0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578107b80 msgr2=0x7f857819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578107b80 0x7f857819cb50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8570006010 tx=0x7f8570003c20 comp rx=0 tx=0).stop 2024-09-15T22:48:35.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 shutdown_connections 2024-09-15T22:48:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8578107b80 0x7f857819cb50 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f85640a3460 0x7f85640a5920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f8578102480 0x7f85781987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 --2- 172.21.15.16:0/1693437180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8578101ae0 0x7f8578198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 >> 172.21.15.16:0/1693437180 conn(0x7f85780fd7e0 msgr2=0x7f85780fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 shutdown_connections 2024-09-15T22:48:35.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:48:35.660+0000 7f857ced2640 1 -- 172.21.15.16:0/1693437180 wait complete. 2024-09-15T22:48:35.672 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-15T22:48:35.683 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-15T22:48:37.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:36 smithi130 ceph-mon[35742]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:48:37.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:36 smithi130 ceph-mon[35742]: from='client.17736 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:48:37.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:36 smithi130 ceph-mon[35742]: from='client.17742 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:48:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:36 smithi016 ceph-mon[31492]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:48:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:36 smithi016 ceph-mon[31492]: from='client.17736 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:48:37.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:36 smithi016 ceph-mon[31492]: from='client.17742 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:48:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:36 smithi059 ceph-mon[34872]: pgmap v316: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:48:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:36 smithi059 ceph-mon[34872]: from='client.17736 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:48:37.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:36 smithi059 ceph-mon[34872]: from='client.17742 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:48:39.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:38 smithi130 ceph-mon[35742]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-15T22:48:39.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:38 smithi016 ceph-mon[31492]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-15T22:48:39.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:38 smithi059 ceph-mon[34872]: pgmap v317: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-15T22:48:39.991 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:39 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:48:40.001 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:39 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:48:40.003 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:39 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:48:41.028 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:40 smithi130 ceph-mon[35742]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:41.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:40 smithi016 ceph-mon[31492]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:41.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:40 smithi059 ceph-mon[34872]: pgmap v318: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:48:42.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:48:42.280 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:48:42.308 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:48:43.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:42 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi130 to 176.0M 2024-09-15T22:48:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:42 smithi059 ceph-mon[34872]: Unable to set osd_memory_target on smithi130 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:48:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:42 smithi059 ceph-mon[34872]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-15T22:48:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:42 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-15T22:48:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:42 smithi059 ceph-mon[34872]: Unable to set osd_memory_target on smithi016 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-15T22:48:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:42 smithi059 ceph-mon[34872]: Adjusting osd_memory_target on smithi059 to 177.0M 2024-09-15T22:48:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:42 smithi059 ceph-mon[34872]: Unable to set osd_memory_target on smithi059 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-15T22:48:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:42 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi130 to 176.0M 2024-09-15T22:48:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:42 smithi130 ceph-mon[35742]: Unable to set osd_memory_target on smithi130 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:48:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:42 smithi130 ceph-mon[35742]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-15T22:48:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:42 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-15T22:48:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:42 smithi130 ceph-mon[35742]: Unable to set osd_memory_target on smithi016 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-15T22:48:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:42 smithi130 ceph-mon[35742]: Adjusting osd_memory_target on smithi059 to 177.0M 2024-09-15T22:48:43.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:42 smithi130 ceph-mon[35742]: Unable to set osd_memory_target on smithi059 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-15T22:48:43.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:42 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi130 to 176.0M 2024-09-15T22:48:43.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:42 smithi016 ceph-mon[31492]: Unable to set osd_memory_target on smithi130 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:48:43.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:42 smithi016 ceph-mon[31492]: pgmap v319: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-15T22:48:43.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:42 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-15T22:48:43.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:42 smithi016 ceph-mon[31492]: Unable to set osd_memory_target on smithi016 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-15T22:48:43.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:42 smithi016 ceph-mon[31492]: Adjusting osd_memory_target on smithi059 to 177.0M 2024-09-15T22:48:43.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:42 smithi016 ceph-mon[31492]: Unable to set osd_memory_target on smithi059 to 185699942: error parsing value: Value '185699942' is below minimum 939524096 2024-09-15T22:48:45.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:44 smithi059 ceph-mon[34872]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:45.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:44 smithi130 ceph-mon[35742]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:45.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:44 smithi016 ceph-mon[31492]: pgmap v320: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:47.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:46 smithi059 ceph-mon[34872]: pgmap v321: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:47.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:46 smithi130 ceph-mon[35742]: pgmap v321: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:47.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:46 smithi016 ceph-mon[31492]: pgmap v321: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:48:49.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:48 smithi059 ceph-mon[34872]: pgmap v322: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-15T22:48:49.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:48 smithi130 ceph-mon[35742]: pgmap v322: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-15T22:48:49.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:48 smithi016 ceph-mon[31492]: pgmap v322: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-15T22:48:51.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:50 smithi059 ceph-mon[34872]: pgmap v323: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:51.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:50 smithi130 ceph-mon[35742]: pgmap v323: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:51.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:50 smithi016 ceph-mon[31492]: pgmap v323: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:53.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:52 smithi059 ceph-mon[34872]: pgmap v324: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:48:53.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:52 smithi130 ceph-mon[35742]: pgmap v324: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:48:53.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:52 smithi016 ceph-mon[31492]: pgmap v324: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:48:55.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:54 smithi059 ceph-mon[34872]: pgmap v325: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:55.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:54 smithi130 ceph-mon[35742]: pgmap v325: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:55.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:54 smithi016 ceph-mon[31492]: pgmap v325: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:57.196 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:56 smithi059 ceph-mon[34872]: pgmap v326: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:57.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:56 smithi130 ceph-mon[35742]: pgmap v326: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:57.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:56 smithi016 ceph-mon[31492]: pgmap v326: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:48:59.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:48:58 smithi059 ceph-mon[34872]: pgmap v327: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:48:59.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:48:58 smithi130 ceph-mon[35742]: pgmap v327: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:48:59.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:48:58 smithi016 ceph-mon[31492]: pgmap v327: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:49:01.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:00 smithi059 ceph-mon[34872]: pgmap v328: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:01.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:00 smithi130 ceph-mon[35742]: pgmap v328: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:01.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:00 smithi016 ceph-mon[31492]: pgmap v328: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:02.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:49:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 -- 172.21.15.16:0/2577975408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 msgr2=0x7f23341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 --2- 172.21.15.16:0/2577975408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f23341067e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f231c009900 tx=0x7f231c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 -- 172.21.15.16:0/2577975408 shutdown_connections 2024-09-15T22:49:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 --2- 172.21.15.16:0/2577975408 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2334106f10 0x7f233410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 --2- 172.21.15.16:0/2577975408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f23341067e0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 --2- 172.21.15.16:0/2577975408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2334101de0 0x7f23341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 -- 172.21.15.16:0/2577975408 >> 172.21.15.16:0/2577975408 conn(0x7f23340fd8c0 msgr2=0x7f23340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 -- 172.21.15.16:0/2577975408 shutdown_connections 2024-09-15T22:49:02.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.926+0000 7f233920e640 1 -- 172.21.15.16:0/2577975408 wait complete. 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.927+0000 7f233920e640 1 Processor -- start 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.927+0000 7f233920e640 1 -- start start 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233920e640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2334101de0 0x7f23341983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233920e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f2334198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233920e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2334106f10 0x7f233419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233920e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2334111f70 con 0x7f2334102790 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233920e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2334111df0 con 0x7f2334106f10 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233920e640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23341120f0 con 0x7f2334101de0 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f2334198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f2334198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51224/0 (socket says 172.21.15.16:51224) 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 -- 172.21.15.16:0/3137588932 learned_addr learned my addr 172.21.15.16:0/3137588932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:02.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f2332d9d640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2334101de0 0x7f23341983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233359e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2334106f10 0x7f233419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 -- 172.21.15.16:0/3137588932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2334101de0 msgr2=0x7f23341983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2334101de0 0x7f23341983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 -- 172.21.15.16:0/3137588932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2334106f10 msgr2=0x7f233419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2334106f10 0x7f233419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 -- 172.21.15.16:0/3137588932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233419d380 con 0x7f2334102790 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.928+0000 7f233259c640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f2334198910 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f231c005ab0 tx=0x7f231c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.929+0000 7f232bfff640 1 -- 172.21.15.16:0/3137588932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c03f030 con 0x7f2334102790 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.929+0000 7f232bfff640 1 -- 172.21.15.16:0/3137588932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f231c003ce0 con 0x7f2334102790 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.929+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233419d610 con 0x7f2334102790 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.929+0000 7f232bfff640 1 -- 172.21.15.16:0/3137588932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c035070 con 0x7f2334102790 2024-09-15T22:49:02.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.929+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23341a4f50 con 0x7f2334102790 2024-09-15T22:49:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.931+0000 7f232bfff640 1 -- 172.21.15.16:0/3137588932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f231c042030 con 0x7f2334102790 2024-09-15T22:49:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.931+0000 7f233b5e7740 1 Processor -- start 2024-09-15T22:49:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.931+0000 7f233b5e7740 1 -- start start 2024-09-15T22:49:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107d3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233b5e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a107392e0 con 0x561a107d3cc0 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233b5e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a106e0ce0 con 0x561a107e2a90 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233b5e7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561a107352a0 con 0x561a107d33a0 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f2332d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f2332d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51230/0 (socket says 172.21.15.16:51230) 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f2332d9d640 1 -- 172.21.15.16:0/1558874183 learned_addr learned my addr 172.21.15.16:0/1558874183 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233359e640 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107d3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233259c640 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f232bfff640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23200a3430 0x7f23200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233359e640 1 -- 172.21.15.16:0/1558874183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 msgr2=0x561a107e4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233359e640 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107e4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233359e640 1 -- 172.21.15.16:0/1558874183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 msgr2=0x561a107e2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233359e640 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f233359e640 1 -- 172.21.15.16:0/1558874183 --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a106c5440 con 0x561a107d33a0 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.932+0000 7f2332d9d640 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.933+0000 7f233359e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23200a3430 0x7f23200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.933+0000 7f232bfff640 1 -- 172.21.15.16:0/3137588932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f231c0f0a30 con 0x7f2334102790 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.933+0000 7f233359e640 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107d3780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2324007e50 tx=0x7f232400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebde597af64ab51 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.933+0000 7f23297fa640 1 -- 172.21.15.16:0/1558874183 <== mon.1 v2:172.21.15.130:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2324022070 con 0x561a107d33a0 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.933+0000 7f23297fa640 1 -- 172.21.15.16:0/1558874183 <== mon.1 v2:172.21.15.130:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232401c470 con 0x561a107d33a0 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.933+0000 7f23297fa640 1 -- 172.21.15.16:0/1558874183 <== mon.1 v2:172.21.15.130:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232400ee30 con 0x561a107d33a0 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.933+0000 7f233359e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23200a3430 0x7f23200a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f232401f000 tx=0x7f2324020000 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 -- 172.21.15.16:0/1558874183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 msgr2=0x561a107d3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107d3780 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2324007e50 tx=0x7f232400e720 comp rx=0 tx=0).stop 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 -- 172.21.15.16:0/1558874183 shutdown_connections 2024-09-15T22:49:02.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1558874183 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107d3780 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 -- 172.21.15.16:0/1558874183 >> 172.21.15.16:0/1558874183 conn(0x561a107cfa30 msgr2=0x561a107cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 -- 172.21.15.16:0/1558874183 shutdown_connections 2024-09-15T22:49:02.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 -- 172.21.15.16:0/1558874183 wait complete. 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 -- start start 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.934+0000 7f233b5e7740 1 Processor -- start 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233b5e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561a107e56c0 con 0x561a107d3cc0 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233b5e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561a106f4050 con 0x561a107e2a90 2024-09-15T22:49:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233b5e7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x561a107e5840 con 0x561a107d33a0 2024-09-15T22:49:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233b5e7740 10 client.0 _ll_register_callbacks cb 0x561a10734160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107df6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51246/0 (socket says 172.21.15.16:51246) 2024-09-15T22:49:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 -- 172.21.15.16:0/2234281124 learned_addr learned my addr 172.21.15.16:0/2234281124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f2332d9d640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233359e640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 -- 172.21.15.16:0/2234281124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 msgr2=0x561a107e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 -- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 msgr2=0x561a107dfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107dfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 -- 172.21.15.16:0/2234281124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a104aa3c0 con 0x561a107d3cc0 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f2332d9d640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233359e640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107dfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f233259c640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107df6c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f231c000c00 tx=0x7f231c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baea4daced7cc7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.935+0000 7f231a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f231a7fc640 1 -- 172.21.15.16:0/2234281124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c049030 con 0x561a107d3cc0 2024-09-15T22:49:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f231a7fc640 1 -- 172.21.15.16:0/2234281124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f231c0b8f70 con 0x561a107d3cc0 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f232401c470 con 0x561a107d3cc0 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f231a7fc640 1 -- 172.21.15.16:0/2234281124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f231c0bc440 con 0x561a107d3cc0 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 10 client.17754 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a107f10f0 con 0x561a107d3cc0 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f231a7fc640 1 -- 172.21.15.16:0/2234281124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f232401c470 con 0x561a107d3cc0 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f231a7fc640 1 -- 172.21.15.16:0/2234281124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f231c0edff0 con 0x561a107d3cc0 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 10 client.17754 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 20 client.17754 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a107f25f0 0x561a107f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 4 client.17754 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a10746850 con 0x561a107f25f0 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.936+0000 7f2332d9d640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a107f25f0 0x561a107f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.937+0000 7f2332d9d640 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a107f25f0 0x561a107f49e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4afc6d20da6779 server_cookie=8e939af0bef638cf in_seq=0 out_seq=0 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.937+0000 7f231a7fc640 10 client.17754 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.937+0000 7f231a7fc640 1 -- 172.21.15.16:0/2234281124 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a10746850 con 0x561a107f25f0 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f231a7fc640 10 client.17754 handle_command_reply: tid=0 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 client.17754 shutdown 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 msgr2=0x561a107df6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107df6c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f231c000c00 tx=0x7f231c034000 comp rx=0 tx=0).stop 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 shutdown_connections 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a107f25f0 0x561a107f49e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a107dfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107df6c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 --2- 172.21.15.16:0/2234281124 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f231a7fc640 0 client.17754 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 >> 172.21.15.16:0/2234281124 conn(0x561a107cfa30 msgr2=0x561a107d5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 shutdown_connections 2024-09-15T22:49:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 1 -- 172.21.15.16:0/2234281124 wait complete. 2024-09-15T22:49:02.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.938+0000 7f233b5e7740 20 client.17754 trim_cache size 0 max 16384 2024-09-15T22:49:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 Processor -- start 2024-09-15T22:49:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 -- start start 2024-09-15T22:49:02.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107d33a0 0x561a107ee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107e2a90 0x561a107fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23341af5c0 con 0x561a107d3cc0 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23341ace90 con 0x561a107d33a0 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.955+0000 7f233b5e7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23341aad00 con 0x561a107e2a90 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51262/0 (socket says 172.21.15.16:51262) 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 -- 172.21.15.16:0/87336128 learned_addr learned my addr 172.21.15.16:0/87336128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f2332d9d640 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107d33a0 0x561a107ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233359e640 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107e2a90 0x561a107fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 -- 172.21.15.16:0/87336128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107e2a90 msgr2=0x561a107fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107e2a90 0x561a107fa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 -- 172.21.15.16:0/87336128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107d33a0 msgr2=0x561a107ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107d33a0 0x561a107ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 -- 172.21.15.16:0/87336128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a107d3af0 con 0x561a107d3cc0 2024-09-15T22:49:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233359e640 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107e2a90 0x561a107fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:49:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f233259c640 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e1630 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5570 tx=0x7f231c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e448afb6aef0ea8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f2328ff9640 1 -- 172.21.15.16:0/87336128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0bc260 con 0x561a107d3cc0 2024-09-15T22:49:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f2328ff9640 1 -- 172.21.15.16:0/87336128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f231c0bd000 con 0x561a107d3cc0 2024-09-15T22:49:02.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.956+0000 7f2328ff9640 1 -- 172.21.15.16:0/87336128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c0c50e0 con 0x561a107d3cc0 2024-09-15T22:49:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 -- 172.21.15.16:0/87336128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 msgr2=0x561a107e1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e1630 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5570 tx=0x7f231c005d50 comp rx=0 tx=0).stop 2024-09-15T22:49:02.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 -- 172.21.15.16:0/87336128 shutdown_connections 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107e2a90 0x561a107fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e1630 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 --2- 172.21.15.16:0/87336128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107d33a0 0x561a107ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 -- 172.21.15.16:0/87336128 >> 172.21.15.16:0/87336128 conn(0x561a107cfa30 msgr2=0x561a107e0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 -- 172.21.15.16:0/87336128 shutdown_connections 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 -- 172.21.15.16:0/87336128 wait complete. 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.957+0000 7f233b5e7740 1 -- start start 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 1 Processor -- start 2024-09-15T22:49:02.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a10855280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23341ab350 con 0x561a107d3cc0 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23341ac7c0 con 0x561a107e2a90 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f23341aba20 con 0x561a107d33a0 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233b5e7740 10 client.0 _ll_register_callbacks cb 0x7f231c0bc7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51276/0 (socket says 172.21.15.16:51276) 2024-09-15T22:49:02.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 -- 172.21.15.16:0/1343242482 learned_addr learned my addr 172.21.15.16:0/1343242482 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233359e640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a10855280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f2332d9d640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 -- 172.21.15.16:0/1343242482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 msgr2=0x561a107e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 -- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 msgr2=0x561a10855280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a10855280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f233259c640 1 -- 172.21.15.16:0/1343242482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a107e2780 con 0x561a107d3cc0 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.958+0000 7f2332d9d640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233259c640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e79d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5040 tx=0x7f231c034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded382e32521b551 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f231a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f231a7fc640 1 -- 172.21.15.16:0/1343242482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c049070 con 0x561a107d3cc0 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f231a7fc640 1 -- 172.21.15.16:0/1343242482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f231c0c52e0 con 0x561a107d3cc0 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f231c0bd000 con 0x561a107d3cc0 2024-09-15T22:49:02.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f231a7fc640 1 -- 172.21.15.16:0/1343242482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f231c0bc260 con 0x561a107d3cc0 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 10 client.17760 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a107e2780 con 0x561a107d3cc0 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f231a7fc640 1 -- 172.21.15.16:0/1343242482 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f231c0bd000 con 0x561a107d3cc0 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f231a7fc640 1 -- 172.21.15.16:0/1343242482 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f231c0bbd50 con 0x561a107d3cc0 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 10 client.17760 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 20 client.17760 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a1085df00 0x561a108602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 4 client.17760 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.959+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a10740d60 con 0x561a1085df00 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f2332d9d640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a1085df00 0x561a108602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f2332d9d640 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a1085df00 0x561a108602f0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1ef06061f339d7 server_cookie=37619499f1407e9f in_seq=0 out_seq=0 2024-09-15T22:49:02.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f231a7fc640 10 client.17760 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f231a7fc640 1 -- 172.21.15.16:0/1343242482 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561a10740d60 con 0x561a1085df00 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f231a7fc640 10 client.17760 handle_command_reply: tid=0 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 client.17760 shutdown 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 msgr2=0x561a107e79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e79d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f231c0c5040 tx=0x7f231c034000 comp rx=0 tx=0).stop 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 shutdown_connections 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x561a1085df00 0x561a108602f0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561a107e2a90 0x561a10855280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561a107d3cc0 0x561a107e79d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 --2- 172.21.15.16:0/1343242482 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x561a107d33a0 0x561a107e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 >> 172.21.15.16:0/1343242482 conn(0x561a107cfa30 msgr2=0x561a107d56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f231a7fc640 0 client.17760 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 shutdown_connections 2024-09-15T22:49:02.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.960+0000 7f233b5e7740 1 -- 172.21.15.16:0/1343242482 wait complete. 2024-09-15T22:49:02.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233b5e7740 20 client.17760 trim_cache size 0 max 16384 2024-09-15T22:49:02.970 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:49:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-15T22:49:02.970 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {} 2024-09-15T22:49:02.970 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23200a3430 msgr2=0x7f23200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23200a3430 0x7f23200a58f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f232401f000 tx=0x7f2324020000 comp rx=0 tx=0).stop 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 msgr2=0x7f2334198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f2334198910 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f231c005ab0 tx=0x7f231c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 shutdown_connections 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2334106f10 0x7f233419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2334102790 0x7f2334198910 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f23200a3430 0x7f23200a58f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 --2- 172.21.15.16:0/3137588932 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2334101de0 0x7f23341983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 >> 172.21.15.16:0/3137588932 conn(0x7f23340fd8c0 msgr2=0x7f23340ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 shutdown_connections 2024-09-15T22:49:02.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:02.961+0000 7f233920e640 1 -- 172.21.15.16:0/3137588932 wait complete. 2024-09-15T22:49:03.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:02 smithi059 ceph-mon[34872]: pgmap v329: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:49:03.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:02 smithi130 ceph-mon[35742]: pgmap v329: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:49:03.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:02 smithi016 ceph-mon[31492]: pgmap v329: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:49:03.746 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:49:03.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:49:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 -- 172.21.15.16:0/1481975842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98102790 msgr2=0x7f2f981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 --2- 172.21.15.16:0/1481975842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98102790 0x7f2f981067e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009900 tx=0x7f2f8c02a410 comp rx=0 tx=0).stop 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 -- 172.21.15.16:0/1481975842 shutdown_connections 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 --2- 172.21.15.16:0/1481975842 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f98106f10 0x7f2f9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 --2- 172.21.15.16:0/1481975842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98102790 0x7f2f981067e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 --2- 172.21.15.16:0/1481975842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f98101de0 0x7f2f981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 -- 172.21.15.16:0/1481975842 >> 172.21.15.16:0/1481975842 conn(0x7f2f980fd8c0 msgr2=0x7f2f980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 -- 172.21.15.16:0/1481975842 shutdown_connections 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 -- 172.21.15.16:0/1481975842 wait complete. 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.426+0000 7f2f9d598640 1 Processor -- start 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f9d598640 1 -- start start 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f9d598640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f98101de0 0x7f2f980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f9d598640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f98102790 0x7f2f98074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f9d598640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98106f10 0x7f2f98074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f9d598640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98111fb0 con 0x7f2f98106f10 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f9d598640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98111e30 con 0x7f2f98102790 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f9d598640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f2f98112130 con 0x7f2f98101de0 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98106f10 0x7f2f98074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98106f10 0x7f2f98074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51298/0 (socket says 172.21.15.16:51298) 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f977fe640 1 -- 172.21.15.16:0/2557968 learned_addr learned my addr 172.21.15.16:0/2557968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f967fc640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f98102790 0x7f2f98074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.427+0000 7f2f96ffd640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f98101de0 0x7f2f980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f977fe640 1 -- 172.21.15.16:0/2557968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f98101de0 msgr2=0x7f2f980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f977fe640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f98101de0 0x7f2f980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f977fe640 1 -- 172.21.15.16:0/2557968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f98102790 msgr2=0x7f2f98074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f977fe640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f98102790 0x7f2f98074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f977fe640 1 -- 172.21.15.16:0/2557968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f981a6990 con 0x7f2f98106f10 2024-09-15T22:49:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f977fe640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98106f10 0x7f2f98074f70 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000eaa0 tx=0x7f2f9000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f7ffff640 1 -- 172.21.15.16:0/2557968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90004d20 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f7ffff640 1 -- 172.21.15.16:0/2557968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f90005020 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f981a6c80 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f7ffff640 1 -- 172.21.15.16:0/2557968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900116d0 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.428+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f981a7190 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.429+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f981021c0 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.430+0000 7f2f7ffff640 1 -- 172.21.15.16:0/2557968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2f900051d0 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.431+0000 7f2f7ffff640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2f800a33b0 0x7f2f800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.431+0000 7f2f7ffff640 1 -- 172.21.15.16:0/2557968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f2f900d0250 con 0x7f2f98106f10 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.431+0000 7f2f96ffd640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2f800a33b0 0x7f2f800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.431+0000 7f2f96ffd640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2f800a33b0 0x7f2f800a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009990 tx=0x7f2f88009110 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.435+0000 7f2f7ffff640 1 -- 172.21.15.16:0/2557968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f90016030 con 0x7f2f98106f10 2024-09-15T22:49:04.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.648+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f9802d7c0 con 0x7f2f98106f10 2024-09-15T22:49:04.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.649+0000 7f2f7ffff640 1 -- 172.21.15.16:0/2557968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f2f9009c880 con 0x7f2f98106f10 2024-09-15T22:49:04.651 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:49:04.651 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:49:04.651 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:49:04.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2f800a33b0 msgr2=0x7f2f800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:04.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2f800a33b0 0x7f2f800a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009990 tx=0x7f2f88009110 comp rx=0 tx=0).stop 2024-09-15T22:49:04.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98106f10 msgr2=0x7f2f98074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:04.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98106f10 0x7f2f98074f70 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000eaa0 tx=0x7f2f9000eea0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 shutdown_connections 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f98106f10 0x7f2f98074f70 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f98102790 0x7f2f98074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f2f800a33b0 0x7f2f800a5870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 --2- 172.21.15.16:0/2557968 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2f98101de0 0x7f2f980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 >> 172.21.15.16:0/2557968 conn(0x7f2f980fd8c0 msgr2=0x7f2f980fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 shutdown_connections 2024-09-15T22:49:04.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:04.651+0000 7f2f9d598640 1 -- 172.21.15.16:0/2557968 wait complete. 2024-09-15T22:49:05.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:04 smithi059 ceph-mon[34872]: pgmap v330: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:49:05.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:04 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2557968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:05.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:04 smithi130 ceph-mon[35742]: pgmap v330: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:49:05.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:04 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2557968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:05.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:04 smithi016 ceph-mon[31492]: pgmap v330: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:49:05.307 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:04 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2557968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:05.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:49:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.203+0000 7f747537f640 1 -- 172.21.15.16:0/3505207133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102570 msgr2=0x7f74701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.203+0000 7f747537f640 1 --2- 172.21.15.16:0/3505207133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102570 0x7f74701029b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f7458009900 tx=0x7f745802a480 comp rx=0 tx=0).stop 2024-09-15T22:49:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 -- 172.21.15.16:0/3505207133 shutdown_connections 2024-09-15T22:49:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 --2- 172.21.15.16:0/3505207133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7470102ef0 0x7f747010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 --2- 172.21.15.16:0/3505207133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102570 0x7f74701029b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 --2- 172.21.15.16:0/3505207133 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7470100e20 0x7f7470101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 -- 172.21.15.16:0/3505207133 >> 172.21.15.16:0/3505207133 conn(0x7f74700fc7d0 msgr2=0x7f74700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 -- 172.21.15.16:0/3505207133 shutdown_connections 2024-09-15T22:49:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 -- 172.21.15.16:0/3505207133 wait complete. 2024-09-15T22:49:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 Processor -- start 2024-09-15T22:49:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.204+0000 7f747537f640 1 -- start start 2024-09-15T22:49:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f747537f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7470100e20 0x7f7470198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f747537f640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7470102570 0x7f7470198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f747537f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102ef0 0x7f747019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f747537f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74701120f0 con 0x7f7470102ef0 2024-09-15T22:49:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f747537f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7470111f70 con 0x7f7470100e20 2024-09-15T22:49:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f747537f640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7470112270 con 0x7f7470102570 2024-09-15T22:49:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102ef0 0x7f747019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102ef0 0x7f747019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51320/0 (socket says 172.21.15.16:51320) 2024-09-15T22:49:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 -- 172.21.15.16:0/1625030633 learned_addr learned my addr 172.21.15.16:0/1625030633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746e7fc640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7470102570 0x7f7470198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746effd640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7470100e20 0x7f7470198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 -- 172.21.15.16:0/1625030633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7470102570 msgr2=0x7f7470198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7470102570 0x7f7470198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 -- 172.21.15.16:0/1625030633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7470100e20 msgr2=0x7f7470198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7470100e20 0x7f7470198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 -- 172.21.15.16:0/1625030633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747019d540 con 0x7f7470102ef0 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746effd640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7470100e20 0x7f7470198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f746f7fe640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102ef0 0x7f747019ce60 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f746000c970 tx=0x7f746000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f7467fff640 1 -- 172.21.15.16:0/1625030633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460007bb0 con 0x7f7470102ef0 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747019d830 con 0x7f7470102ef0 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f7467fff640 1 -- 172.21.15.16:0/1625030633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7460007d50 con 0x7f7470102ef0 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.205+0000 7f7467fff640 1 -- 172.21.15.16:0/1625030633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746000f450 con 0x7f7470102ef0 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.206+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74701a5110 con 0x7f7470102ef0 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.208+0000 7f7467fff640 1 -- 172.21.15.16:0/1625030633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7460004f60 con 0x7f7470102ef0 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.208+0000 7f7477758740 1 Processor -- start 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.208+0000 7f7477758740 1 -- start start 2024-09-15T22:49:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7477758740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f105090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7477758740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f105a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7477758740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f116b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7477758740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e1f06b2f0 con 0x559e1f1055d0 2024-09-15T22:49:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7477758740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e1f012cf0 con 0x559e1f114780 2024-09-15T22:49:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7477758740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559e1f0672b0 con 0x559e1f106b40 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f105a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7467fff640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f745c0a33b0 0x7f745c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f105a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51330/0 (socket says 172.21.15.16:51330) 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 -- 172.21.15.16:0/3749246576 learned_addr learned my addr 172.21.15.16:0/3749246576 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746e7fc640 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f105090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746e7fc640 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f116b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746f7fe640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f745c0a33b0 0x7f745c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f7467fff640 1 -- 172.21.15.16:0/1625030633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f74600cfdf0 con 0x7f7470102ef0 2024-09-15T22:49:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 -- 172.21.15.16:0/3749246576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 msgr2=0x559e1f105090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f105090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 -- 172.21.15.16:0/3749246576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 msgr2=0x559e1f116b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f116b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 -- 172.21.15.16:0/3749246576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1eff73a0 con 0x559e1f1055d0 2024-09-15T22:49:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746e7fc640 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.209+0000 7f746effd640 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f105a10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7468007fb0 tx=0x7f746800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee70512ae6043e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.210+0000 7f746f7fe640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f745c0a33b0 0x7f745c0a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74600cd790 tx=0x7f746006b940 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.210+0000 7f74657fa640 1 -- 172.21.15.16:0/3749246576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468014070 con 0x559e1f1055d0 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.210+0000 7f74657fa640 1 -- 172.21.15.16:0/3749246576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f746800fd30 con 0x559e1f1055d0 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.210+0000 7f74657fa640 1 -- 172.21.15.16:0/3749246576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746800ed00 con 0x559e1f1055d0 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.210+0000 7f7477758740 1 -- 172.21.15.16:0/3749246576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 msgr2=0x559e1f105a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.210+0000 7f7477758740 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f105a10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f7468007fb0 tx=0x7f746800c750 comp rx=0 tx=0).stop 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- 172.21.15.16:0/3749246576 shutdown_connections 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f105a10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 --2- 172.21.15.16:0/3749246576 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- 172.21.15.16:0/3749246576 >> 172.21.15.16:0/3749246576 conn(0x559e1f0ce310 msgr2=0x559e1f101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- 172.21.15.16:0/3749246576 shutdown_connections 2024-09-15T22:49:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- 172.21.15.16:0/3749246576 wait complete. 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- start start 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 Processor -- start 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f114130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f111980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559e1f117480 con 0x559e1f1055d0 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559e1f026060 con 0x559e1f114780 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.211+0000 7f7477758740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x559e1f117600 con 0x559e1f106b40 2024-09-15T22:49:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f7477758740 10 client.0 _ll_register_callbacks cb 0x559e1ed798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f114130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f114130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51332/0 (socket says 172.21.15.16:51332) 2024-09-15T22:49:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 -- 172.21.15.16:0/3678693819 learned_addr learned my addr 172.21.15.16:0/3678693819 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746effd640 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f111980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 -- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 msgr2=0x559e1f1113e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1113e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 -- 172.21.15.16:0/3678693819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 msgr2=0x559e1f111980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f111980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 -- 172.21.15.16:0/3678693819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1eddc3c0 con 0x559e1f1055d0 2024-09-15T22:49:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f746e7fc640 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f114130 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7458009230 tx=0x7f7458009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cae0fbddc7bdea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f74567fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f74567fc640 1 -- 172.21.15.16:0/3678693819 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458035040 con 0x559e1f1055d0 2024-09-15T22:49:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f74567fc640 1 -- 172.21.15.16:0/3678693819 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f7458003dc0 con 0x559e1f1055d0 2024-09-15T22:49:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.212+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f746800fd30 con 0x559e1f1055d0 2024-09-15T22:49:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f74567fc640 1 -- 172.21.15.16:0/3678693819 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7458005020 con 0x559e1f1055d0 2024-09-15T22:49:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 10 client.17778 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e1f122ed0 con 0x559e1f1055d0 2024-09-15T22:49:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f74567fc640 1 -- 172.21.15.16:0/3678693819 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f746800fd30 con 0x559e1f1055d0 2024-09-15T22:49:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f74567fc640 1 -- 172.21.15.16:0/3678693819 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7458031540 con 0x559e1f1055d0 2024-09-15T22:49:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 10 client.17778 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 10 client.17778 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:49:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 20 client.17778 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f124390 0x559e1f126780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 4 client.17778 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e1f078860 con 0x559e1f124390 2024-09-15T22:49:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.213+0000 7f746effd640 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f124390 0x559e1f126780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.214+0000 7f746effd640 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f124390 0x559e1f126780 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=163b11762702c690 server_cookie=5175cf1f550c57c3 in_seq=0 out_seq=0 2024-09-15T22:49:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.214+0000 7f74567fc640 10 client.17778 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:49:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.214+0000 7f74567fc640 1 -- 172.21.15.16:0/3678693819 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e1f078860 con 0x559e1f124390 2024-09-15T22:49:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.214+0000 7f74567fc640 10 client.17778 handle_command_reply: tid=0 2024-09-15T22:49:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.214+0000 7f7477758740 1 client.17778 shutdown 2024-09-15T22:49:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 msgr2=0x559e1f114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f114130 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7458009230 tx=0x7f7458009260 comp rx=0 tx=0).stop 2024-09-15T22:49:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 shutdown_connections 2024-09-15T22:49:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f124390 0x559e1f126780 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 --2- 172.21.15.16:0/3678693819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f114130 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 >> 172.21.15.16:0/3678693819 conn(0x559e1f0ce310 msgr2=0x559e1f107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f74567fc640 0 client.17778 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:49:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 shutdown_connections 2024-09-15T22:49:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 1 -- 172.21.15.16:0/3678693819 wait complete. 2024-09-15T22:49:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.215+0000 7f7477758740 20 client.17778 trim_cache size 0 max 16384 2024-09-15T22:49:06.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 Processor -- start 2024-09-15T22:49:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 -- start start 2024-09-15T22:49:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f1055d0 0x559e1f120720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f114780 0x559e1f115b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74701af820 con 0x559e1f114780 2024-09-15T22:49:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74701ad0f0 con 0x559e1f1055d0 2024-09-15T22:49:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f7477758740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f74701aaf60 con 0x559e1f106b40 2024-09-15T22:49:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f746e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f114780 0x559e1f115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f746e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f114780 0x559e1f115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51346/0 (socket says 172.21.15.16:51346) 2024-09-15T22:49:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.228+0000 7f746e7fc640 1 -- 172.21.15.16:0/2877522781 learned_addr learned my addr 172.21.15.16:0/2877522781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746e7fc640 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f1055d0 0x559e1f120720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746effd640 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746e7fc640 1 -- 172.21.15.16:0/2877522781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 msgr2=0x559e1f1263a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746e7fc640 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1263a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746e7fc640 1 -- 172.21.15.16:0/2877522781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f1055d0 msgr2=0x559e1f120720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746e7fc640 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f1055d0 0x559e1f120720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746e7fc640 1 -- 172.21.15.16:0/2877522781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1f10f930 con 0x559e1f114780 2024-09-15T22:49:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f746e7fc640 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f114780 0x559e1f115b70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7458009850 tx=0x7f745802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fda2fe21b9383c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f7464ff9640 1 -- 172.21.15.16:0/2877522781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458035030 con 0x559e1f114780 2024-09-15T22:49:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f7464ff9640 1 -- 172.21.15.16:0/2877522781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7458003750 con 0x559e1f114780 2024-09-15T22:49:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.229+0000 7f7464ff9640 1 -- 172.21.15.16:0/2877522781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580047a0 con 0x559e1f114780 2024-09-15T22:49:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 -- 172.21.15.16:0/2877522781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f114780 msgr2=0x559e1f115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f114780 0x559e1f115b70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7458009850 tx=0x7f745802a480 comp rx=0 tx=0).stop 2024-09-15T22:49:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 -- 172.21.15.16:0/2877522781 shutdown_connections 2024-09-15T22:49:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f114780 0x559e1f115b70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f1263a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 --2- 172.21.15.16:0/2877522781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f1055d0 0x559e1f120720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 -- 172.21.15.16:0/2877522781 >> 172.21.15.16:0/2877522781 conn(0x559e1f0ce310 msgr2=0x559e1f10fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 -- 172.21.15.16:0/2877522781 shutdown_connections 2024-09-15T22:49:06.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 -- 172.21.15.16:0/2877522781 wait complete. 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 -- start start 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.230+0000 7f7477758740 1 Processor -- start 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f12bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f12c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f12c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74701ab5b0 con 0x559e1f1055d0 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f74701aca20 con 0x559e1f114780 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f74701abc80 con 0x559e1f106b40 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f12bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 10 client.0 _ll_register_callbacks cb 0x7f7458009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f12bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51352/0 (socket says 172.21.15.16:51352) 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 -- 172.21.15.16:0/1036329503 learned_addr learned my addr 172.21.15.16:0/1036329503 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746effd640 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f12c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f12c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 -- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 msgr2=0x559e1f12c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f12c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 -- 172.21.15.16:0/1036329503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 msgr2=0x559e1f12c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f12c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 -- 172.21.15.16:0/1036329503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e1f11b1c0 con 0x559e1f1055d0 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f746e7fc640 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f12bd60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f745802aa40 tx=0x7f745802ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8d4a8de20107b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f74567fc640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f74567fc640 1 -- 172.21.15.16:0/1036329503 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74580051c0 con 0x559e1f1055d0 2024-09-15T22:49:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f74567fc640 1 -- 172.21.15.16:0/1036329503 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f7458005360 con 0x559e1f1055d0 2024-09-15T22:49:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.231+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7458003750 con 0x559e1f1055d0 2024-09-15T22:49:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f74567fc640 1 -- 172.21.15.16:0/1036329503 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74580047a0 con 0x559e1f1055d0 2024-09-15T22:49:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 10 client.17784 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e1f18f440 con 0x559e1f1055d0 2024-09-15T22:49:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f74567fc640 1 -- 172.21.15.16:0/1036329503 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7458003750 con 0x559e1f1055d0 2024-09-15T22:49:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f74567fc640 1 -- 172.21.15.16:0/1036329503 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7458062020 con 0x559e1f1055d0 2024-09-15T22:49:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 10 client.17784 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 10 client.17784 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 20 client.17784 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f18f970 0x559e1f191d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 4 client.17784 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 --> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559e1f072d70 con 0x559e1f18f970 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.232+0000 7f746effd640 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f18f970 0x559e1f191d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f746effd640 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f18f970 0x559e1f191d60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=444f1f013bbdc5f4 server_cookie=d286d526da0ffab6 in_seq=0 out_seq=0 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f74567fc640 10 client.17784 ms_handle_connect on v2:172.21.15.130:6838/1236682624 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f74567fc640 1 -- 172.21.15.16:0/1036329503 <== mds.2 v2:172.21.15.130:6838/1236682624 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559e1f072d70 con 0x559e1f18f970 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f74567fc640 10 client.17784 handle_command_reply: tid=0 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 client.17784 shutdown 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 msgr2=0x559e1f12bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f12bd60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f745802aa40 tx=0x7f745802ac00 comp rx=0 tx=0).stop 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 shutdown_connections 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:6838/1236682624,v1:172.21.15.130:6839/1236682624] conn(0x559e1f18f970 0x559e1f191d60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559e1f114780 0x559e1f12c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x559e1f106b40 0x559e1f12c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 --2- 172.21.15.16:0/1036329503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559e1f1055d0 0x559e1f12bd60 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 >> 172.21.15.16:0/1036329503 conn(0x559e1f0ce310 msgr2=0x559e1f1004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f74567fc640 0 client.17784 ms_handle_reset on v2:172.21.15.130:6838/1236682624 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 shutdown_connections 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 1 -- 172.21.15.16:0/1036329503 wait complete. 2024-09-15T22:49:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.233+0000 7f7477758740 20 client.17784 trim_cache size 0 max 16384 2024-09-15T22:49:06.262 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f745c0a33b0 msgr2=0x7f745c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f745c0a33b0 0x7f745c0a5870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f74600cd790 tx=0x7f746006b940 comp rx=0 tx=0).stop 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102ef0 msgr2=0x7f747019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102ef0 0x7f747019ce60 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f746000c970 tx=0x7f746000ce30 comp rx=0 tx=0).stop 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 shutdown_connections 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7470102ef0 0x7f747019ce60 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f745c0a33b0 0x7f745c0a5870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7470102570 0x7f7470198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 --2- 172.21.15.16:0/1625030633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7470100e20 0x7f7470198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 >> 172.21.15.16:0/1625030633 conn(0x7f74700fc7d0 msgr2=0x7f74700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 shutdown_connections 2024-09-15T22:49:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:06.234+0000 7f747537f640 1 -- 172.21.15.16:0/1625030633 wait complete. 2024-09-15T22:49:06.930 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:49:07.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:06 smithi059 ceph-mon[34872]: pgmap v331: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:49:07.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:06 smithi130 ceph-mon[35742]: pgmap v331: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:49:07.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:07.306 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:06 smithi016 ceph-mon[31492]: pgmap v331: 129 pgs: 129 active+clean; 17 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-09-15T22:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 -- 172.21.15.16:0/2382548436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 msgr2=0x7f26c0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 --2- 172.21.15.16:0/2382548436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c0105eb0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009900 tx=0x7f26bc02a410 comp rx=0 tx=0).stop 2024-09-15T22:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 -- 172.21.15.16:0/2382548436 shutdown_connections 2024-09-15T22:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 --2- 172.21.15.16:0/2382548436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26c0101ba0 0x7f26c010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 --2- 172.21.15.16:0/2382548436 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c0101200 0x7f26c0101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 --2- 172.21.15.16:0/2382548436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c0105eb0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 -- 172.21.15.16:0/2382548436 >> 172.21.15.16:0/2382548436 conn(0x7f26c0077210 msgr2=0x7f26c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 -- 172.21.15.16:0/2382548436 shutdown_connections 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.787+0000 7f26c72a0640 1 -- 172.21.15.16:0/2382548436 wait complete. 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 Processor -- start 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 -- start start 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c0101200 0x7f26c01992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26c0101ba0 0x7f26c0199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c01a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26c0110f10 con 0x7f26c0105ad0 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26c0110d90 con 0x7f26c0101ba0 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c72a0640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26c0111090 con 0x7f26c0101200 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c583d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c01a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c583d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c01a4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51364/0 (socket says 172.21.15.16:51364) 2024-09-15T22:49:07.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c583d640 1 -- 172.21.15.16:0/156764770 learned_addr learned my addr 172.21.15.16:0/156764770 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c503c640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c0101200 0x7f26c01992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.788+0000 7f26c483b640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26c0101ba0 0x7f26c0199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c583d640 1 -- 172.21.15.16:0/156764770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c0101200 msgr2=0x7f26c01992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c583d640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c0101200 0x7f26c01992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c583d640 1 -- 172.21.15.16:0/156764770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26c0101ba0 msgr2=0x7f26c0199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c583d640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26c0101ba0 0x7f26c0199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c583d640 1 -- 172.21.15.16:0/156764770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c01a4880 con 0x7f26c0105ad0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c583d640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c01a4340 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f26b800b9e0 tx=0x7f26b800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26b67fc640 1 -- 172.21.15.16:0/156764770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b8019070 con 0x7f26c0105ad0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26b67fc640 1 -- 172.21.15.16:0/156764770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b8014440 con 0x7f26c0105ad0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c01a4a50 con 0x7f26c0105ad0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26b67fc640 1 -- 172.21.15.16:0/156764770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80125c0 con 0x7f26c0105ad0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.789+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c01a4f60 con 0x7f26c0105ad0 2024-09-15T22:49:07.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.791+0000 7f26b67fc640 1 -- 172.21.15.16:0/156764770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f26b801c070 con 0x7f26c0105ad0 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 Processor -- start 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 -- start start 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ea6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8ea6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8eb5780 0x5640b8eb7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640b8e0c2f0 con 0x5640b8eb5780 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5640b8db3cf0 con 0x5640b8ea7b40 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26c9679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5640b8e082b0 con 0x5640b8ea65d0 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.792+0000 7f26b67fc640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f26ac0a3430 0x7f26ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8eb5780 0x5640b8eb7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8eb5780 0x5640b8eb7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51378/0 (socket says 172.21.15.16:51378) 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 -- 172.21.15.16:0/2695440 learned_addr learned my addr 172.21.15.16:0/2695440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:07.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26b67fc640 1 -- 172.21.15.16:0/156764770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f26b80cf530 con 0x7f26c0105ad0 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c583d640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f26ac0a3430 0x7f26ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c503c640 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8ea6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ea6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 -- 172.21.15.16:0/2695440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 msgr2=0x5640b8ea6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8ea6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 -- 172.21.15.16:0/2695440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 msgr2=0x5640b8ea6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ea6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 -- 172.21.15.16:0/2695440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b8d983a0 con 0x5640b8eb5780 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c583d640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f26ac0a3430 0x7f26ac0a58f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26b800b9b0 tx=0x7f26b806d340 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26c483b640 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8eb5780 0x5640b8eb7b70 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f26b00125a0 tx=0x7f26b000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0e368fb60b4c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26ab7fe640 1 -- 172.21.15.16:0/2695440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0004250 con 0x5640b8eb5780 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26ab7fe640 1 -- 172.21.15.16:0/2695440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b00043f0 con 0x5640b8eb5780 2024-09-15T22:49:07.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.793+0000 7f26ab7fe640 1 -- 172.21.15.16:0/2695440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0004e10 con 0x5640b8eb5780 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 -- 172.21.15.16:0/2695440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8eb5780 msgr2=0x5640b8eb7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8eb5780 0x5640b8eb7b70 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f26b00125a0 tx=0x7f26b000b870 comp rx=0 tx=0).stop 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 -- 172.21.15.16:0/2695440 shutdown_connections 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8eb5780 0x5640b8eb7b70 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8ea6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 --2- 172.21.15.16:0/2695440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ea6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 -- 172.21.15.16:0/2695440 >> 172.21.15.16:0/2695440 conn(0x5640b8e6f310 msgr2=0x5640b8ea2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 -- 172.21.15.16:0/2695440 shutdown_connections 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.794+0000 7f26c9679740 1 -- 172.21.15.16:0/2695440 wait complete. 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 -- start start 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 Processor -- start 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8eb5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea7b40 0x5640b8eb2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5640b8eb8480 con 0x5640b8ea7b40 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5640b8dc7060 con 0x5640b8eb5780 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x5640b8eb8600 con 0x5640b8ea65d0 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c9679740 10 client.0 _ll_register_callbacks cb 0x5640b8b1a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c503c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea7b40 0x5640b8eb2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c503c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea7b40 0x5640b8eb2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51394/0 (socket says 172.21.15.16:51394) 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c503c640 1 -- 172.21.15.16:0/1425218192 learned_addr learned my addr 172.21.15.16:0/1425218192 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.795+0000 7f26c483b640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8eb5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c483b640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c503c640 1 -- 172.21.15.16:0/1425218192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 msgr2=0x5640b8eb5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c503c640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8eb5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c503c640 1 -- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 msgr2=0x5640b8eb29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c503c640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c503c640 1 -- 172.21.15.16:0/1425218192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b8b7d3c0 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c483b640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8eb5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c503c640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea7b40 0x5640b8eb2400 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f26bc02a930 tx=0x7f26bc0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d497f30336f6f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1425218192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc035040 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1425218192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f26bc003b40 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b00043f0 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1425218192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26bc04e050 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c9679740 10 client.17796 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640b8ec3f40 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.796+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1425218192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b00043f0 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1425218192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26bc03c4d0 con 0x5640b8ea7b40 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c9679740 10 client.17796 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:07.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c9679740 10 client.17796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c9679740 20 client.17796 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c9679740 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8ec5450 0x5640b8ec7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c9679740 4 client.17796 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640b8e19860 con 0x5640b8ec5450 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c483b640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8ec5450 0x5640b8ec7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26a8ff9640 10 client.17796 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.797+0000 7f26c483b640 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8ec5450 0x5640b8ec7890 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c345115ec9e10f1 server_cookie=5465aa20eb736db5 in_seq=0 out_seq=0 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.798+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1425218192 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640b8e19860 con 0x5640b8ec5450 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.798+0000 7f26a8ff9640 10 client.17796 handle_command_reply: tid=0 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.798+0000 7f26c9679740 1 client.17796 shutdown 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea7b40 msgr2=0x5640b8eb2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea7b40 0x5640b8eb2400 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f26bc02a930 tx=0x7f26bc0044f0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 shutdown_connections 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8ec5450 0x5640b8ec7890 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea7b40 0x5640b8eb2400 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 --2- 172.21.15.16:0/1425218192 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea65d0 0x5640b8eb5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26a8ff9640 0 client.17796 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 >> 172.21.15.16:0/1425218192 conn(0x5640b8e6f310 msgr2=0x5640b8ea8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 shutdown_connections 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 1 -- 172.21.15.16:0/1425218192 wait complete. 2024-09-15T22:49:07.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.799+0000 7f26c9679740 20 client.17796 trim_cache size 0 max 16384 2024-09-15T22:49:07.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 Processor -- start 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 -- start start 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ea2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea7b40 0x5640b8ec1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01af660 con 0x5640b8ea65d0 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01acf30 con 0x5640b8eb5780 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c9679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01aada0 con 0x5640b8ea7b40 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c483b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ea2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c483b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ea2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.812+0000 7f26c483b640 1 -- 172.21.15.16:0/3293669126 learned_addr learned my addr 172.21.15.16:0/3293669126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c483b640 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c503c640 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea7b40 0x5640b8ec1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c483b640 1 -- 172.21.15.16:0/3293669126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea7b40 msgr2=0x5640b8ec1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c483b640 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea7b40 0x5640b8ec1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c483b640 1 -- 172.21.15.16:0/3293669126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 msgr2=0x5640b8eb13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c483b640 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c483b640 1 -- 172.21.15.16:0/3293669126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640b8ea9120 con 0x5640b8ea65d0 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26c483b640 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ea2d70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f26b0004a20 tx=0x7f26b0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a076f87b444ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26aaffd640 1 -- 172.21.15.16:0/3293669126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0028040 con 0x5640b8ea65d0 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26aaffd640 1 -- 172.21.15.16:0/3293669126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b0033030 con 0x5640b8ea65d0 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.813+0000 7f26aaffd640 1 -- 172.21.15.16:0/3293669126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b002e360 con 0x5640b8ea65d0 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 -- 172.21.15.16:0/3293669126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 msgr2=0x5640b8ea2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ea2d70 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f26b0004a20 tx=0x7f26b0005c50 comp rx=0 tx=0).stop 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 -- 172.21.15.16:0/3293669126 shutdown_connections 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8eb5780 0x5640b8eb13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8ea7b40 0x5640b8ec1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 --2- 172.21.15.16:0/3293669126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ea2d70 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 -- 172.21.15.16:0/3293669126 >> 172.21.15.16:0/3293669126 conn(0x5640b8e6f310 msgr2=0x5640b8eb0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 -- 172.21.15.16:0/3293669126 shutdown_connections 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 -- 172.21.15.16:0/3293669126 wait complete. 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 -- start start 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.814+0000 7f26c9679740 1 Processor -- start 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ebdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ebe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8eb5780 0x5640b8eaee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01ab3f0 con 0x5640b8ea65d0 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01ac860 con 0x5640b8ea7b40 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01abac0 con 0x5640b8eb5780 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 10 client.0 _ll_register_callbacks cb 0x7f26bc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ebdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ebdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51408/0 (socket says 172.21.15.16:51408) 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 -- 172.21.15.16:0/1892151130 learned_addr learned my addr 172.21.15.16:0/1892151130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c503c640 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ebe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8eb5780 0x5640b8eaee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 -- 172.21.15.16:0/1892151130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8eb5780 msgr2=0x5640b8eaee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8eb5780 0x5640b8eaee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 -- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 msgr2=0x5640b8ebe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ebe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 -- 172.21.15.16:0/1892151130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640b8eaf670 con 0x5640b8ea65d0 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c483b640 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ebdb70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f26b0003fd0 tx=0x7f26b0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1636d202798b4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1892151130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b003e020 con 0x5640b8ea65d0 2024-09-15T22:49:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1892151130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9467+0+0 (secure 0 0 0) 0x7f26b0004040 con 0x5640b8ea65d0 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.815+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b0033030 con 0x5640b8ea65d0 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1892151130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26b0046050 con 0x5640b8ea65d0 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 10 client.17802 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640b8f2fd50 con 0x5640b8ea65d0 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1892151130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b0033030 con 0x5640b8ea65d0 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1892151130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26b002e040 con 0x5640b8ea65d0 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 10 client.17802 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 10 client.17802 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 20 client.17802 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8f30170 0x5640b8f32560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 4 client.17802 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 --> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640b8e13d70 con 0x5640b8f30170 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.816+0000 7f26c503c640 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8f30170 0x5640b8f32560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:07.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c503c640 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8f30170 0x5640b8f32560 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edb0bd86261620ae server_cookie=d74716901b1ba2ef in_seq=0 out_seq=0 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26a8ff9640 10 client.17802 ms_handle_connect on v2:172.21.15.59:6836/2084675816 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26a8ff9640 1 -- 172.21.15.16:0/1892151130 <== mds.1 v2:172.21.15.59:6836/2084675816 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640b8e13d70 con 0x5640b8f30170 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26a8ff9640 10 client.17802 handle_command_reply: tid=0 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 client.17802 shutdown 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 msgr2=0x5640b8ebdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ebdb70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f26b0003fd0 tx=0x7f26b0003c80 comp rx=0 tx=0).stop 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 shutdown_connections 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:6836/2084675816,v1:172.21.15.59:6837/2084675816] conn(0x5640b8f30170 0x5640b8f32560 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x5640b8eb5780 0x5640b8eaee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5640b8ea7b40 0x5640b8ebe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26a8ff9640 0 client.17802 ms_handle_reset on v2:172.21.15.59:6836/2084675816 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 --2- 172.21.15.16:0/1892151130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5640b8ea65d0 0x5640b8ebdb70 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 >> 172.21.15.16:0/1892151130 conn(0x5640b8e6f310 msgr2=0x5640b8eb14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 shutdown_connections 2024-09-15T22:49:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 1 -- 172.21.15.16:0/1892151130 wait complete. 2024-09-15T22:49:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.817+0000 7f26c9679740 20 client.17802 trim_cache size 0 max 16384 2024-09-15T22:49:07.825 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:49:07.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f26ac0a3430 msgr2=0x7f26ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f26ac0a3430 0x7f26ac0a58f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f26b800b9b0 tx=0x7f26b806d340 comp rx=0 tx=0).stop 2024-09-15T22:49:07.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 msgr2=0x7f26c01a4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:07.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c01a4340 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f26b800b9e0 tx=0x7f26b800bea0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 shutdown_connections 2024-09-15T22:49:07.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26c0105ad0 0x7f26c01a4340 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26c0101ba0 0x7f26c0199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f26ac0a3430 0x7f26ac0a58f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 --2- 172.21.15.16:0/156764770 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f26c0101200 0x7f26c01992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 >> 172.21.15.16:0/156764770 conn(0x7f26c0077210 msgr2=0x7f26c0103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 shutdown_connections 2024-09-15T22:49:07.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:07.818+0000 7f26c72a0640 1 -- 172.21.15.16:0/156764770 wait complete. 2024-09-15T22:49:08.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:49:08.862 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:09.128 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:08 smithi016 ceph-mon[31492]: pgmap v332: 129 pgs: 129 active+clean; 17 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:49:09.349 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:08 smithi130 ceph-mon[35742]: pgmap v332: 129 pgs: 129 active+clean; 17 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:49:09.349 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:08 smithi059 ceph-mon[34872]: pgmap v332: 129 pgs: 129 active+clean; 17 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-09-15T22:49:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.363+0000 7f909a8dd640 1 -- 172.21.15.16:0/1042506173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9094074620 msgr2=0x7f9094074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.363+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1042506173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9094074620 0x7f9094074a00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f9084009900 tx=0x7f908402a480 comp rx=0 tx=0).stop 2024-09-15T22:49:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.363+0000 7f909a8dd640 1 -- 172.21.15.16:0/1042506173 shutdown_connections 2024-09-15T22:49:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.363+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1042506173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f909410e3c0 0x7f9094110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.363+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1042506173 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9094074f40 0x7f909410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.363+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1042506173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9094074620 0x7f9094074a00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.363+0000 7f909a8dd640 1 -- 172.21.15.16:0/1042506173 >> 172.21.15.16:0/1042506173 conn(0x7f90940fd8c0 msgr2=0x7f90940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 -- 172.21.15.16:0/1042506173 shutdown_connections 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 -- 172.21.15.16:0/1042506173 wait complete. 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 Processor -- start 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 -- start start 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9094074620 0x7f909419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9094074f40 0x7f909419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f909410e3c0 0x7f909419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.364+0000 7f909a8dd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90941141a0 con 0x7f909410e3c0 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f909a8dd640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9094114020 con 0x7f9094074620 2024-09-15T22:49:09.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f909a8dd640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9094114320 con 0x7f9094074f40 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f909410e3c0 0x7f909419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f909410e3c0 0x7f909419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51428/0 (socket says 172.21.15.16:51428) 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 -- 172.21.15.16:0/1522154637 learned_addr learned my addr 172.21.15.16:0/1522154637 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9093fff640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9094074620 0x7f909419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 -- 172.21.15.16:0/1522154637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9094074f40 msgr2=0x7f909419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9094074f40 0x7f909419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 -- 172.21.15.16:0/1522154637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9094074620 msgr2=0x7f909419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9094074620 0x7f909419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 -- 172.21.15.16:0/1522154637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909419f5f0 con 0x7f909410e3c0 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9093fff640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9094074620 0x7f909419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f9098e7a640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f909410e3c0 0x7f909419ef10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f908c00eaa0 tx=0x7f908c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f90917fa640 1 -- 172.21.15.16:0/1522154637 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c00cdb0 con 0x7f909410e3c0 2024-09-15T22:49:09.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909419f8e0 con 0x7f909410e3c0 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f90917fa640 1 -- 172.21.15.16:0/1522154637 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c005020 con 0x7f909410e3c0 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f90917fa640 1 -- 172.21.15.16:0/1522154637 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c0116b0 con 0x7f909410e3c0 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.365+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90941a71c0 con 0x7f909410e3c0 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f90917fa640 1 -- 172.21.15.16:0/1522154637 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f908c005560 con 0x7f909410e3c0 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 Processor -- start 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 -- start start 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac7a370 0x56086ac7c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56086abd52f0 con 0x56086ac7a370 2024-09-15T22:49:09.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56086ab7ccf0 con 0x56086ac6f5d0 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f909ccb6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56086abd12b0 con 0x56086ac70b40 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f90937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac7a370 0x56086ac7c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f90937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac7a370 0x56086ac7c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51434/0 (socket says 172.21.15.16:51434) 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f90937fe640 1 -- 172.21.15.16:0/1065948815 learned_addr learned my addr 172.21.15.16:0/1065948815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f9093fff640 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f90917fa640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f907c0a33b0 0x7f907c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f90937fe640 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.368+0000 7f90937fe640 1 -- 172.21.15.16:0/1065948815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 msgr2=0x56086ac6f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f9098e7a640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f907c0a33b0 0x7f907c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f90937fe640 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac6f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f90937fe640 1 -- 172.21.15.16:0/1065948815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 msgr2=0x56086ac6fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f90937fe640 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac6fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f90937fe640 1 -- 172.21.15.16:0/1065948815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086ab613a0 con 0x56086ac7a370 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f9093fff640 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac6fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f90937fe640 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac7a370 0x56086ac7c760 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9088009fc0 tx=0x7f90880094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0949944d5660363 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f90917fa640 1 -- 172.21.15.16:0/1522154637 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7f908c0d0430 con 0x7f909410e3c0 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f907a7fc640 1 -- 172.21.15.16:0/1065948815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088005220 con 0x56086ac7a370 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f907a7fc640 1 -- 172.21.15.16:0/1065948815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908800fe10 con 0x56086ac7a370 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.369+0000 7f907a7fc640 1 -- 172.21.15.16:0/1065948815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088012770 con 0x56086ac7a370 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 -- 172.21.15.16:0/1065948815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac7a370 msgr2=0x56086ac7c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f9098e7a640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f907c0a33b0 0x7f907c0a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f908c06be90 tx=0x7f908c097e20 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac7a370 0x56086ac7c760 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9088009fc0 tx=0x7f90880094e0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 -- 172.21.15.16:0/1065948815 shutdown_connections 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac7a370 0x56086ac7c760 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac6fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1065948815 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac6f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 -- 172.21.15.16:0/1065948815 >> 172.21.15.16:0/1065948815 conn(0x56086ac38310 msgr2=0x56086ac6bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 -- 172.21.15.16:0/1065948815 shutdown_connections 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 -- 172.21.15.16:0/1065948815 wait complete. 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 -- start start 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.370+0000 7f909ccb6740 1 Processor -- start 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac79d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac77030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac7a370 0x56086ac77640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56086ac7d070 con 0x56086ac70b40 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56086ab90060 con 0x56086ac6f5d0 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x56086ac7d1f0 con 0x56086ac7a370 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 10 client.0 _ll_register_callbacks cb 0x56086a8e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac77030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac77030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51444/0 (socket says 172.21.15.16:51444) 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 -- 172.21.15.16:0/575218389 learned_addr learned my addr 172.21.15.16:0/575218389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f90937fe640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac7a370 0x56086ac77640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f90937fe640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac79d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 -- 172.21.15.16:0/575218389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac7a370 msgr2=0x56086ac77640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac7a370 0x56086ac77640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 -- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 msgr2=0x56086ac79d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac79d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 -- 172.21.15.16:0/575218389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086a9463c0 con 0x56086ac70b40 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f90937fe640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac7a370 0x56086ac77640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f9093fff640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac77030 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9084009340 tx=0x7f90840052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1021f286091d007 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f90737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f90737fe640 1 -- 172.21.15.16:0/575218389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084035030 con 0x56086ac70b40 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f90737fe640 1 -- 172.21.15.16:0/575218389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7f90840048c0 con 0x56086ac70b40 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.371+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908800fe10 con 0x56086ac70b40 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f90737fe640 1 -- 172.21.15.16:0/575218389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f908404d050 con 0x56086ac70b40 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 10 client.17814 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56086ac88b40 con 0x56086ac70b40 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f90737fe640 1 -- 172.21.15.16:0/575218389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908800fe10 con 0x56086ac70b40 2024-09-15T22:49:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f90737fe640 1 -- 172.21.15.16:0/575218389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9084003bb0 con 0x56086ac70b40 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 10 client.17814 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 20 client.17814 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086ac8a000 0x56086ac8c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 4 client.17814 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56086abe2860 con 0x56086ac8a000 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.372+0000 7f90937fe640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086ac8a000 0x56086ac8c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.373+0000 7f90937fe640 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086ac8a000 0x56086ac8c410 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967468ca8f484fe8 server_cookie=c634c31ccb1b5789 in_seq=0 out_seq=0 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.373+0000 7f90737fe640 10 client.17814 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.373+0000 7f90737fe640 1 -- 172.21.15.16:0/575218389 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56086abe2860 con 0x56086ac8a000 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.373+0000 7f90737fe640 10 client.17814 handle_command_reply: tid=0 2024-09-15T22:49:09.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.373+0000 7f909ccb6740 1 client.17814 shutdown 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 msgr2=0x56086ac77030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac77030 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9084009340 tx=0x7f90840052f0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 shutdown_connections 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086ac8a000 0x56086ac8c410 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac7a370 0x56086ac77640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac77030 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 --2- 172.21.15.16:0/575218389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac6f5d0 0x56086ac79d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f90737fe640 0 client.17814 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 >> 172.21.15.16:0/575218389 conn(0x56086ac38310 msgr2=0x56086ac71960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 shutdown_connections 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 1 -- 172.21.15.16:0/575218389 wait complete. 2024-09-15T22:49:09.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.374+0000 7f909ccb6740 20 client.17814 trim_cache size 0 max 16384 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 Processor -- start 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 -- start start 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac6f5d0 0x56086ac78fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac72120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac72660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90941b3bf0 con 0x56086ac70b40 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90941b14c0 con 0x56086ac7a370 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.387+0000 7f909ccb6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90941af330 con 0x56086ac6f5d0 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac72120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac72120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51460/0 (socket says 172.21.15.16:51460) 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 -- 172.21.15.16:0/1902749956 learned_addr learned my addr 172.21.15.16:0/1902749956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:09.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f90937fe640 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac6f5d0 0x56086ac78fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f90937fe640 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac72660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 -- 172.21.15.16:0/1902749956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac6f5d0 msgr2=0x56086ac78fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac6f5d0 0x56086ac78fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 -- 172.21.15.16:0/1902749956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 msgr2=0x56086ac72660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac72660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 -- 172.21.15.16:0/1902749956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086ac8ca90 con 0x56086ac70b40 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f90937fe640 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac72660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9093fff640 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac72120 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9084009900 tx=0x7f9084009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98579f3cdff95f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9079ffb640 1 -- 172.21.15.16:0/1902749956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084000e60 con 0x56086ac70b40 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9079ffb640 1 -- 172.21.15.16:0/1902749956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908403f070 con 0x56086ac70b40 2024-09-15T22:49:09.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.388+0000 7f9079ffb640 1 -- 172.21.15.16:0/1902749956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084035030 con 0x56086ac70b40 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 -- 172.21.15.16:0/1902749956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 msgr2=0x56086ac72120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac72120 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9084009900 tx=0x7f9084009620 comp rx=0 tx=0).stop 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 -- 172.21.15.16:0/1902749956 shutdown_connections 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac72660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac70b40 0x56086ac72120 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 --2- 172.21.15.16:0/1902749956 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac6f5d0 0x56086ac78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 -- 172.21.15.16:0/1902749956 >> 172.21.15.16:0/1902749956 conn(0x56086ac38310 msgr2=0x56086ac78890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 -- 172.21.15.16:0/1902749956 shutdown_connections 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 -- 172.21.15.16:0/1902749956 wait complete. 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.389+0000 7f909ccb6740 1 -- start start 2024-09-15T22:49:09.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 1 Processor -- start 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac6f5d0 0x56086ac740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac74600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac74b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f90941af980 con 0x56086ac6f5d0 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90941b0df0 con 0x56086ac7a370 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f90941b0050 con 0x56086ac70b40 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f909ccb6740 10 client.0 _ll_register_callbacks cb 0x7f9084009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac6f5d0 0x56086ac740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac6f5d0 0x56086ac740c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51470/0 (socket says 172.21.15.16:51470) 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 -- 172.21.15.16:0/3778534847 learned_addr learned my addr 172.21.15.16:0/3778534847 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:09.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f9093fff640 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac74600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 -- 172.21.15.16:0/3778534847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 msgr2=0x56086ac74600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac74600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 -- 172.21.15.16:0/3778534847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 msgr2=0x56086ac74b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac74b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.390+0000 7f90937fe640 1 -- 172.21.15.16:0/3778534847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086acf3d00 con 0x56086ac6f5d0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f90937fe640 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac6f5d0 0x56086ac740c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9088018000 tx=0x7f90880096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbfa9cd634b9d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f90737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f90737fe640 1 -- 172.21.15.16:0/3778534847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088005380 con 0x56086ac6f5d0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f90737fe640 1 -- 172.21.15.16:0/3778534847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7f90880047a0 con 0x56086ac6f5d0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908403f070 con 0x56086ac6f5d0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f90737fe640 1 -- 172.21.15.16:0/3778534847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9088038050 con 0x56086ac6f5d0 2024-09-15T22:49:09.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 10 client.17820 fetch_fsmap learned FSMap version 26 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56086acf5000 con 0x56086ac6f5d0 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f90737fe640 1 -- 172.21.15.16:0/3778534847 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908403f070 con 0x56086ac6f5d0 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f90737fe640 1 -- 172.21.15.16:0/3778534847 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f908800c070 con 0x56086ac6f5d0 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 10 client.17820 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 20 client.17820 populate_metadata read hostname 'smithi016' 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086acf53a0 0x56086acf77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 4 client.17820 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.391+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 --> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56086abdcd70 con 0x56086acf53a0 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.392+0000 7f9093fff640 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086acf53a0 0x56086acf77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:09.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.392+0000 7f9093fff640 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086acf53a0 0x56086acf77c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601a8a025c4c9566 server_cookie=988a66d0eaa1136d in_seq=0 out_seq=0 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.392+0000 7f90737fe640 10 client.17820 ms_handle_connect on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.392+0000 7f90737fe640 1 -- 172.21.15.16:0/3778534847 <== mds.0 v2:172.21.15.59:6838/384228279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56086abdcd70 con 0x56086acf53a0 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.392+0000 7f90737fe640 10 client.17820 handle_command_reply: tid=0 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.392+0000 7f909ccb6740 1 client.17820 shutdown 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac6f5d0 msgr2=0x56086ac740c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac6f5d0 0x56086ac740c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9088018000 tx=0x7f90880096c0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 shutdown_connections 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.59:6838/384228279,v1:172.21.15.59:6839/384228279] conn(0x56086acf53a0 0x56086acf77c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56086ac7a370 0x56086ac74b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x56086ac70b40 0x56086ac74600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f90737fe640 0 client.17820 ms_handle_reset on v2:172.21.15.59:6838/384228279 2024-09-15T22:49:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 --2- 172.21.15.16:0/3778534847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56086ac6f5d0 0x56086ac740c0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 >> 172.21.15.16:0/3778534847 conn(0x56086ac38310 msgr2=0x56086ac790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 shutdown_connections 2024-09-15T22:49:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 1 -- 172.21.15.16:0/3778534847 wait complete. 2024-09-15T22:49:09.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909ccb6740 20 client.17820 trim_cache size 0 max 16384 2024-09-15T22:49:09.405 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-15T22:49:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f907c0a33b0 msgr2=0x7f907c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f907c0a33b0 0x7f907c0a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f908c06be90 tx=0x7f908c097e20 comp rx=0 tx=0).stop 2024-09-15T22:49:09.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f909410e3c0 msgr2=0x7f909419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.393+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f909410e3c0 0x7f909419ef10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f908c00eaa0 tx=0x7f908c00ef60 comp rx=0 tx=0).stop 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 shutdown_connections 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f909410e3c0 0x7f909419ef10 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f907c0a33b0 0x7f907c0a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9094074f40 0x7f909419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 --2- 172.21.15.16:0/1522154637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9094074620 0x7f909419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 >> 172.21.15.16:0/1522154637 conn(0x7f90940fd8c0 msgr2=0x7f90940fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 shutdown_connections 2024-09-15T22:49:09.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:09.394+0000 7f909a8dd640 1 -- 172.21.15.16:0/1522154637 wait complete. 2024-09-15T22:49:10.044 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:09 smithi016 ceph-mon[31492]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:49:10.080 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-15T22:49:10.081 INFO:tasks.fwd_scrub:done joining 2024-09-15T22:49:10.081 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-15T22:49:10.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T22:49:10.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:09 smithi059 ceph-mon[34872]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:49:10.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 -- 172.21.15.16:0/1553049269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84101aa0 msgr2=0x7fbc84101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/1553049269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84101aa0 0x7fbc84101f00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009930 tx=0x7fbc7802a420 comp rx=0 tx=0).stop 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 -- 172.21.15.16:0/1553049269 shutdown_connections 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/1553049269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc84102440 0x7fbc8410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/1553049269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84101aa0 0x7fbc84101f00 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/1553049269 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc84107ab0 0x7fbc84107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 -- 172.21.15.16:0/1553049269 >> 172.21.15.16:0/1553049269 conn(0x7fbc840fd7c0 msgr2=0x7fbc840ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 -- 172.21.15.16:0/1553049269 shutdown_connections 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 -- 172.21.15.16:0/1553049269 wait complete. 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 Processor -- start 2024-09-15T22:49:10.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.205+0000 7fbc8a65a640 1 -- start start 2024-09-15T22:49:10.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc8a65a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc84101aa0 0x7fbc840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc8a65a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84102440 0x7fbc84074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc8a65a640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc84107ab0 0x7fbc84074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc8a65a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84111ef0 con 0x7fbc84102440 2024-09-15T22:49:10.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc8a65a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84111d70 con 0x7fbc84101aa0 2024-09-15T22:49:10.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc8a65a640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84112070 con 0x7fbc84107ab0 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84102440 0x7fbc84074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84102440 0x7fbc84074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51488/0 (socket says 172.21.15.16:51488) 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 -- 172.21.15.16:0/4194950397 learned_addr learned my addr 172.21.15.16:0/4194950397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc83fff640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc84101aa0 0x7fbc840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc88bf7640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc84107ab0 0x7fbc84074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 -- 172.21.15.16:0/4194950397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc84107ab0 msgr2=0x7fbc84074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc84107ab0 0x7fbc84074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 -- 172.21.15.16:0/4194950397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc84101aa0 msgr2=0x7fbc840744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc84101aa0 0x7fbc840744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 -- 172.21.15.16:0/4194950397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc841a6920 con 0x7fbc84102440 2024-09-15T22:49:10.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc837fe640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84102440 0x7fbc84074a30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009230 tx=0x7fbc78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.206+0000 7fbc817fa640 1 -- 172.21.15.16:0/4194950397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7803f030 con 0x7fbc84102440 2024-09-15T22:49:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.207+0000 7fbc817fa640 1 -- 172.21.15.16:0/4194950397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc78003ad0 con 0x7fbc84102440 2024-09-15T22:49:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.207+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc841a6bb0 con 0x7fbc84102440 2024-09-15T22:49:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.207+0000 7fbc817fa640 1 -- 172.21.15.16:0/4194950397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78035070 con 0x7fbc84102440 2024-09-15T22:49:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.207+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc841a70a0 con 0x7fbc84102440 2024-09-15T22:49:10.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.208+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc84101f00 con 0x7fbc84102440 2024-09-15T22:49:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.209+0000 7fbc817fa640 1 -- 172.21.15.16:0/4194950397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbc78042030 con 0x7fbc84102440 2024-09-15T22:49:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.209+0000 7fbc817fa640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbc6c0a33c0 0x7fbc6c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.210+0000 7fbc83fff640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbc6c0a33c0 0x7fbc6c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.210+0000 7fbc817fa640 1 -- 172.21.15.16:0/4194950397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7fbc780f0990 con 0x7fbc84102440 2024-09-15T22:49:10.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.210+0000 7fbc83fff640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbc6c0a33c0 0x7fbc6c0a5880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbc74005fd0 tx=0x7fbc7400b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:10.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.213+0000 7fbc817fa640 1 -- 172.21.15.16:0/4194950397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc780bc8b0 con 0x7fbc84102440 2024-09-15T22:49:10.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:09 smithi130 ceph-mon[35742]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:49:10.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.391+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc8410a070 con 0x7fbc84102440 2024-09-15T22:49:10.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.393+0000 7fbc817fa640 1 -- 172.21.15.16:0/4194950397 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+25454 (secure 0 0 0) 0x7fbc780c0bb0 con 0x7fbc84102440 2024-09-15T22:49:10.395 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:49:10.395 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":99,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:49:08.874737+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3275020767":"2024-09-16T22:38:25.000235+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3377264779":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3984482177":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/1746300554":"2024-09-16T22:38:25.000235+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:49:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbc6c0a33c0 msgr2=0x7fbc6c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbc6c0a33c0 0x7fbc6c0a5880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbc74005fd0 tx=0x7fbc7400b390 comp rx=0 tx=0).stop 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84102440 msgr2=0x7fbc84074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84102440 0x7fbc84074a30 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009230 tx=0x7fbc78004270 comp rx=0 tx=0).stop 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 shutdown_connections 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fbc6c0a33c0 0x7fbc6c0a5880 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fbc84107ab0 0x7fbc84074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc84102440 0x7fbc84074a30 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 --2- 172.21.15.16:0/4194950397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc84101aa0 0x7fbc840744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 >> 172.21.15.16:0/4194950397 conn(0x7fbc840fd7c0 msgr2=0x7fbc840fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:10.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 shutdown_connections 2024-09-15T22:49:10.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.395+0000 7fbc8a65a640 1 -- 172.21.15.16:0/4194950397 wait complete. 2024-09-15T22:49:10.408 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:31:31.005703+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T22:36:04.983944+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T22:36:07.225026+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-15T22:36:13.700134+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-15T22:49:10.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T22:49:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 -- 172.21.15.16:0/689541416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1009b0 msgr2=0x7ff02c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 --2- 172.21.15.16:0/689541416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1009b0 0x7ff02c100e10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff020009930 tx=0x7ff02002a420 comp rx=0 tx=0).stop 2024-09-15T22:49:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 -- 172.21.15.16:0/689541416 shutdown_connections 2024-09-15T22:49:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 --2- 172.21.15.16:0/689541416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c101350 0x7ff02c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 --2- 172.21.15.16:0/689541416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1009b0 0x7ff02c100e10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 --2- 172.21.15.16:0/689541416 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff02c1069c0 0x7ff02c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 -- 172.21.15.16:0/689541416 >> 172.21.15.16:0/689541416 conn(0x7ff02c0fc690 msgr2=0x7ff02c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 -- 172.21.15.16:0/689541416 shutdown_connections 2024-09-15T22:49:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.514+0000 7ff03088d640 1 -- 172.21.15.16:0/689541416 wait complete. 2024-09-15T22:49:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 Processor -- start 2024-09-15T22:49:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 -- start start 2024-09-15T22:49:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff02c1009b0 0x7ff02c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c101350 0x7ff02c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1069c0 0x7ff02c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c110e00 con 0x7ff02c1069c0 2024-09-15T22:49:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c110c80 con 0x7ff02c101350 2024-09-15T22:49:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff03088d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c110f80 con 0x7ff02c1009b0 2024-09-15T22:49:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1069c0 0x7ff02c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1069c0 0x7ff02c19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51508/0 (socket says 172.21.15.16:51508) 2024-09-15T22:49:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 -- 172.21.15.16:0/3056203662 learned_addr learned my addr 172.21.15.16:0/3056203662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02a59c640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff02c1009b0 0x7ff02c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff029d9b640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c101350 0x7ff02c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 -- 172.21.15.16:0/3056203662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff02c1009b0 msgr2=0x7ff02c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff02c1009b0 0x7ff02c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 -- 172.21.15.16:0/3056203662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c101350 msgr2=0x7ff02c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c101350 0x7ff02c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 -- 172.21.15.16:0/3056203662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c19d4a0 con 0x7ff02c1069c0 2024-09-15T22:49:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02a59c640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff02c1009b0 0x7ff02c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:49:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.515+0000 7ff02ad9d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1069c0 0x7ff02c19cd70 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eaa0 tx=0x7ff02400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.516+0000 7ff01b7fe640 1 -- 172.21.15.16:0/3056203662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400cdb0 con 0x7ff02c1069c0 2024-09-15T22:49:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.516+0000 7ff01b7fe640 1 -- 172.21.15.16:0/3056203662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff024005020 con 0x7ff02c1069c0 2024-09-15T22:49:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.516+0000 7ff01b7fe640 1 -- 172.21.15.16:0/3056203662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240116b0 con 0x7ff02c1069c0 2024-09-15T22:49:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.516+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c19d790 con 0x7ff02c1069c0 2024-09-15T22:49:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.516+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a5020 con 0x7ff02c1069c0 2024-09-15T22:49:10.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.517+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff02c100e10 con 0x7ff02c1069c0 2024-09-15T22:49:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.518+0000 7ff01b7fe640 1 -- 172.21.15.16:0/3056203662 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff024005560 con 0x7ff02c1069c0 2024-09-15T22:49:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.519+0000 7ff01b7fe640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0140a33c0 0x7ff0140a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.519+0000 7ff02a59c640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0140a33c0 0x7ff0140a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.519+0000 7ff01b7fe640 1 -- 172.21.15.16:0/3056203662 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7ff0240d0850 con 0x7ff02c1069c0 2024-09-15T22:49:10.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.519+0000 7ff02a59c640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0140a33c0 0x7ff0140a5880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c0094d0 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:10.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.522+0000 7ff01b7fe640 1 -- 172.21.15.16:0/3056203662 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff024016030 con 0x7ff02c1069c0 2024-09-15T22:49:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.697+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff02c02d7c0 con 0x7ff02c1069c0 2024-09-15T22:49:10.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.698+0000 7ff01b7fe640 1 -- 172.21.15.16:0/3056203662 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7ff02409cdc0 con 0x7ff02c1069c0 2024-09-15T22:49:10.699 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-15T22:49:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0140a33c0 msgr2=0x7ff0140a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0140a33c0 0x7ff0140a5880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff01c005fd0 tx=0x7ff01c0094d0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1069c0 msgr2=0x7ff02c19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1069c0 0x7ff02c19cd70 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eaa0 tx=0x7ff02400ef60 comp rx=0 tx=0).stop 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 shutdown_connections 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff02c1069c0 0x7ff02c19cd70 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff02c101350 0x7ff02c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7ff0140a33c0 0x7ff0140a5880 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 --2- 172.21.15.16:0/3056203662 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7ff02c1009b0 0x7ff02c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 >> 172.21.15.16:0/3056203662 conn(0x7ff02c0fc690 msgr2=0x7ff02c06a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 shutdown_connections 2024-09-15T22:49:10.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.700+0000 7ff03088d640 1 -- 172.21.15.16:0/3056203662 wait complete. 2024-09-15T22:49:10.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.818+0000 7fcf1f96d640 1 -- 172.21.15.16:0/2472773751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18106990 msgr2=0x7fcf18106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.818+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/2472773751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18106990 0x7fcf18106d70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009930 tx=0x7fcf1402a420 comp rx=0 tx=0).stop 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- 172.21.15.16:0/2472773751 shutdown_connections 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/2472773751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf18101320 0x7fcf1810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/2472773751 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf18100980 0x7fcf18100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/2472773751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18106990 0x7fcf18106d70 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- 172.21.15.16:0/2472773751 >> 172.21.15.16:0/2472773751 conn(0x7fcf180779d0 msgr2=0x7fcf180fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- 172.21.15.16:0/2472773751 shutdown_connections 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- 172.21.15.16:0/2472773751 wait complete. 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 Processor -- start 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- start start 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18100980 0x7fcf18197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf18101320 0x7fcf18197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf18106990 0x7fcf1819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18110dd0 con 0x7fcf18100980 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18110c50 con 0x7fcf18101320 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.819+0000 7fcf1f96d640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fcf18110f50 con 0x7fcf18106990 2024-09-15T22:49:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18100980 0x7fcf18197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18100980 0x7fcf18197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51520/0 (socket says 172.21.15.16:51520) 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 -- 172.21.15.16:0/1182272676 learned_addr learned my addr 172.21.15.16:0/1182272676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1df0a640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf18106990 0x7fcf1819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1cf08640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf18101320 0x7fcf18197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 -- 172.21.15.16:0/1182272676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf18106990 msgr2=0x7fcf1819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf18106990 0x7fcf1819bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 -- 172.21.15.16:0/1182272676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf18101320 msgr2=0x7fcf18197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf18101320 0x7fcf18197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 -- 172.21.15.16:0/1182272676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1819c320 con 0x7fcf18100980 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1d709640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18100980 0x7fcf18197330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009230 tx=0x7fcf14004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf0e7fc640 1 -- 172.21.15.16:0/1182272676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1403f030 con 0x7fcf18100980 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf0e7fc640 1 -- 172.21.15.16:0/1182272676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf14004440 con 0x7fcf18100980 2024-09-15T22:49:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1819c5b0 con 0x7fcf18100980 2024-09-15T22:49:10.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf0e7fc640 1 -- 172.21.15.16:0/1182272676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14035070 con 0x7fcf18100980 2024-09-15T22:49:10.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.820+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf181a3ea0 con 0x7fcf18100980 2024-09-15T22:49:10.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.821+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf181020c0 con 0x7fcf18100980 2024-09-15T22:49:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.823+0000 7fcf0e7fc640 1 -- 172.21.15.16:0/1182272676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcf14042030 con 0x7fcf18100980 2024-09-15T22:49:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.823+0000 7fcf0e7fc640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcf040a33c0 0x7fcf040a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.823+0000 7fcf0e7fc640 1 -- 172.21.15.16:0/1182272676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7fcf140f08d0 con 0x7fcf18100980 2024-09-15T22:49:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.823+0000 7fcf1cf08640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcf040a33c0 0x7fcf040a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.824+0000 7fcf1cf08640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcf040a33c0 0x7fcf040a5880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcf180fe9c0 tx=0x7fcf08005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:10.826+0000 7fcf0e7fc640 1 -- 172.21.15.16:0/1182272676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf140bcea0 con 0x7fcf18100980 2024-09-15T22:49:11.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.004+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf18108ef0 con 0x7fcf18100980 2024-09-15T22:49:11.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.004+0000 7fcf0e7fc640 1 -- 172.21.15.16:0/1182272676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7fcf140bc7a0 con 0x7fcf18100980 2024-09-15T22:49:11.006 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcf040a33c0 msgr2=0x7fcf040a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcf040a33c0 0x7fcf040a5880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcf180fe9c0 tx=0x7fcf08005f90 comp rx=0 tx=0).stop 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18100980 msgr2=0x7fcf18197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18100980 0x7fcf18197330 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009230 tx=0x7fcf14004040 comp rx=0 tx=0).stop 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 shutdown_connections 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fcf040a33c0 0x7fcf040a5880 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fcf18106990 0x7fcf1819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf18101320 0x7fcf18197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 --2- 172.21.15.16:0/1182272676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf18100980 0x7fcf18197330 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 >> 172.21.15.16:0/1182272676 conn(0x7fcf180779d0 msgr2=0x7fcf180ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 shutdown_connections 2024-09-15T22:49:11.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.006+0000 7fcf1f96d640 1 -- 172.21.15.16:0/1182272676 wait complete. 2024-09-15T22:49:11.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:10 smithi016 ceph-mon[31492]: pgmap v334: 129 pgs: 129 active+clean; 17 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:49:11.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:10 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/4194950397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:49:11.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:10 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3056203662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:49:11.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T22:49:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 -- 172.21.15.16:0/2055783160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec1035a0 msgr2=0x7f76ec10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 --2- 172.21.15.16:0/2055783160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec1035a0 0x7f76ec10ba80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009900 tx=0x7f76e002a480 comp rx=0 tx=0).stop 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 -- 172.21.15.16:0/2055783160 shutdown_connections 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 --2- 172.21.15.16:0/2055783160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec10c0d0 0x7f76ec10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 --2- 172.21.15.16:0/2055783160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec1035a0 0x7f76ec10ba80 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 --2- 172.21.15.16:0/2055783160 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76ec102c80 0x7f76ec103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 -- 172.21.15.16:0/2055783160 >> 172.21.15.16:0/2055783160 conn(0x7f76ec0fc690 msgr2=0x7f76ec0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 -- 172.21.15.16:0/2055783160 shutdown_connections 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 -- 172.21.15.16:0/2055783160 wait complete. 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 Processor -- start 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.137+0000 7f76f3053640 1 -- start start 2024-09-15T22:49:11.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f3053640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76ec102c80 0x7f76ec198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f3053640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1035a0 0x7f76ec198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f3053640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec10c0d0 0x7f76ec19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f3053640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec111ef0 con 0x7f76ec10c0d0 2024-09-15T22:49:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f3053640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec111d70 con 0x7f76ec1035a0 2024-09-15T22:49:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f3053640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec112070 con 0x7f76ec102c80 2024-09-15T22:49:11.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec10c0d0 0x7f76ec19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec10c0d0 0x7f76ec19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51526/0 (socket says 172.21.15.16:51526) 2024-09-15T22:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 -- 172.21.15.16:0/3523685041 learned_addr learned my addr 172.21.15.16:0/3523685041 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:11.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f0def640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76ec102c80 0x7f76ec198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76ebfff640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1035a0 0x7f76ec198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 -- 172.21.15.16:0/3523685041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76ec102c80 msgr2=0x7f76ec198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76ec102c80 0x7f76ec198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 -- 172.21.15.16:0/3523685041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1035a0 msgr2=0x7f76ec198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1035a0 0x7f76ec198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 -- 172.21.15.16:0/3523685041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec19d520 con 0x7f76ec10c0d0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76ebfff640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1035a0 0x7f76ec198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76f15f0640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec10c0d0 0x7f76ec19cdf0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f76e400b970 tx=0x7f76e400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76e9ffb640 1 -- 172.21.15.16:0/3523685041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4019070 con 0x7f76ec10c0d0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.138+0000 7f76e9ffb640 1 -- 172.21.15.16:0/3523685041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e400cb40 con 0x7f76ec10c0d0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.139+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ec19d810 con 0x7f76ec10c0d0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.139+0000 7f76e9ffb640 1 -- 172.21.15.16:0/3523685041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4007650 con 0x7f76ec10c0d0 2024-09-15T22:49:11.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.139+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ec1a50a0 con 0x7f76ec10c0d0 2024-09-15T22:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.140+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76ec103060 con 0x7f76ec10c0d0 2024-09-15T22:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.141+0000 7f76e9ffb640 1 -- 172.21.15.16:0/3523685041 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f76e401c070 con 0x7f76ec10c0d0 2024-09-15T22:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.142+0000 7f76e9ffb640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76d40a3440 0x7f76d40a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.142+0000 7f76f0def640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76d40a3440 0x7f76d40a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.142+0000 7f76e9ffb640 1 -- 172.21.15.16:0/3523685041 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7f76e40cf620 con 0x7f76ec10c0d0 2024-09-15T22:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.142+0000 7f76f0def640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76d40a3440 0x7f76d40a5900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f76dc005fd0 tx=0x7f76dc00b590 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:11.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.146+0000 7f76e9ffb640 1 -- 172.21.15.16:0/3523685041 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76e409bd00 con 0x7f76ec10c0d0 2024-09-15T22:49:11.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:10 smithi059 ceph-mon[34872]: pgmap v334: 129 pgs: 129 active+clean; 17 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:49:11.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:10 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/4194950397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:49:11.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:10 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3056203662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:49:11.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:10 smithi130 ceph-mon[35742]: pgmap v334: 129 pgs: 129 active+clean; 17 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-09-15T22:49:11.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:10 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/4194950397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:49:11.279 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:10 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3056203662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:49:11.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.326+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f76ec02d7c0 con 0x7f76ec10c0d0 2024-09-15T22:49:11.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.326+0000 7f76e9ffb640 1 -- 172.21.15.16:0/3523685041 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7f76e409b600 con 0x7f76ec10c0d0 2024-09-15T22:49:11.328 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-15T22:49:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76d40a3440 msgr2=0x7f76d40a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76d40a3440 0x7f76d40a5900 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f76dc005fd0 tx=0x7f76dc00b590 comp rx=0 tx=0).stop 2024-09-15T22:49:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec10c0d0 msgr2=0x7f76ec19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec10c0d0 0x7f76ec19cdf0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f76e400b970 tx=0x7f76e400be30 comp rx=0 tx=0).stop 2024-09-15T22:49:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 shutdown_connections 2024-09-15T22:49:11.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ec10c0d0 0x7f76ec19cdf0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1035a0 0x7f76ec198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76d40a3440 0x7f76d40a5900 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 --2- 172.21.15.16:0/3523685041 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f76ec102c80 0x7f76ec198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 >> 172.21.15.16:0/3523685041 conn(0x7f76ec0fc690 msgr2=0x7f76ec0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 shutdown_connections 2024-09-15T22:49:11.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.329+0000 7f76f3053640 1 -- 172.21.15.16:0/3523685041 wait complete. 2024-09-15T22:49:11.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-15T22:49:11.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 -- 172.21.15.16:0/3901924787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700106990 msgr2=0x7f7700106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3901924787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700106990 0x7f7700106d70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009930 tx=0x7f76f802a420 comp rx=0 tx=0).stop 2024-09-15T22:49:11.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 -- 172.21.15.16:0/3901924787 shutdown_connections 2024-09-15T22:49:11.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3901924787 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7700101320 0x7f770010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3901924787 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7700100980 0x7f7700100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3901924787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700106990 0x7f7700106d70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 -- 172.21.15.16:0/3901924787 >> 172.21.15.16:0/3901924787 conn(0x7f77000779d0 msgr2=0x7f77000fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:11.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 -- 172.21.15.16:0/3901924787 shutdown_connections 2024-09-15T22:49:11.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.460+0000 7f7704ae6640 1 -- 172.21.15.16:0/3901924787 wait complete. 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 Processor -- start 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 -- start start 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7700100980 0x7f7700197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700101320 0x7f77001978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7700106990 0x7f770019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7700110dd0 con 0x7f7700101320 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7700110c50 con 0x7f7700100980 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f7704ae6640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f7700110f50 con 0x7f7700106990 2024-09-15T22:49:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700101320 0x7f77001978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700101320 0x7f77001978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51556/0 (socket says 172.21.15.16:51556) 2024-09-15T22:49:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 -- 172.21.15.16:0/3942309724 learned_addr learned my addr 172.21.15.16:0/3942309724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fe59c640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7700100980 0x7f7700197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fed9d640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7700106990 0x7f770019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 -- 172.21.15.16:0/3942309724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7700106990 msgr2=0x7f770019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7700106990 0x7f770019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 -- 172.21.15.16:0/3942309724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7700100980 msgr2=0x7f7700197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7700100980 0x7f7700197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 -- 172.21.15.16:0/3942309724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770019c350 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fe59c640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7700100980 0x7f7700197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.461+0000 7f76fdd9b640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700101320 0x7f77001978a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009860 tx=0x7f76e8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.462+0000 7f76f77fe640 1 -- 172.21.15.16:0/3942309724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8005020 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.462+0000 7f76f77fe640 1 -- 172.21.15.16:0/3942309724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76e80047a0 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.462+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770019c640 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.462+0000 7f76f77fe640 1 -- 172.21.15.16:0/3942309724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8007ba0 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.462+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7700069380 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.463+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7700100de0 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.464+0000 7f76f77fe640 1 -- 172.21.15.16:0/3942309724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f76e8004950 con 0x7f7700101320 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.465+0000 7f76f77fe640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76ec0a33c0 0x7f76ec0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.465+0000 7f76fe59c640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76ec0a33c0 0x7f76ec0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.465+0000 7f76f77fe640 1 -- 172.21.15.16:0/3942309724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7f76e80d1ee0 con 0x7f7700101320 2024-09-15T22:49:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.465+0000 7f76fe59c640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76ec0a33c0 0x7f76ec0a5880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76f8000c00 tx=0x7f76f8032c70 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.468+0000 7f76f77fe640 1 -- 172.21.15.16:0/3942309724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76e809e4b0 con 0x7f7700101320 2024-09-15T22:49:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.642+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7700069da0 con 0x7f7700101320 2024-09-15T22:49:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.643+0000 7f76f77fe640 1 -- 172.21.15.16:0/3942309724 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7f76e809ddb0 con 0x7f7700101320 2024-09-15T22:49:11.645 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-15T22:49:11.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76ec0a33c0 msgr2=0x7f76ec0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76ec0a33c0 0x7f76ec0a5880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76f8000c00 tx=0x7f76f8032c70 comp rx=0 tx=0).stop 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700101320 msgr2=0x7f77001978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700101320 0x7f77001978a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009860 tx=0x7f76e8009d20 comp rx=0 tx=0).stop 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 shutdown_connections 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f76ec0a33c0 0x7f76ec0a5880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f7700106990 0x7f770019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7700101320 0x7f77001978a0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 --2- 172.21.15.16:0/3942309724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7700100980 0x7f7700197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 >> 172.21.15.16:0/3942309724 conn(0x7f77000779d0 msgr2=0x7f77000fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 shutdown_connections 2024-09-15T22:49:11.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.645+0000 7f7704ae6640 1 -- 172.21.15.16:0/3942309724 wait complete. 2024-09-15T22:49:11.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-15T22:49:11.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 -- 172.21.15.16:0/3564117683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc10c0c0 msgr2=0x7fd3fc10e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 --2- 172.21.15.16:0/3564117683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc10c0c0 0x7fd3fc10e530 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009ca0 tx=0x7fd3f402a950 comp rx=0 tx=0).stop 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 -- 172.21.15.16:0/3564117683 shutdown_connections 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 --2- 172.21.15.16:0/3564117683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc10c0c0 0x7fd3fc10e530 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 --2- 172.21.15.16:0/3564117683 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3fc103610 0x7fd3fc10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 --2- 172.21.15.16:0/3564117683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3fc102cf0 0x7fd3fc1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 -- 172.21.15.16:0/3564117683 >> 172.21.15.16:0/3564117683 conn(0x7fd3fc077a10 msgr2=0x7fd3fc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 -- 172.21.15.16:0/3564117683 shutdown_connections 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 -- 172.21.15.16:0/3564117683 wait complete. 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 Processor -- start 2024-09-15T22:49:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.760+0000 7fd4028ed640 1 -- start start 2024-09-15T22:49:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd4028ed640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc102cf0 0x7fd3fc100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd4028ed640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3fc103610 0x7fd3fc100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd4028ed640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3fc10c0c0 0x7fd3fc1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd4028ed640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc111e70 con 0x7fd3fc102cf0 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd4028ed640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc111cf0 con 0x7fd3fc103610 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd4028ed640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc111ff0 con 0x7fd3fc10c0c0 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc102cf0 0x7fd3fc100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc102cf0 0x7fd3fc100940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39546/0 (socket says 172.21.15.16:39546) 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 -- 172.21.15.16:0/2393467478 learned_addr learned my addr 172.21.15.16:0/2393467478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd400e8a640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3fc10c0c0 0x7fd3fc1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fb7fe640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3fc103610 0x7fd3fc100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 -- 172.21.15.16:0/2393467478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3fc10c0c0 msgr2=0x7fd3fc1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3fc10c0c0 0x7fd3fc1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 -- 172.21.15.16:0/2393467478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3fc103610 msgr2=0x7fd3fc100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3fc103610 0x7fd3fc100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 -- 172.21.15.16:0/2393467478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc1a6880 con 0x7fd3fc102cf0 2024-09-15T22:49:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3fbfff640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc102cf0 0x7fd3fc100940 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d950 tx=0x7fd3ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd3f97fa640 1 -- 172.21.15.16:0/2393467478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec017030 con 0x7fd3fc102cf0 2024-09-15T22:49:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.761+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc1a6b70 con 0x7fd3fc102cf0 2024-09-15T22:49:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.762+0000 7fd3f97fa640 1 -- 172.21.15.16:0/2393467478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec004020 con 0x7fd3fc102cf0 2024-09-15T22:49:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.762+0000 7fd3f97fa640 1 -- 172.21.15.16:0/2393467478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec0047a0 con 0x7fd3fc102cf0 2024-09-15T22:49:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.762+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc1a7110 con 0x7fd3fc102cf0 2024-09-15T22:49:11.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.763+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc1030d0 con 0x7fd3fc102cf0 2024-09-15T22:49:11.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.764+0000 7fd3f97fa640 1 -- 172.21.15.16:0/2393467478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd3ec0041d0 con 0x7fd3fc102cf0 2024-09-15T22:49:11.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.765+0000 7fd3f97fa640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd3e40a33c0 0x7fd3e40a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:11.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.765+0000 7fd3fb7fe640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd3e40a33c0 0x7fd3e40a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.765+0000 7fd3f97fa640 1 -- 172.21.15.16:0/2393467478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9555+0+0 (secure 0 0 0) 0x7fd3ec0cfbc0 con 0x7fd3fc102cf0 2024-09-15T22:49:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.765+0000 7fd3fb7fe640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd3e40a33c0 0x7fd3e40a5880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc1023d0 tx=0x7fd3f0005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:11.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.769+0000 7fd3f97fa640 1 -- 172.21.15.16:0/2393467478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3ec09c190 con 0x7fd3fc102cf0 2024-09-15T22:49:11.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:11.992+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd3fc02d7c0 con 0x7fd3fc102cf0 2024-09-15T22:49:12.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.000+0000 7fd3f97fa640 1 -- 172.21.15.16:0/2393467478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742872 (secure 0 0 0) 0x7fd3ec09ba90 con 0x7fd3fc102cf0 2024-09-15T22:49:12.011 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:49:12.011 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:49:12.011 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 26, 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 15234, 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1157292381 2024-09-15T22:49:12.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1157292381 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/1157292381", 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-09-15T22:38:25.000605+0000", 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540701547738038271, 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-09-15T22:49:12.013 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24925, 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.018 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-15T22:49:12.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-15T22:49:12.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.022 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-15T22:49:12.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-15T22:49:12.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.030 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.042 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-15T22:49:12.047 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-15T22:49:12.048 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-15T22:49:12.048 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-15T22:49:12.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:49:12.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:49:12.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.054 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-15T22:49:12.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:49:12.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.056 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-15T22:49:12.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-15T22:49:12.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-15T22:49:12.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:49:12.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-15T22:49:12.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:49:12.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:49:12.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:49:12.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.073 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.074 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.075 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-15T22:49:12.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:49:12.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.089 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:49:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:49:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:49:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:49:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-15T22:49:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-15T22:49:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-15T22:49:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:49:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-15T22:49:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:49:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:49:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:49:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:49:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:49:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:49:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:49:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:49:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.173 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:49:12.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.175 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.184 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.185 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.187 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:49:12.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:49:12.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:49:12.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-15T22:49:12.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:49:12.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-15T22:49:12.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-15T22:49:12.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.200 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-15T22:49:12.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.214 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-15T22:49:12.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:49:12.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.231 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.232 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.236 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.241 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.263 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.265 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.267 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.269 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-15T22:49:12.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.273 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.275 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-15T22:49:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-15T22:49:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:49:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-15T22:49:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-15T22:49:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-15T22:49:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:49:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-15T22:49:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-15T22:49:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-15T22:49:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-15T22:49:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-15T22:49:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-15T22:49:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-15T22:49:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-15T22:49:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24941, 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540701547738038271, 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-15T22:49:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:49:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:49:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:49:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-15T22:49:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-15T22:49:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:49:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:49:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:49:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-15T22:49:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-15T22:49:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-15T22:49:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:49:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-15T22:49:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-15T22:49:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-15T22:49:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-15T22:49:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-15T22:49:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-15T22:49:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:49:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:49:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:49:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-15T22:49:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-15T22:49:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-15T22:49:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-15T22:49:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-15T22:49:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-15T22:49:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-15T22:49:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-15T22:49:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-15T22:49:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-15T22:49:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:49:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-15T22:49:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:49:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-15T22:49:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:49:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:49:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-15T22:49:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-15T22:49:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.848 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-15T22:49:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-15T22:49:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-15T22:49:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-15T22:49:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-15T22:49:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-15T22:49:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-15T22:49:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-15T22:49:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-15T22:49:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-15T22:49:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-15T22:49:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:49:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-15T22:49:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "restful", 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_schedule" 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:49:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-15T22:49:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-15T22:49:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-15T22:49:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-15T22:49:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-15T22:49:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-15T22:49:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:49:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:49:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:49:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:49:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:49:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:49:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:49:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:49:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:49:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:49:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:49:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-15T22:49:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-15T22:49:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-15T22:49:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-15T22:49:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.055 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.066 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:49:13.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:13.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-15T22:49:13.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-15T22:49:13.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-15T22:49:13.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:13.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.090 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-15T22:49:13.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-15T22:49:13.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.104 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.105 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.106 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-15T22:49:13.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-15T22:49:13.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-15T22:49:13.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-15T22:49:13.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.115 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.116 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.119 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.124 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.125 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.129 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.143 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-15T22:49:13.145 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.146 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:49:13.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-15T22:49:13.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.154 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.158 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.163 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.187 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-15T22:49:13.189 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.191 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-15T22:49:13.196 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.197 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.198 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.202 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.205 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.206 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.209 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.215 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.219 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.220 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.221 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.225 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.226 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.227 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.228 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.231 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.233 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.234 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.235 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.236 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.237 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-15T22:49:13.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.244 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.246 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.249 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.252 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:49:13.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.266 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:49:13.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.275 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-15T22:49:13.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-15T22:49:13.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-15T22:49:13.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.280 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.281 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.282 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.283 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.284 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.285 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.295 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.298 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.306 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.308 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.314 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-15T22:49:13.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.317 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.318 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.319 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.320 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.321 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.322 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.323 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.330 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.341 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.344 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-15T22:49:13.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-15T22:49:13.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-15T22:49:13.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-15T22:49:13.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-15T22:49:13.359 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-15T22:49:13.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-15T22:49:13.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-15T22:49:13.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-15T22:49:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.364 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-15T22:49:13.365 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-15T22:49:13.366 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.367 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-09-15T22:49:13.368 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 90, 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2736330462 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-15T22:49:13.370 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1661514991 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3960833451 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.371 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 436993741 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-15T22:49:13.372 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2292770399 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-15T22:49:13.373 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1654070069 2024-09-15T22:49:13.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.374 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:13.374 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-15T22:49:13.374 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:49:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd3e40a33c0 msgr2=0x7fd3e40a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd3e40a33c0 0x7fd3e40a5880 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc1023d0 tx=0x7fd3f0005f90 comp rx=0 tx=0).stop 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc102cf0 msgr2=0x7fd3fc100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc102cf0 0x7fd3fc100940 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d950 tx=0x7fd3ec00de10 comp rx=0 tx=0).stop 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 shutdown_connections 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fd3e40a33c0 0x7fd3e40a5880 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fd3fc10c0c0 0x7fd3fc1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3fc103610 0x7fd3fc100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 --2- 172.21.15.16:0/2393467478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3fc102cf0 0x7fd3fc100940 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 >> 172.21.15.16:0/2393467478 conn(0x7fd3fc077a10 msgr2=0x7fd3fc0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 shutdown_connections 2024-09-15T22:49:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:12.006+0000 7fd4028ed640 1 -- 172.21.15.16:0/2393467478 wait complete. 2024-09-15T22:49:13.376 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:11 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1182272676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:11 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3523685041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:11 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3942309724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:12 smithi130 ceph-mon[35742]: pgmap v335: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:12 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2393467478' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:11 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1182272676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:11 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3523685041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:11 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3942309724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:12 smithi059 ceph-mon[34872]: pgmap v335: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:12 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2393467478' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:11 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1182272676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:11 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3523685041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.377 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:11 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3942309724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:49:13.427 DEBUG:tasks.check_counter:Getting stats from a 2024-09-15T22:49:13.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-15T22:49:13.427 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:12 smithi016 ceph-mon[31492]: pgmap v335: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:13.428 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:12 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2393467478' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:49:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:14.107 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:13 smithi016 ceph-mon[31492]: pgmap v336: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:14.193 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:13 smithi059 ceph-mon[34872]: pgmap v336: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:14.212 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:49:14.212 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:14.212 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-15T22:49:14.212 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-15T22:49:14.212 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:49:14.212 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-15T22:49:14.212 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001459177, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000280469, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:14.213 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 230, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 217, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1590672, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 132352, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.043728077, 2024-09-15T22:49:14.214 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.016772068, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.039144536, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 217, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.017156064, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000079060 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.215 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1590672, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 132352 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 164, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 186000, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.014631762, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.010083329, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001944912, 2024-09-15T22:49:14.216 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 161, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.009802817, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000060887 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 186000 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.217 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:14.218 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:14.219 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:14.220 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 149034, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 32, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-15T22:49:14.221 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-15T22:49:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 228, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1555782, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 228, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-15T22:49:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 228, 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1555782, 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:14.224 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:49:14.278 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:13 smithi130 ceph-mon[35742]: pgmap v336: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:15.030 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-15T22:49:15.031 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-15T22:49:15.031 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-15T22:49:15.031 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-15T22:49:15.031 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-15T22:49:15.031 DEBUG:tasks.check_counter:Getting stats from d 2024-09-15T22:49:15.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-15T22:49:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:15.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:15 smithi016 ceph-mon[31492]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:49:15.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:15 smithi059 ceph-mon[34872]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:49:15.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:15 smithi130 ceph-mon[35742]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:49:15.779 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:49:15.779 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:15.779 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 231, 2024-09-15T22:49:15.780 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 220, 2024-09-15T22:49:15.780 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1589040, 2024-09-15T22:49:15.780 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 135104, 2024-09-15T22:49:15.780 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:15.780 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-15T22:49:15.780 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.043409212, 2024-09-15T22:49:15.780 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.015979160, 2024-09-15T22:49:15.781 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.038644968, 2024-09-15T22:49:15.781 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:15.781 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:15.781 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 219, 2024-09-15T22:49:15.781 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012018480, 2024-09-15T22:49:15.781 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000054878 2024-09-15T22:49:15.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.782 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:15.782 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:15.782 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:15.782 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:15.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1589040, 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 135104 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 6, 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 170, 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3312, 2024-09-15T22:49:15.783 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 187152, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.016138979, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.009745482, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003968427, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 168, 2024-09-15T22:49:15.784 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.018126706, 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000107897 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 187152 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.785 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.000285839, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000201104, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:15.786 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-15T22:49:15.787 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:15.788 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:15.789 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148977, 2024-09-15T22:49:15.790 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:15.791 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-15T22:49:15.792 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 233, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1556432, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 233, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 233, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1556432, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:15.793 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:15.794 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:15.794 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:15.794 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:15.794 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:49:16.556 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:16 smithi016 ceph-mon[31492]: pgmap v338: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:49:16.665 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-15T22:49:16.666 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-15T22:49:16.666 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-15T22:49:16.666 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-15T22:49:16.666 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-15T22:49:16.666 DEBUG:tasks.check_counter:Getting stats from g 2024-09-15T22:49:16.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-15T22:49:16.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:16 smithi059 ceph-mon[34872]: pgmap v338: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:49:16.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:16 smithi130 ceph-mon[35742]: pgmap v338: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-09-15T22:49:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:17.397 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:49:17.397 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:17.397 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-15T22:49:17.397 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-15T22:49:17.397 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001379038, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000151217, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 232, 2024-09-15T22:49:17.399 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 219, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1591216, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 133632, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.043021465, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.015536481, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.039932776, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 219, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.015897163, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000072589 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.400 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:17.401 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:17.401 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:17.401 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:17.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.401 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1591216, 2024-09-15T22:49:17.401 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 133632 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 166, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 186352, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.016219872, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.011552310, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002329546, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:17.402 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 163, 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.011579554, 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000071040 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 186352 2024-09-15T22:49:17.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.404 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-15T22:49:17.404 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-15T22:49:17.404 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-15T22:49:17.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:17.405 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:17.406 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:17.407 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148977, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:17.408 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:17.409 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 230, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1556042, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 230, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 230, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1556042, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:17.410 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:17.411 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:17.411 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:17.411 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:49:18.098 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-15T22:49:18.099 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-15T22:49:18.099 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-15T22:49:18.099 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-15T22:49:18.099 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-15T22:49:18.099 DEBUG:tasks.check_counter:Getting stats from j 2024-09-15T22:49:18.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-15T22:49:18.361 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:18.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:18 smithi059 ceph-mon[34872]: pgmap v339: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2024-09-15T22:49:18.727 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:18 smithi016 ceph-mon[31492]: pgmap v339: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2024-09-15T22:49:18.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:18 smithi130 ceph-mon[35742]: pgmap v339: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 245 B/s wr, 7 op/s 2024-09-15T22:49:18.832 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 233, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 222, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 1589584, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 136384, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.044082527, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.016504414, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.040488692, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:18.833 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 221, 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.012016040, 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000054371 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 1589584, 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 136384 2024-09-15T22:49:18.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 2, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 170, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 320, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 187056, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.014312808, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.009927989, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002155272, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 167, 2024-09-15T22:49:18.835 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.014238154, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000085258 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 187056 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-15T22:49:18.836 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2992, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.002476097, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000043852, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002437729, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004388480, 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001462826 2024-09-15T22:49:18.837 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-15T22:49:18.838 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:18.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-15T22:49:18.841 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148955, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:18.842 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 928, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:18.843 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 235, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 1556692, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 235, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-15T22:49:18.844 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 235, 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 1556692, 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-15T22:49:18.845 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-15T22:49:19.678 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-15T22:49:19.678 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-15T22:49:19.678 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-15T22:49:19.679 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-15T22:49:19.679 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-15T22:49:19.679 DEBUG:tasks.check_counter:Getting stats from b 2024-09-15T22:49:19.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-15T22:49:19.953 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:49:20.564 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 3464, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 3225, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 924445, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 11291820, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 170, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.308091020, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.078919818, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.109761755, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.088270346, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3219, 2024-09-15T22:49:20.565 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.302278923, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000093904 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 540, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000099338, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000183 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 139248, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 32400 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 7803, 2024-09-15T22:49:20.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 12064, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2852880, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 5236700, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 10, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 6, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.559893010, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.197058263, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.336778959, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.087196978, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 12066, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.437452440, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000036254 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 14527, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.002304586, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000158 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 461824 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 5178, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 4429, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 3181717, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 9420859, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 19, 2024-09-15T22:49:20.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 7, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.461518833, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.153324919, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.198388282, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.129218423, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4428, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.407568716, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000092043 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1905, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000366269, 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000192 2024-09-15T22:49:20.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1983008, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 265600 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4560, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2.070503567, 2024-09-15T22:49:20.570 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000454057 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 5255, 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.227462873, 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000043285 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 2608, 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 2608, 2024-09-15T22:49:20.571 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2608, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2.632439044, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001009370 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 12, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 33, 2024-09-15T22:49:20.572 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 2, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 31, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 30, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 3, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 3, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 2, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 2686, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 2666, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:49:20.573 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 93, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 238, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 4, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 234, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 6, 2024-09-15T22:49:20.574 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 238, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 152, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 21, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 131, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 86, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 1, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 26, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 3643, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:49:20.575 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 2447, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 1214, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 14, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 1243 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 3, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 2, 2024-09-15T22:49:20.576 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 1212, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 1210, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 1, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 12, 2024-09-15T22:49:20.577 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 21, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 6184, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 6184, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-09-15T22:49:20.578 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 9, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 9, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 20256019, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2501, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 3.915873202, 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001565722 2024-09-15T22:49:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 26, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 1257, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 1231, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 20, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 24, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 4, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 33, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 1248, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 1215, 2024-09-15T22:49:20.580 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 3, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 1228, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 1225, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 58848, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 231712 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 2671, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 1, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 2608, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 36, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 2, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1210, 2024-09-15T22:49:20.581 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 1.048381313, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000866430 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 120, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.021697457, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000180812 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-09-15T22:49:20.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 7, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001697168, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000242452 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000267150, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000267150 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 11, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001516135, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000137830 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 12, 2024-09-15T22:49:20.585 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.016416451, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001368037 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 11, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.057886629, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005262420 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.586 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000265924, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000265924 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.007171373, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001195228 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.002122839, 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001061419 2024-09-15T22:49:20.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 6, 2024-09-15T22:49:20.588 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.019274229, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.003212371 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 5, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.023019225, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.004603845 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-09-15T22:49:20.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001704842, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000852421 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 8, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.020641096, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.002580137 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1206, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 1.410377213, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001169467 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 6, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 4, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 2, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 33, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 16, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 636, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:20.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:20.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:20.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 336245, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 122, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 352, 2024-09-15T22:49:20.596 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 4, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 9512, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 192, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 130194, 2024-09-15T22:49:20.597 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 1028, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 8167, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 16314979, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 37, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 8167, 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-09-15T22:49:20.598 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 8167, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 42.752427539, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005234777 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 8167, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 8248 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 8167, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 310, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 7857, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-09-15T22:49:20.599 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 11, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 51, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 2531, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 38, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 5217, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-09-15T22:49:20.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 259, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 22, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-09-15T22:49:20.601 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 100, 2024-09-15T22:49:20.602 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 21, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 12, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 12, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 99, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 0, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 20 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:49:20.603 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 13 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 19, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 8, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 4702, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 1210, 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-09-15T22:49:20.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 16441, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 5410598, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 16441, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 16441, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 5410598, 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:20.605 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 8167, 2024-09-15T22:49:20.606 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 16446845, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 8167, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 16446845, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:20.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 8167, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 8167, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 8167, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 8167, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:49:20.608 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 1210, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 131866, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 1210, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 30, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 131866, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.609 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x556f65730960": { 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 6184, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 16061715, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 6184, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2501, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 16061715, 2024-09-15T22:49:20.610 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:20.611 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:20.611 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:20.611 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:20.611 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.611 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:20.611 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-15T22:49:20.684 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:20 smithi059 ceph-mon[34872]: pgmap v340: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:20.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:20 smithi130 ceph-mon[35742]: pgmap v340: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:20.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:20 smithi016 ceph-mon[31492]: pgmap v340: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-09-15T22:49:21.447 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-15T22:49:21.447 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-15T22:49:21.447 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-15T22:49:21.447 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-15T22:49:21.447 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-15T22:49:21.447 DEBUG:tasks.check_counter:Getting stats from e 2024-09-15T22:49:21.448 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-15T22:49:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1010, 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 997, 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1877696, 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 379624, 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:22.275 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.131711137, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.051533684, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.065178803, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.022103259, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 996, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.058179424, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000058413 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1623680, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 139216 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 8, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 174, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 3868, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 450778, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.012681985, 2024-09-15T22:49:22.277 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.006823196, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.003890974, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000070567, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 172, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.016846229, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000097943 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-15T22:49:22.278 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 450080 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1434, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1436, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 367328, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 456019, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.090197030, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.021115485, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.025892918, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.037372292, 2024-09-15T22:49:22.279 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1434, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.039061299, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000027239 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.280 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3699, 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.093995170, 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000025410 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-09-15T22:49:22.281 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-09-15T22:49:22.283 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-09-15T22:49:22.284 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:49:22.285 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-09-15T22:49:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-09-15T22:49:22.287 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-09-15T22:49:22.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 6, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 2, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-09-15T22:49:22.289 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 4196127, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 4195734, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 6 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 3, 2024-09-15T22:49:22.290 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 3, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 1, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 1, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 48956, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 215328 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.291 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 0, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.292 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-09-15T22:49:22.293 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.294 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.296 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-09-15T22:49:22.297 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.299 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-09-15T22:49:22.300 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.301 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.302 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.303 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 3, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 1, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 2, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 1917, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-09-15T22:49:22.304 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:22.305 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:22.306 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 181330, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 48, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-09-15T22:49:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 9512, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 192, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 8512, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 13, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:22.308 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 2192, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 21, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 3, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 2192, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2192, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 1.205983748, 2024-09-15T22:49:22.309 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000550175 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2192, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2193 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 2192, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 2192, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 0, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 1426, 2024-09-15T22:49:22.310 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 764, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 1, 2024-09-15T22:49:22.311 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-09-15T22:49:22.312 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 100, 2024-09-15T22:49:22.313 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 15, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 4, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 4, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 0, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 2, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 0 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 0, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-09-15T22:49:22.314 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.315 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 2448, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 2012528, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 2448, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:22.316 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2448, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2012528, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:22.317 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2192, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 1828, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2192, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1828, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-09-15T22:49:22.318 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2192, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 2192, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2192, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2192, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.319 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:22.320 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x55fd135beb40": { 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:22.321 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:22.322 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:22.323 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-15T22:49:22.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:22 smithi059 ceph-mon[34872]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:49:22.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:22 smithi130 ceph-mon[35742]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:49:22.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:22 smithi016 ceph-mon[31492]: pgmap v341: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:49:23.009 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-15T22:49:23.009 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-15T22:49:23.009 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-15T22:49:23.009 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:49:23.009 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:49:23.009 DEBUG:tasks.check_counter:Getting stats from h 2024-09-15T22:49:23.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-15T22:49:23.264 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:49:23.846 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-15T22:49:23.846 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:23.846 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 47, 2024-09-15T22:49:23.846 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 216, 2024-09-15T22:49:23.846 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 11357, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 399488, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 16, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.023408644, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.007248507, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.009254943, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000577699, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 211, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.062131371, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000294461 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.847 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:23.848 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 72, 2024-09-15T22:49:23.848 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000014258, 2024-09-15T22:49:23.848 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000198 2024-09-15T22:49:23.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.848 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 4784, 2024-09-15T22:49:23.848 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 338160 2024-09-15T22:49:23.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.849 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:23.849 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 477, 2024-09-15T22:49:23.849 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 683, 2024-09-15T22:49:23.849 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 121210, 2024-09-15T22:49:23.849 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 958181, 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 51, 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 9, 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.091711470, 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.036088754, 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.048917135, 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.004145989, 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:23.850 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 682, 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.046939883, 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000068826 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1051, 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000270959, 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000257 2024-09-15T22:49:23.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.852 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-15T22:49:23.852 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 124896 2024-09-15T22:49:23.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.852 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:23.852 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 344, 2024-09-15T22:49:23.852 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 325, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1657844, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 195420, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 8, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.052954397, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.018728928, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.050772539, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000259901, 2024-09-15T22:49:23.853 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 326, 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.028293651, 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000086790 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 96, 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000029994, 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000312 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.854 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1637200, 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 143536 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-09-15T22:49:23.855 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 56, 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.001512240, 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000027004 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2, 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000016012, 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000008006 2024-09-15T22:49:23.856 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 6, 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 4, 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4, 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000767194, 2024-09-15T22:49:23.857 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000191798 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 2, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 11, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 10, 2024-09-15T22:49:23.858 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 10, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 10, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 1, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 2, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 9, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 8, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:49:23.859 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 4, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 234, 2024-09-15T22:49:23.860 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 6, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 2, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:49:23.861 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 2, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 1, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-09-15T22:49:23.862 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 1, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 1, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:49:23.863 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 6, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 6, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-09-15T22:49:23.864 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 2, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 2, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 4196127, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 0, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 5, 2024-09-15T22:49:23.865 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.008520441, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001704088 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 0 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 13, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 13, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 0, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 11, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 12, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 1, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 10, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 10, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 0, 2024-09-15T22:49:23.866 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 2, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 2, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 49876, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 223520 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 6, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 6, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 22, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.867 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000648174, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000216058 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-09-15T22:49:23.868 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.869 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000119020, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000119020 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-09-15T22:49:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-09-15T22:49:23.871 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.872 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.873 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-09-15T22:49:23.874 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.875 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.876 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:49:23.879 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 3, 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 1, 2024-09-15T22:49:23.880 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 2, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 3, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 1, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 627, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:23.881 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:23.882 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:23.883 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 247795, 2024-09-15T22:49:23.884 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 93, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 88, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 1, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 9512, 2024-09-15T22:49:23.885 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 192, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 51608, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 202, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.886 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 41, 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 12125, 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 30, 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 41, 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 41, 2024-09-15T22:49:23.887 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.236996825, 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.005780410 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 41, 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 63 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.888 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 41, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 41, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 10, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 5, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 7, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-09-15T22:49:23.889 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 6, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 20, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:49:23.890 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-09-15T22:49:23.891 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 100, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 15, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 9, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 9, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 14, 2024-09-15T22:49:23.892 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 0, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 1 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 1, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 2 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-09-15T22:49:23.893 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 864, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1694345, 2024-09-15T22:49:23.894 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 864, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 864, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1694345, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-09-15T22:49:23.895 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 41, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 12125, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 41, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 12125, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.896 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 41, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 41, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 41, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 41, 2024-09-15T22:49:23.897 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:23.898 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x5648950c4960": { 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-09-15T22:49:23.899 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 6, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1823, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 6, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 5, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1823, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.900 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:23.901 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-15T22:49:24.685 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:24 smithi059 ceph-mon[34872]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:49:24.712 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-15T22:49:24.712 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-15T22:49:24.712 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-15T22:49:24.712 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:49:24.712 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:49:24.712 DEBUG:tasks.check_counter:Getting stats from k 2024-09-15T22:49:24.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-15T22:49:24.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:24 smithi130 ceph-mon[35742]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:49:24.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:24 smithi016 ceph-mon[31492]: pgmap v342: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-15T22:49:24.963 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:49:25.655 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-09-15T22:49:25.655 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:25.655 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 4, 2024-09-15T22:49:25.655 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 4, 2024-09-15T22:49:25.655 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2992, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 448, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.003000503, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000040332, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002192023, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.003992858, 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001330952 2024-09-15T22:49:25.656 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 236, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 225, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1590400, 2024-09-15T22:49:25.657 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 138304, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.045409597, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.016199337, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.043197012, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 224, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.013524667, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000060377 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:25.658 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1590400, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 138304 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 2, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 172, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 320, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 187408, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:25.659 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.012004598, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.007894463, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.001623082, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 169, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.017032296, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000100782 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:25.660 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 187408 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:25.661 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:25.662 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:25.663 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 148977, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 30, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-09-15T22:49:25.664 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-09-15T22:49:25.665 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:25.666 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-09-15T22:49:25.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-09-15T22:49:25.668 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 238, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1557082, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 238, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 238, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1557082, 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-09-15T22:49:25.669 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-09-15T22:49:25.670 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-09-15T22:49:25.670 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-09-15T22:49:25.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:25.670 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-09-15T22:49:25.670 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-09-15T22:49:26.371 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-15T22:49:26.371 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-15T22:49:26.371 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-15T22:49:26.371 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-15T22:49:26.371 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-15T22:49:26.371 DEBUG:tasks.check_counter:Getting stats from c 2024-09-15T22:49:26.371 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-15T22:49:26.629 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:49:26.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:26 smithi059 ceph-mon[34872]: pgmap v343: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:49:26.750 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:26 smithi130 ceph-mon[35742]: pgmap v343: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:49:26.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:26 smithi016 ceph-mon[31492]: pgmap v343: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 94 B/s wr, 5 op/s 2024-09-15T22:49:27.177 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-15T22:49:27.177 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:27.177 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1508, 2024-09-15T22:49:27.177 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1510, 2024-09-15T22:49:27.177 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 4637669, 2024-09-15T22:49:27.177 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 479554, 2024-09-15T22:49:27.177 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 6, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.162617260, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.047241950, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.043907215, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.056568260, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1508, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.060999723, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000040450 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.178 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 851, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 836, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 5260707, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 328146, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:27.179 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.108394779, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.035338837, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.064011077, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.018059466, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 836, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.046019288, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000055046 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.180 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1626832, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 139824 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 276, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 444, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 8457347, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 538381, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 7, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 5, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.037196674, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.015646053, 2024-09-15T22:49:27.181 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.009730800, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.007674745, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 441, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.023807406, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000053985 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:49:27.182 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 451248 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3924, 2024-09-15T22:49:27.183 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.118147983, 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000030109 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-09-15T22:49:27.184 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-09-15T22:49:27.185 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 1238, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 1238, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 1238, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 2, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-09-15T22:49:27.186 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 4, 2024-09-15T22:49:27.187 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 234, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 6, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:49:27.188 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-09-15T22:49:27.189 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:49:27.190 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 6184, 2024-09-15T22:49:27.191 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 9, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 20256019, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 20255883, 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-09-15T22:49:27.192 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 6184 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 24, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 1234, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 1210, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 18, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 22, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 4, 2024-09-15T22:49:27.193 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 1238, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 1243, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 5, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 50556, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215328 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-09-15T22:49:27.194 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.195 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.196 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.197 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.198 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.199 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.199 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-09-15T22:49:27.199 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.199 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.199 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-09-15T22:49:27.200 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.201 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.202 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.203 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.204 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.205 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.206 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 6, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 4, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 4, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 3170, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.207 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:27.208 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:27.209 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 198425, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 62, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-09-15T22:49:27.210 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9512, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 192, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 967090, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 5524, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:27.211 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2374, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 26, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 4, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2374, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2374, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.280975351, 2024-09-15T22:49:27.212 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000539585 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2374, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2375 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2374, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2374, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1525, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-09-15T22:49:27.213 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 846, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 2, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-09-15T22:49:27.214 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-09-15T22:49:27.215 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-09-15T22:49:27.216 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 100, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 15, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 6, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 6, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:49:27.217 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-09-15T22:49:27.218 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2631, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 18102843, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2631, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2631, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 18102843, 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:27.219 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:27.220 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2374, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 16061725, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2374, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 16061725, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:27.221 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2374, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2374, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2374, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2374, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.222 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.223 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x55d067b08b40": { 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:27.224 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:27.225 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-15T22:49:28.121 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-15T22:49:28.121 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-15T22:49:28.121 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-15T22:49:28.121 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:49:28.121 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:49:28.121 DEBUG:tasks.check_counter:Getting stats from f 2024-09-15T22:49:28.121 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-15T22:49:28.373 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:49:28.683 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:28 smithi130 ceph-mon[35742]: pgmap v344: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:49:28.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:28 smithi059 ceph-mon[34872]: pgmap v344: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:49:28.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:28 smithi016 ceph-mon[31492]: pgmap v344: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T22:49:28.908 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-15T22:49:28.908 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:28.908 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 256, 2024-09-15T22:49:28.908 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 243, 2024-09-15T22:49:28.908 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1624786, 2024-09-15T22:49:28.908 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 140180, 2024-09-15T22:49:28.908 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.046372385, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.016409937, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.043970242, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000028997, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 242, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.015618284, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000064538 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.909 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1623952, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 139856 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 669, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 837, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 164424, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 663359, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 7, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 5, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.083803567, 2024-09-15T22:49:28.910 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.025167069, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.022858346, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.027546956, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 834, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.042886760, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000051422 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.911 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 451248 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 1537, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 1537, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 463444, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 488000, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.157409968, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.049972473, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.043414700, 2024-09-15T22:49:28.912 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.050067441, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1536, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.066789561, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000043482 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.913 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3616, 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.111404982, 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000030808 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-09-15T22:49:28.914 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 0, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 0, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.915 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-09-15T22:49:28.916 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 1, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 0, 2024-09-15T22:49:28.917 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:49:28.918 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 0 2024-09-15T22:49:28.919 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-09-15T22:49:28.920 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-09-15T22:49:28.921 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 8, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-09-15T22:49:28.922 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 4196661, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 4196520, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 8 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-09-15T22:49:28.923 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 3, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 3, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 1, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 1, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 0, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 48620, 2024-09-15T22:49:28.924 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 215328 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 0, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 0, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 0, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 0, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-09-15T22:49:28.925 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.926 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-09-15T22:49:28.927 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.930 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-09-15T22:49:28.931 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-09-15T22:49:28.932 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.933 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.934 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.935 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.936 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.937 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-09-15T22:49:28.938 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.939 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 4, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 2, 2024-09-15T22:49:28.940 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 4, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 3172, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:28.941 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:28.942 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-09-15T22:49:28.943 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 189400, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 51, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:28.944 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9512, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 192, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 8512, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 13, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.945 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 2201, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 21, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 3, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 2201, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2201, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 1.214976639, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000552011 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-09-15T22:49:28.946 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2201, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 2202 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 2201, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 2201, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 0, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 1428, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 0, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 771, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 0, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 0, 2024-09-15T22:49:28.947 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 1, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:49:28.948 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-09-15T22:49:28.949 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 100, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 15, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 5, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 5, 2024-09-15T22:49:28.950 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 0, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 2, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 0 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 0, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-09-15T22:49:28.951 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 0 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-09-15T22:49:28.952 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 2458, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 2015338, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 2458, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2458, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2015338, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.953 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2201, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2362, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2201, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2362, 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:28.954 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:28.955 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 2201, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 2201, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 2201, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2201, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:28.956 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:28.957 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x556d965ecb40": { 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:28.958 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:28.959 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:28.959 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.959 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:28.959 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-15T22:49:29.743 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-15T22:49:29.743 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-15T22:49:29.743 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-15T22:49:29.743 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:49:29.743 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:49:29.743 DEBUG:tasks.check_counter:Getting stats from i 2024-09-15T22:49:29.743 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-15T22:49:29.989 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:49:30.500 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-15T22:49:30.500 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 333, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 368, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1646117, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 243918, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 16, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 2, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.065578602, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.021155063, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.054251914, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000373379, 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:30.501 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 368, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.029891438, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000081226 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 92, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000017671, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000192 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1637936, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 149856 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:30.502 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 79, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 115, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 16483, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 657343, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 51, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.026090986, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.003338277, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.020467923, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000431904, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 112, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.015516480, 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000138540 2024-09-15T22:49:30.503 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 78, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000007142, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000091 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 5456, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 131440 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 524, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 777, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 147149, 2024-09-15T22:49:30.504 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 692379, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 8, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 6, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.067086159, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.029076883, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.032238821, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.004871338, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 775, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.051189196, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000066050 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:30.505 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 1126, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000324346, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000288 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 326624 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 1, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-MDSRank": { 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-09-15T22:49:30.506 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 51, 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.002089830, 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000040977 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "finisher-PurgeQueue": { 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "queue_len": 0, 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "complete_latency": { 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2, 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000012624, 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000006312 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.507 INFO:teuthology.orchestra.run.smithi130.stdout: "mds": { 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "request": 8, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "reply": 5, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "reply_latency": { 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 5, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.004766674, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000953334 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_reply": 0, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "forward": 3, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_complete": 0, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_fetch_keys": 0, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_commit": 11, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_split": 0, 2024-09-15T22:49:30.508 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_merge": 0, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes": 14, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_top": 0, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_bottom": 14, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pin_tail": 0, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_pinned": 14, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_expired": 0, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "inodes_with_caps": 1, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "caps": 1, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "subtrees": 2, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse": 14, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_hit": 10, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_forward": 2, 2024-09-15T22:49:30.509 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_discover": 1, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_remote_ino": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "traverse_lock": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "load_cent": 1000, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "q": 1, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "exported": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "exported_inodes": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "imported": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "imported_inodes": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_dir_fetch": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "openino_peer_discover": 0, 2024-09-15T22:49:30.510 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rfiles": 4, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rbytes": 234, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "root_rsnaps": 6, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_set_tag": 0, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:49:30.511 INFO:teuthology.orchestra.run.smithi130.stdout: "scrub_file_inodes": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_revoke": 7, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_grant": 4, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps": 7, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_cap_release": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "process_request_cap_release": 1 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_cache": { 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update": 0, 2024-09-15T22:49:30.512 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_update_receipt": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_try_discover": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_send_discover": 2, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "dir_handle_discover": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_delayed": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_created": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_enqueued": 0, 2024-09-15T22:49:30.513 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_reintegrated": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "strays_migrated": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_processing": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_started": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "recovery_completed": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_path": 1, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_quiesce_inode": 5, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_exportdir": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_flush": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_fragstats": 0, 2024-09-15T22:49:30.514 INFO:teuthology.orchestra.run.smithi130.stdout: "ireq_inodestats": 0 2024-09-15T22:49:30.515 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.515 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_log": { 2024-09-15T22:49:30.515 INFO:teuthology.orchestra.run.smithi130.stdout: "evlrg": 0, 2024-09-15T22:49:30.515 INFO:teuthology.orchestra.run.smithi130.stdout: "evadd": 8, 2024-09-15T22:49:30.515 INFO:teuthology.orchestra.run.smithi130.stdout: "evex": 0, 2024-09-15T22:49:30.515 INFO:teuthology.orchestra.run.smithi130.stdout: "evtrm": 0, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "ev": 8, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "evexg": 0, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "evexd": 0, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "segadd": 2, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "segex": 0, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "segtrm": 0, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "seg": 2, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "segmjr": 2, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "segexg": 0, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "segexd": 0, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "expos": 4194304, 2024-09-15T22:49:30.516 INFO:teuthology.orchestra.run.smithi130.stdout: "wrpos": 4196661, 2024-09-15T22:49:30.517 INFO:teuthology.orchestra.run.smithi130.stdout: "rdpos": 0, 2024-09-15T22:49:30.517 INFO:teuthology.orchestra.run.smithi130.stdout: "jlat": { 2024-09-15T22:49:30.517 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 7, 2024-09-15T22:49:30.517 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.012653478, 2024-09-15T22:49:30.517 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001807639 2024-09-15T22:49:30.517 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "replayed": 0 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_mem": { 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "ino": 17, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "ino+": 17, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "ino-": 0, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "dir": 15, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "dir+": 16, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "dir-": 1, 2024-09-15T22:49:30.518 INFO:teuthology.orchestra.run.smithi130.stdout: "dn": 14, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "dn+": 14, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "dn-": 0, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "cap": 1, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "cap+": 5, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "cap-": 4, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "rss": 50300, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "heap": 223520 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_server": { 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_client_request": 12, 2024-09-15T22:49:30.519 INFO:teuthology.orchestra.run.smithi130.stdout: "dispatch_server_request": 0, 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_request": 8, 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_client_session": 26, 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: "handle_peer_request": 1, 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: "req_create_latency": { 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.520 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getattr_latency": { 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 3, 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.003603083, 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.001201027 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getfilelock_latency": { 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "req_link_latency": { 2024-09-15T22:49:30.521 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookup_latency": { 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 2, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.001163591, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000581795 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookuphash_latency": { 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.522 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupino_latency": { 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupname_latency": { 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.523 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupparent_latency": { 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lookupsnap_latency": { 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "req_lssnap_latency": { 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.524 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mkdir_latency": { 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mknod_latency": { 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "req_mksnap_latency": { 2024-09-15T22:49:30.525 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "req_open_latency": { 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "req_readdir_latency": { 2024-09-15T22:49:30.526 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rename_latency": { 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "req_renamesnap_latency": { 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.527 INFO:teuthology.orchestra.run.smithi130.stdout: "req_snapdiff_latency": { 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmdir_latency": { 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.528 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmsnap_latency": { 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "req_rmxattr_latency": { 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setattr_latency": { 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.529 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setdirlayout_latency": { 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setfilelock_latency": { 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setlayout_latency": { 2024-09-15T22:49:30.530 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "req_setxattr_latency": { 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "req_symlink_latency": { 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.531 INFO:teuthology.orchestra.run.smithi130.stdout: "req_unlink_latency": { 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "req_getvxattr_latency": { 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.532 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_sessions": { 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "session_count": 2, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "session_add": 4, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "session_remove": 2, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_open": 2, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "sessions_stale": 0, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "total_load": 4, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "average_load": 1, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "avg_session_uptime": 647, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "mdthresh_evicted": 0 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:30.533 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:30.534 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:30.535 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:30.536 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 272095, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 97, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 9512, 2024-09-15T22:49:30.537 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 192, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 68101, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 296, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-09-15T22:49:30.538 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "objecter": { 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "op_active": 0, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "op_laggy": 0, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send": 48, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "op_send_bytes": 13338, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "op_resend": 30, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "op_reply": 48, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "op_latency": { 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 48, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.230995609, 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.004812408 2024-09-15T22:49:30.539 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "op_inflight": 0, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "oplen_avg": { 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 48, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 74 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "op": 48, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "op_r": 0, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "op_w": 48, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "op_rmw": 0, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "op_pg": 0, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_stat": 0, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_create": 10, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_read": 0, 2024-09-15T22:49:30.540 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_write": 7, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writefull": 8, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_writesame": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_append": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_zero": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_truncate": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_delete": 6, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_mapext": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_sparse_read": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_clonerange": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_getxattr": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_setxattr": 20, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_rmxattr": 0, 2024-09-15T22:49:30.541 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_call": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_watch": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_notify": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "osdop_other": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_active": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_send": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_resend": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "linger_ping": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_active": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_send": 0, 2024-09-15T22:49:30.542 INFO:teuthology.orchestra.run.smithi130.stdout: "poolop_resend": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_active": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_send": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "poolstat_resend": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_active": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_send": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "statfs_resend": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "command_active": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "command_send": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "command_resend": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "map_epoch": 100, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "map_full": 0, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "map_inc": 15, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_sessions": 8, 2024-09-15T22:49:30.543 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_open": 8, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_session_close": 0, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_laggy": 0, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_wr": 22, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_rd": 0, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_del": 1 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "oft": { 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_objs": 1, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_updates": 0, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "omap_total_removes": 0 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "osd-slow-ops": { 2024-09-15T22:49:30.544 INFO:teuthology.orchestra.run.smithi130.stdout: "slow_ops_count": 6 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "purge_queue": { 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops": 0, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing": 0, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executing_high_water": 0, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed_ops": 0, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_executed": 0, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "pq_item_in_journal": 0 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-09-15T22:49:30.545 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 932, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 1707304, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 932, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 932, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 1707304, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.546 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_bytes": { 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 48, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 13338, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 48, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 13338, 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:30.547 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-objecter_ops": { 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 1024, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 48, 2024-09-15T22:49:30.548 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 48, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 48, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 48, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 0, 2024-09-15T22:49:30.549 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-write_buf_throttle-0x5619e7226960": { 2024-09-15T22:49:30.550 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 3758096384, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 8, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 2357, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 8, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 7, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 2357, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:30.551 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:30.552 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.552 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:30.552 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-15T22:49:30.552 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:30 smithi130 ceph-mon[35742]: pgmap v345: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:30.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:30 smithi059 ceph-mon[34872]: pgmap v345: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:30.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:30 smithi016 ceph-mon[31492]: pgmap v345: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:31.290 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-15T22:49:31.291 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-15T22:49:31.291 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-15T22:49:31.291 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:49:31.291 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:49:31.291 DEBUG:tasks.check_counter:Getting stats from l 2024-09-15T22:49:31.291 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-15T22:49:31.544 INFO:teuthology.orchestra.run.smithi130.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:49:32.228 INFO:teuthology.orchestra.run.smithi130.stdout:{ 2024-09-15T22:49:32.228 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:49:32.228 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 0, 2024-09-15T22:49:32.228 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 2, 2024-09-15T22:49:32.228 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:49:32.228 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 0, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 4, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 0, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.000981213, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.000212692, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:32.229 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 6, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 176, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 3312, 2024-09-15T22:49:32.230 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 188208, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 5, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 3, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.015200926, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.009259266, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.003334602, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 174, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.022288676, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000128095 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:32.231 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 188208 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_messages": 238, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages": 227, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_bytes": 1590944, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_bytes": 139584, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_created_connections": 2, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_active_connections": 1, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_total_time": 0.046470153, 2024-09-15T22:49:32.232 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_send_time": 0.016167032, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_recv_time": 0.045546419, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 226, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.015218061, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000067336 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_recv_encrypted_bytes": 1590944, 2024-09-15T22:49:32.233 INFO:teuthology.orchestra.run.smithi130.stdout: "msgr_send_encrypted_bytes": 139584 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: "cct": { 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: "total_workers": 0, 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: "unhealthy_workers": 0 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: "mempool": { 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: "bloom_filter_items": 0, 2024-09-15T22:49:32.234 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_extent_items": 0, 2024-09-15T22:49:32.235 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_blob_items": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_txc_items": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:49:32.236 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "bluestore_writing_items": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_bytes": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_items": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_bytes": 148967, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_anon_items": 29, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "buffer_meta_items": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_bytes": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_items": 0, 2024-09-15T22:49:32.237 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_mapbl_items": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "osd_pglog_items": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_bytes": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_items": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_bytes": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "pgmap_items": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_bytes": 928, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "mds_co_items": 2, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_bytes": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_1_items": 0, 2024-09-15T22:49:32.238 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_bytes": 0, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "unittest_2_items": 0 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: }, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "val": 0, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "max": 104857600, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "get_started": 0, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "get": 240, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "get_sum": 1557342, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_fail": 0, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "get_or_fail_success": 240, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "take": 0, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "take_sum": 0, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "put": 240, 2024-09-15T22:49:32.239 INFO:teuthology.orchestra.run.smithi130.stdout: "put_sum": 1557342, 2024-09-15T22:49:32.240 INFO:teuthology.orchestra.run.smithi130.stdout: "wait": { 2024-09-15T22:49:32.240 INFO:teuthology.orchestra.run.smithi130.stdout: "avgcount": 0, 2024-09-15T22:49:32.240 INFO:teuthology.orchestra.run.smithi130.stdout: "sum": 0.000000000, 2024-09-15T22:49:32.240 INFO:teuthology.orchestra.run.smithi130.stdout: "avgtime": 0.000000000 2024-09-15T22:49:32.240 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:32.240 INFO:teuthology.orchestra.run.smithi130.stdout: } 2024-09-15T22:49:32.240 INFO:teuthology.orchestra.run.smithi130.stdout:} 2024-09-15T22:49:32.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:32 smithi130 ceph-mon[35742]: pgmap v346: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:49:32.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:32 smithi059 ceph-mon[34872]: pgmap v346: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:49:32.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:32 smithi016 ceph-mon[31492]: pgmap v346: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-15T22:49:32.971 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-15T22:49:32.972 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-15T22:49:32.972 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-15T22:49:32.972 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-15T22:49:32.972 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-15T22:49:32.972 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-15T22:49:32.984 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-15T22:49:32.984 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:49:32.984 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:49:33.007 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-15T22:49:33.007 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:49:33.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:49:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 -- 172.21.15.16:0/4227381170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 msgr2=0x7f0c1c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 --2- 172.21.15.16:0/4227381170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c10bb50 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009900 tx=0x7f0c0c02a480 comp rx=0 tx=0).stop 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 -- 172.21.15.16:0/4227381170 shutdown_connections 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 --2- 172.21.15.16:0/4227381170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1c10c170 0x7f0c1c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 --2- 172.21.15.16:0/4227381170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c10bb50 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 --2- 172.21.15.16:0/4227381170 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c1c102d40 0x7f0c1c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 -- 172.21.15.16:0/4227381170 >> 172.21.15.16:0/4227381170 conn(0x7f0c1c0fc7a0 msgr2=0x7f0c1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 -- 172.21.15.16:0/4227381170 shutdown_connections 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.858+0000 7f0c23c05640 1 -- 172.21.15.16:0/4227381170 wait complete. 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 Processor -- start 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 -- start start 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1c102d40 0x7f0c1c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c1c10c170 0x7f0c1c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c111fd0 con 0x7f0c1c103660 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c111e50 con 0x7f0c1c102d40 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c23c05640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c112150 con 0x7f0c1c10c170 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52618/0 (socket says 172.21.15.16:52618) 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 -- 172.21.15.16:0/3098847489 learned_addr learned my addr 172.21.15.16:0/3098847489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c219a1640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1c102d40 0x7f0c1c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c221a2640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c1c10c170 0x7f0c1c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 -- 172.21.15.16:0/3098847489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c1c10c170 msgr2=0x7f0c1c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c1c10c170 0x7f0c1c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 -- 172.21.15.16:0/3098847489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1c102d40 msgr2=0x7f0c1c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1c102d40 0x7f0c1c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.859+0000 7f0c211a0640 1 -- 172.21.15.16:0/3098847489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c19d460 con 0x7f0c1c103660 2024-09-15T22:49:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.860+0000 7f0c211a0640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c1989f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c006010 tx=0x7f0c0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.860+0000 7f0c12ffd640 1 -- 172.21.15.16:0/3098847489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c03f030 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.860+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c19d6f0 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.860+0000 7f0c12ffd640 1 -- 172.21.15.16:0/3098847489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c0c003ce0 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.860+0000 7f0c12ffd640 1 -- 172.21.15.16:0/3098847489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c035070 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.860+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c1a5030 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.861+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c103120 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.862+0000 7f0c12ffd640 1 -- 172.21.15.16:0/3098847489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0c0c042030 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.863+0000 7f0c12ffd640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0c080a33b0 0x7f0c080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.863+0000 7f0c219a1640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0c080a33b0 0x7f0c080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.863+0000 7f0c12ffd640 1 -- 172.21.15.16:0/3098847489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9467+0+0 (secure 0 0 0) 0x7f0c0c0f0970 con 0x7f0c1c103660 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.863+0000 7f0c219a1640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0c080a33b0 0x7f0c080a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005fd0 tx=0x7f0c1800b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:33.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:33.866+0000 7f0c12ffd640 1 -- 172.21.15.16:0/3098847489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c0c0bc9b0 con 0x7f0c1c103660 2024-09-15T22:49:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.054+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0c1c199810 con 0x7f0c1c103660 2024-09-15T22:49:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.055+0000 7f0c12ffd640 1 -- 172.21.15.16:0/3098847489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+942 (secure 0 0 0) 0x7f0c1c199810 con 0x7f0c1c103660 2024-09-15T22:49:34.058 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3949807891 2024-09-16T22:28:46.507397+0000 2024-09-15T22:49:34.058 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3275020767 2024-09-16T22:38:25.000235+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3949807891 2024-09-16T22:28:46.507397+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/81475521 2024-09-16T22:38:25.000235+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1563124617 2024-09-16T22:28:46.507397+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2652317700 2024-09-16T22:29:10.483605+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3860212428 2024-09-16T22:28:46.507397+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2652317700 2024-09-16T22:29:10.483605+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/81475521 2024-09-16T22:38:25.000235+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3381483588 2024-09-16T22:29:10.483605+0000 2024-09-15T22:49:34.059 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3905659611 2024-09-16T22:29:10.483605+0000 2024-09-15T22:49:34.060 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1740394357 2024-09-16T22:29:10.483605+0000 2024-09-15T22:49:34.060 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3377264779 2024-09-16T22:38:25.000235+0000 2024-09-15T22:49:34.060 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3984482177 2024-09-16T22:38:25.000235+0000 2024-09-15T22:49:34.060 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3985544937 2024-09-16T22:28:46.507397+0000 2024-09-15T22:49:34.060 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1746300554 2024-09-16T22:38:25.000235+0000 2024-09-15T22:49:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:listed 16 entries 2024-09-15T22:49:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0c080a33b0 msgr2=0x7f0c080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0c080a33b0 0x7f0c080a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005fd0 tx=0x7f0c1800b390 comp rx=0 tx=0).stop 2024-09-15T22:49:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 msgr2=0x7f0c1c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c1989f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c006010 tx=0x7f0c0c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 shutdown_connections 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f0c080a33b0 0x7f0c080a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f0c1c10c170 0x7f0c1c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c103660 0x7f0c1c1989f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 --2- 172.21.15.16:0/3098847489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c1c102d40 0x7f0c1c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 >> 172.21.15.16:0/3098847489 conn(0x7f0c1c0fc7a0 msgr2=0x7f0c1c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 shutdown_connections 2024-09-15T22:49:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:34.057+0000 7f0c23c05640 1 -- 172.21.15.16:0/3098847489 wait complete. 2024-09-15T22:49:34.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:34 smithi059 ceph-mon[34872]: pgmap v347: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:34.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:34 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3098847489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:49:34.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:34 smithi130 ceph-mon[35742]: pgmap v347: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:34.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:34 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3098847489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:49:34.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:34 smithi016 ceph-mon[31492]: pgmap v347: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:34.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:34 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3098847489' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:49:34.864 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-15T22:49:34.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:49:34.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-15T22:49:34.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:49:34.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:49:34.936 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-15T22:49:34.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:49:34.954 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-15T22:49:35.011 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-15T22:49:35.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:49:35.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T22:49:35.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:49:35.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-15T22:49:35.182 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-15T22:49:36.048 INFO:tasks.cephadm:Teardown begin 2024-09-15T22:49:36.049 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T22:49:36.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:49:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:36.693 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:36 smithi059 ceph-mon[34872]: pgmap v348: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:36.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:36 smithi016 ceph-mon[31492]: pgmap v348: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:36.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:36 smithi130 ceph-mon[35742]: pgmap v348: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:49:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 -- 172.21.15.16:0/3362152084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 msgr2=0x7f1c0010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 --2- 172.21.15.16:0/3362152084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c0010d950 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009cd0 tx=0x7f1bf802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:49:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 -- 172.21.15.16:0/3362152084 shutdown_connections 2024-09-15T22:49:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 --2- 172.21.15.16:0/3362152084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c0010d950 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 --2- 172.21.15.16:0/3362152084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c00100a80 0x7f1c00100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 --2- 172.21.15.16:0/3362152084 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c00106a80 0x7f1c00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 -- 172.21.15.16:0/3362152084 >> 172.21.15.16:0/3362152084 conn(0x7f1c000fc7a0 msgr2=0x7f1c000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 -- 172.21.15.16:0/3362152084 shutdown_connections 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 -- 172.21.15.16:0/3362152084 wait complete. 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 Processor -- start 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.820+0000 7f1c05e35640 1 -- start start 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1c05e35640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c00100a80 0x7f1c00198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1c05e35640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c00198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1c05e35640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c00106a80 0x7f1c0019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1c05e35640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00110e80 con 0x7f1c00101420 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1c05e35640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00110d00 con 0x7f1c00106a80 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1c05e35640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00111000 con 0x7f1c00100a80 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c00198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:36.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c00198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52642/0 (socket says 172.21.15.16:52642) 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/832931338 learned_addr learned my addr 172.21.15.16:0/832931338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bff7fe640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c00100a80 0x7f1c00198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bfffff640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c00106a80 0x7f1c0019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/832931338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c00100a80 msgr2=0x7f1c00198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c00100a80 0x7f1c00198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/832931338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c00106a80 msgr2=0x7f1c0019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c00106a80 0x7f1c0019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 -- 172.21.15.16:0/832931338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0019d3e0 con 0x7f1c00101420 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bf6ffd640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c00198970 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1be800c920 tx=0x7f1be800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bfd7fa640 1 -- 172.21.15.16:0/832931338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8007c70 con 0x7f1c00101420 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.821+0000 7f1bfd7fa640 1 -- 172.21.15.16:0/832931338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be8007e10 con 0x7f1c00101420 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.822+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0019d6d0 con 0x7f1c00101420 2024-09-15T22:49:36.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.822+0000 7f1bfd7fa640 1 -- 172.21.15.16:0/832931338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8018890 con 0x7f1c00101420 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.822+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c001a4fb0 con 0x7f1c00101420 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.823+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c00100ee0 con 0x7f1c00101420 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.824+0000 7f1bfd7fa640 1 -- 172.21.15.16:0/832931338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1be8018a70 con 0x7f1c00101420 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.825+0000 7f1bfd7fa640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.825+0000 7f1bff7fe640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.825+0000 7f1bfd7fa640 1 -- 172.21.15.16:0/832931338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9467+0+0 (secure 0 0 0) 0x7f1be80d01c0 con 0x7f1c00101420 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.826+0000 7f1bff7fe640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1bec0a3430 0x7f1bec0a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005fd0 tx=0x7f1bf000b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:36.829+0000 7f1bfd7fa640 1 -- 172.21.15.16:0/832931338 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1be8014030 con 0x7f1c00101420 2024-09-15T22:49:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.039+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c0010a060 con 0x7f1c00101420 2024-09-15T22:49:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.040+0000 7f1bfd7fa640 1 -- 172.21.15.16:0/832931338 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7f1be809c7f0 con 0x7f1c00101420 2024-09-15T22:49:37.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:49:37.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:49:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:49:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1bec0a3430 msgr2=0x7f1bec0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1bec0a3430 0x7f1bec0a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0005fd0 tx=0x7f1bf000b390 comp rx=0 tx=0).stop 2024-09-15T22:49:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 msgr2=0x7f1c00198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c00198970 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1be800c920 tx=0x7f1be800cde0 comp rx=0 tx=0).stop 2024-09-15T22:49:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 shutdown_connections 2024-09-15T22:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c00106a80 0x7f1c0019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00101420 0x7f1c00198970 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f1bec0a3430 0x7f1bec0a58f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 --2- 172.21.15.16:0/832931338 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f1c00100a80 0x7f1c00198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 >> 172.21.15.16:0/832931338 conn(0x7f1c000fc7a0 msgr2=0x7f1c0006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 shutdown_connections 2024-09-15T22:49:37.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:37.043+0000 7f1c05e35640 1 -- 172.21.15.16:0/832931338 wait complete. 2024-09-15T22:49:37.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:37 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/832931338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:37.709 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:37 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/832931338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:37.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:49:37.766 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:37 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/832931338' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:38.011 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 -- 172.21.15.16:0/1306684695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 msgr2=0x7fa8981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/1306684695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981021c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa894009900 tx=0x7fa89402a410 comp rx=0 tx=0).stop 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 -- 172.21.15.16:0/1306684695 shutdown_connections 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/1306684695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa898106f10 0x7fa89810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/1306684695 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa898102790 0x7fa8981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/1306684695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981021c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 -- 172.21.15.16:0/1306684695 >> 172.21.15.16:0/1306684695 conn(0x7fa8980fd8c0 msgr2=0x7fa8980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.511+0000 7fa8a01ea640 1 -- 172.21.15.16:0/1306684695 shutdown_connections 2024-09-15T22:49:38.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 -- 172.21.15.16:0/1306684695 wait complete. 2024-09-15T22:49:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 Processor -- start 2024-09-15T22:49:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 -- start start 2024-09-15T22:49:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa898102790 0x7fa898198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa898106f10 0x7fa89819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:38.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa898111f70 con 0x7fa898101de0 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa898111df0 con 0x7fa898106f10 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.512+0000 7fa8a01ea640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7fa8981120f0 con 0x7fa898102790 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52652/0 (socket says 172.21.15.16:52652) 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 -- 172.21.15.16:0/943092763 learned_addr learned my addr 172.21.15.16:0/943092763 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89d785640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa898102790 0x7fa898198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89e787640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa898106f10 0x7fa89819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 -- 172.21.15.16:0/943092763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa898102790 msgr2=0x7fa898198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa898102790 0x7fa898198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 -- 172.21.15.16:0/943092763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa898106f10 msgr2=0x7fa89819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa898106f10 0x7fa89819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 -- 172.21.15.16:0/943092763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89819d480 con 0x7fa898101de0 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89e787640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa898106f10 0x7fa89819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa89df86640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981984d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa894006010 tx=0x7fa8940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa88effd640 1 -- 172.21.15.16:0/943092763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89403f030 con 0x7fa898101de0 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89819d710 con 0x7fa898101de0 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa88effd640 1 -- 172.21.15.16:0/943092763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa894003ce0 con 0x7fa898101de0 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa88effd640 1 -- 172.21.15.16:0/943092763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894035070 con 0x7fa898101de0 2024-09-15T22:49:38.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.513+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8981a5050 con 0x7fa898101de0 2024-09-15T22:49:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.515+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8981021c0 con 0x7fa898101de0 2024-09-15T22:49:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.516+0000 7fa88effd640 1 -- 172.21.15.16:0/943092763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa894042030 con 0x7fa898101de0 2024-09-15T22:49:38.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.516+0000 7fa88effd640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa8840a33b0 0x7fa8840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.517+0000 7fa89d785640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa8840a33b0 0x7fa8840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.517+0000 7fa88effd640 1 -- 172.21.15.16:0/943092763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9467+0+0 (secure 0 0 0) 0x7fa8940f0950 con 0x7fa898101de0 2024-09-15T22:49:38.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.517+0000 7fa89d785640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa8840a33b0 0x7fa8840a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8980ff840 tx=0x7fa888005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:38.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.520+0000 7fa88effd640 1 -- 172.21.15.16:0/943092763 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8940bcfe0 con 0x7fa898101de0 2024-09-15T22:49:38.694 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:38 smithi059 ceph-mon[34872]: pgmap v349: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-15T22:49:38.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.730+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa89810a0f0 con 0x7fa898101de0 2024-09-15T22:49:38.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:38 smithi016 ceph-mon[31492]: pgmap v349: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-15T22:49:38.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.731+0000 7fa88effd640 1 -- 172.21.15.16:0/943092763 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11912 (secure 0 0 0) 0x7fa8940bc8e0 con 0x7fa898101de0 2024-09-15T22:49:38.733 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:49:38.733 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"btime":"2024-09-15T22:38:23:707563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14532,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2727578831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2727578831},{"type":"v1","addr":"172.21.15.16:6835","nonce":2727578831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/4208747900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":4208747900},{"type":"v1","addr":"172.21.15.16:6837","nonce":4208747900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14592,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/1483095493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":1483095493},{"type":"v1","addr":"172.21.15.16:6839","nonce":1483095493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14604,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/433045305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":433045305},{"type":"v1","addr":"172.21.15.16:6841","nonce":433045305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.130:6833/2632425547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6832","nonce":2632425547},{"type":"v1","addr":"172.21.15.130:6833","nonce":2632425547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/1429864622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1429864622},{"type":"v1","addr":"172.21.15.59:6833","nonce":1429864622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:36:08.871593+0000","modified":"2024-09-15T22:38:23.657387+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24500,"mds_1":24479,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24445":{"gid":24445,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6835/2738921095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6834","nonce":2738921095},{"type":"v1","addr":"172.21.15.130:6835","nonce":2738921095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6835/2202482741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":2202482741},{"type":"v1","addr":"172.21.15.59:6835","nonce":2202482741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.130:6837/1773539140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6836","nonce":1773539140},{"type":"v1","addr":"172.21.15.130:6837","nonce":1773539140}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":15,"addr":"172.21.15.130:6839/1236682624","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6838","nonce":1236682624},{"type":"v1","addr":"172.21.15.130:6839","nonce":1236682624}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.59:6837/2084675816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":2084675816},{"type":"v1","addr":"172.21.15.59:6837","nonce":2084675816}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6839/384228279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":384228279},{"type":"v1","addr":"172.21.15.59:6839","nonce":384228279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24479,24469]},"id":1}]} 2024-09-15T22:49:38.733 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-09-15T22:49:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa8840a33b0 msgr2=0x7fa8840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa8840a33b0 0x7fa8840a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8980ff840 tx=0x7fa888005f90 comp rx=0 tx=0).stop 2024-09-15T22:49:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 msgr2=0x7fa8981984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981984d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa894006010 tx=0x7fa8940042f0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 shutdown_connections 2024-09-15T22:49:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa898106f10 0x7fa89819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7fa8840a33b0 0x7fa8840a5870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7fa898102790 0x7fa898198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 --2- 172.21.15.16:0/943092763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa898101de0 0x7fa8981984d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 >> 172.21.15.16:0/943092763 conn(0x7fa8980fd8c0 msgr2=0x7fa8980fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 shutdown_connections 2024-09-15T22:49:38.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:38.734+0000 7fa8a01ea640 1 -- 172.21.15.16:0/943092763 wait complete. 2024-09-15T22:49:38.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:38 smithi130 ceph-mon[35742]: pgmap v349: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-15T22:49:39.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:49:39.728 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:39.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:39 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/943092763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:39.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:39 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/943092763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:39.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:39 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/943092763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 -- 172.21.15.16:0/936560887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601082a0 msgr2=0x7f9b6010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 --2- 172.21.15.16:0/936560887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601082a0 0x7f9b6010ec40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009930 tx=0x7f9b5802a420 comp rx=0 tx=0).stop 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 -- 172.21.15.16:0/936560887 shutdown_connections 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 --2- 172.21.15.16:0/936560887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601082a0 0x7f9b6010ec40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 --2- 172.21.15.16:0/936560887 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b601036e0 0x7f9b60107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 --2- 172.21.15.16:0/936560887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b60102d30 0x7f9b60103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 -- 172.21.15.16:0/936560887 >> 172.21.15.16:0/936560887 conn(0x7f9b600fc7d0 msgr2=0x7f9b600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 -- 172.21.15.16:0/936560887 shutdown_connections 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 -- 172.21.15.16:0/936560887 wait complete. 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 Processor -- start 2024-09-15T22:49:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.219+0000 7f9b676b4640 1 -- start start 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b676b4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b60102d30 0x7f9b60198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b676b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601036e0 0x7f9b60198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b676b4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b601082a0 0x7f9b6019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b676b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60112080 con 0x7f9b601036e0 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b676b4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60111f00 con 0x7f9b60102d30 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b676b4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f9b60112200 con 0x7f9b601082a0 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601036e0 0x7f9b60198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601036e0 0x7f9b60198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52670/0 (socket says 172.21.15.16:52670) 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 -- 172.21.15.16:0/3098435096 learned_addr learned my addr 172.21.15.16:0/3098435096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b65c51640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b601082a0 0x7f9b6019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b65450640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b60102d30 0x7f9b60198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 -- 172.21.15.16:0/3098435096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b601082a0 msgr2=0x7f9b6019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b601082a0 0x7f9b6019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 -- 172.21.15.16:0/3098435096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b60102d30 msgr2=0x7f9b60198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b60102d30 0x7f9b60198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 -- 172.21.15.16:0/3098435096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6019d530 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b65450640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b60102d30 0x7f9b60198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b64c4f640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601036e0 0x7f9b60198ac0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400ea50 tx=0x7f9b5400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.220+0000 7f9b5e7fc640 1 -- 172.21.15.16:0/3098435096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400cdc0 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.221+0000 7f9b5e7fc640 1 -- 172.21.15.16:0/3098435096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54011950 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.221+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6019d820 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.221+0000 7f9b5e7fc640 1 -- 172.21.15.16:0/3098435096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5401aaf0 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.221+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b601a5100 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.221+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b60103110 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.223+0000 7f9b5e7fc640 1 -- 172.21.15.16:0/3098435096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9b5401ac90 con 0x7f9b601036e0 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.223+0000 7f9b5e7fc640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9b480a3430 0x7f9b480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.223+0000 7f9b65450640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9b480a3430 0x7f9b480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.223+0000 7f9b5e7fc640 1 -- 172.21.15.16:0/3098435096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9467+0+0 (secure 0 0 0) 0x7f9b540d07d0 con 0x7f9b601036e0 2024-09-15T22:49:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.224+0000 7f9b65450640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9b480a3430 0x7f9b480a58f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b500058c0 tx=0x7f9b50005d30 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.227+0000 7f9b5e7fc640 1 -- 172.21.15.16:0/3098435096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b54016030 con 0x7f9b601036e0 2024-09-15T22:49:40.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.405+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b601998e0 con 0x7f9b601036e0 2024-09-15T22:49:40.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.406+0000 7f9b5e7fc640 1 -- 172.21.15.16:0/3098435096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25320 (secure 0 0 0) 0x7f9b601998e0 con 0x7f9b601036e0 2024-09-15T22:49:40.409 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-15T22:49:40.409 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":100,"fsid":"af43fc90-73b1-11ef-bceb-c7b262605968","created":"2024-09-15T22:28:23.528863+0000","modified":"2024-09-15T22:49:14.395507+0000","last_up_change":"2024-09-15T22:35:17.944406+0000","last_in_change":"2024-09-15T22:34:55.417138+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:31:31.005703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:36:04.983944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:36:07.225026+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:36:13.700134+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"61d5e373-f70b-4535-bbb2-815a227c3404","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6803","nonce":211991038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6805","nonce":211991038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6809","nonce":211991038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":211991038},{"type":"v1","addr":"172.21.15.16:6807","nonce":211991038}]},"public_addr":"172.21.15.16:6803/211991038","cluster_addr":"172.21.15.16:6805/211991038","heartbeat_back_addr":"172.21.15.16:6809/211991038","heartbeat_front_addr":"172.21.15.16:6807/211991038","state":["exists","up"]},{"osd":1,"uuid":"d9d5166a-3b14-4810-aa73-0fc1f5defad1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6801","nonce":1149325299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6803","nonce":1149325299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6807","nonce":1149325299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":1149325299},{"type":"v1","addr":"172.21.15.59:6805","nonce":1149325299}]},"public_addr":"172.21.15.59:6801/1149325299","cluster_addr":"172.21.15.59:6803/1149325299","heartbeat_back_addr":"172.21.15.59:6807/1149325299","heartbeat_front_addr":"172.21.15.59:6805/1149325299","state":["exists","up"]},{"osd":2,"uuid":"ad8259e8-4042-4f91-8ca4-10bb8d40315a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6800","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6801","nonce":4062265693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6802","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6803","nonce":4062265693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6806","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6807","nonce":4062265693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6804","nonce":4062265693},{"type":"v1","addr":"172.21.15.130:6805","nonce":4062265693}]},"public_addr":"172.21.15.130:6801/4062265693","cluster_addr":"172.21.15.130:6803/4062265693","heartbeat_back_addr":"172.21.15.130:6807/4062265693","heartbeat_front_addr":"172.21.15.130:6805/4062265693","state":["exists","up"]},{"osd":3,"uuid":"157fd25f-1750-4f7e-8745-5131ea71a1a4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6811","nonce":1167232164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6813","nonce":1167232164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6817","nonce":1167232164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":1167232164},{"type":"v1","addr":"172.21.15.16:6815","nonce":1167232164}]},"public_addr":"172.21.15.16:6811/1167232164","cluster_addr":"172.21.15.16:6813/1167232164","heartbeat_back_addr":"172.21.15.16:6817/1167232164","heartbeat_front_addr":"172.21.15.16:6815/1167232164","state":["exists","up"]},{"osd":4,"uuid":"89b8e735-5627-4a56-b82c-8a9477c44fa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6809","nonce":2983797571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6811","nonce":2983797571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6815","nonce":2983797571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":2983797571},{"type":"v1","addr":"172.21.15.59:6813","nonce":2983797571}]},"public_addr":"172.21.15.59:6809/2983797571","cluster_addr":"172.21.15.59:6811/2983797571","heartbeat_back_addr":"172.21.15.59:6815/2983797571","heartbeat_front_addr":"172.21.15.59:6813/2983797571","state":["exists","up"]},{"osd":5,"uuid":"5a628733-0612-40c3-97b1-cf54a3d990d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6808","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6809","nonce":4126131610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6810","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6811","nonce":4126131610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6814","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6815","nonce":4126131610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6812","nonce":4126131610},{"type":"v1","addr":"172.21.15.130:6813","nonce":4126131610}]},"public_addr":"172.21.15.130:6809/4126131610","cluster_addr":"172.21.15.130:6811/4126131610","heartbeat_back_addr":"172.21.15.130:6815/4126131610","heartbeat_front_addr":"172.21.15.130:6813/4126131610","state":["exists","up"]},{"osd":6,"uuid":"8da934b1-44dc-4944-87e9-21e7b09dddfe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6819","nonce":2712238672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6821","nonce":2712238672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6825","nonce":2712238672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":2712238672},{"type":"v1","addr":"172.21.15.16:6823","nonce":2712238672}]},"public_addr":"172.21.15.16:6819/2712238672","cluster_addr":"172.21.15.16:6821/2712238672","heartbeat_back_addr":"172.21.15.16:6825/2712238672","heartbeat_front_addr":"172.21.15.16:6823/2712238672","state":["exists","up"]},{"osd":7,"uuid":"d4d481db-97f6-4231-9da0-fc59f0dd4f27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6817","nonce":1929682102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6819","nonce":1929682102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6823","nonce":1929682102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":1929682102},{"type":"v1","addr":"172.21.15.59:6821","nonce":1929682102}]},"public_addr":"172.21.15.59:6817/1929682102","cluster_addr":"172.21.15.59:6819/1929682102","heartbeat_back_addr":"172.21.15.59:6823/1929682102","heartbeat_front_addr":"172.21.15.59:6821/1929682102","state":["exists","up"]},{"osd":8,"uuid":"07aa015e-5371-433a-b080-07fef3dd8c8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6816","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6817","nonce":3134922308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6818","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6819","nonce":3134922308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6822","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6823","nonce":3134922308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6820","nonce":3134922308},{"type":"v1","addr":"172.21.15.130:6821","nonce":3134922308}]},"public_addr":"172.21.15.130:6817/3134922308","cluster_addr":"172.21.15.130:6819/3134922308","heartbeat_back_addr":"172.21.15.130:6823/3134922308","heartbeat_front_addr":"172.21.15.130:6821/3134922308","state":["exists","up"]},{"osd":9,"uuid":"995607d1-c7a6-4ad1-9538-192ed3d439fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6827","nonce":2698575932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6829","nonce":2698575932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6833","nonce":2698575932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":2698575932},{"type":"v1","addr":"172.21.15.16:6831","nonce":2698575932}]},"public_addr":"172.21.15.16:6827/2698575932","cluster_addr":"172.21.15.16:6829/2698575932","heartbeat_back_addr":"172.21.15.16:6833/2698575932","heartbeat_front_addr":"172.21.15.16:6831/2698575932","state":["exists","up"]},{"osd":10,"uuid":"421e58d7-471b-43aa-908e-828f72805842","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6825","nonce":2888838982}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6827","nonce":2888838982}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6831","nonce":2888838982}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":2888838982},{"type":"v1","addr":"172.21.15.59:6829","nonce":2888838982}]},"public_addr":"172.21.15.59:6825/2888838982","cluster_addr":"172.21.15.59:6827/2888838982","heartbeat_back_addr":"172.21.15.59:6831/2888838982","heartbeat_front_addr":"172.21.15.59:6829/2888838982","state":["exists","up"]},{"osd":11,"uuid":"aa7b2bbd-e5b1-4b42-a4a1-934185f54929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6824","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6825","nonce":790310360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6826","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6827","nonce":790310360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6830","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6831","nonce":790310360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.130:6828","nonce":790310360},{"type":"v1","addr":"172.21.15.130:6829","nonce":790310360}]},"public_addr":"172.21.15.130:6825/790310360","cluster_addr":"172.21.15.130:6827/790310360","heartbeat_back_addr":"172.21.15.130:6831/790310360","heartbeat_front_addr":"172.21.15.130:6829/790310360","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:30:36.678970+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:01.997554+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:27.336967+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:31:51.744485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:18.267563+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:32:42.377620+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:08.663859+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:34.265207+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:33:59.310422+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:25.726307+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:34:51.770599+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:35:16.471940+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6800/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/3275020767":"2024-09-16T22:38:25.000235+0000","172.21.15.16:6801/3949807891":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/1563124617":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6800/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3860212428":"2024-09-16T22:28:46.507397+0000","172.21.15.16:6801/2652317700":"2024-09-16T22:29:10.483605+0000","172.21.15.16:6801/81475521":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3381483588":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3905659611":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/1740394357":"2024-09-16T22:29:10.483605+0000","172.21.15.16:0/3377264779":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3984482177":"2024-09-16T22:38:25.000235+0000","172.21.15.16:0/3985544937":"2024-09-16T22:28:46.507397+0000","172.21.15.16:0/1746300554":"2024-09-16T22:38:25.000235+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:49:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9b480a3430 msgr2=0x7f9b480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9b480a3430 0x7f9b480a58f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b500058c0 tx=0x7f9b50005d30 comp rx=0 tx=0).stop 2024-09-15T22:49:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601036e0 msgr2=0x7f9b60198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:40.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601036e0 0x7f9b60198ac0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400ea50 tx=0x7f9b5400ef10 comp rx=0 tx=0).stop 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 shutdown_connections 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f9b480a3430 0x7f9b480a58f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f9b601082a0 0x7f9b6019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b601036e0 0x7f9b60198ac0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 --2- 172.21.15.16:0/3098435096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b60102d30 0x7f9b60198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 >> 172.21.15.16:0/3098435096 conn(0x7f9b600fc7d0 msgr2=0x7f9b60106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 shutdown_connections 2024-09-15T22:49:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:40.409+0000 7f9b676b4640 1 -- 172.21.15.16:0/3098435096 wait complete. 2024-09-15T22:49:40.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:40 smithi016 ceph-mon[31492]: pgmap v350: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-15T22:49:40.774 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:40 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3098435096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:49:40.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:40 smithi130 ceph-mon[35742]: pgmap v350: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-15T22:49:40.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:40 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3098435096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:49:40.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:40 smithi059 ceph-mon[34872]: pgmap v350: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-15T22:49:40.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:40 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3098435096' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:49:41.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-15T22:49:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:41.737 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:41 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:49:41.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:41 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:49:41.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 -- 172.21.15.16:0/1983162632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901082a0 msgr2=0x7f469010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 --2- 172.21.15.16:0/1983162632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901082a0 0x7f469010ec40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f4688009cd0 tx=0x7f468802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 -- 172.21.15.16:0/1983162632 shutdown_connections 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 --2- 172.21.15.16:0/1983162632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901082a0 0x7f469010ec40 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 --2- 172.21.15.16:0/1983162632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46901036e0 0x7f4690107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 --2- 172.21.15.16:0/1983162632 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f4690102d30 0x7f4690103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 -- 172.21.15.16:0/1983162632 >> 172.21.15.16:0/1983162632 conn(0x7f46900fc7d0 msgr2=0x7f46900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 -- 172.21.15.16:0/1983162632 shutdown_connections 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 -- 172.21.15.16:0/1983162632 wait complete. 2024-09-15T22:49:41.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.820+0000 7f46955b9640 1 Processor -- start 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f46955b9640 1 -- start start 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f46955b9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4690102d30 0x7f4690198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f46955b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901036e0 0x7f4690198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f46955b9640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901082a0 0x7f469019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f46955b9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f46901120a0 con 0x7f46901036e0 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f46955b9640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4690111f20 con 0x7f4690102d30 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f46955b9640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f4690112220 con 0x7f46901082a0 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f468e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901036e0 0x7f4690198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f468effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4690102d30 0x7f4690198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f468e7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901036e0 0x7f4690198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47876/0 (socket says 172.21.15.16:47876) 2024-09-15T22:49:41.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f468e7fc640 1 -- 172.21.15.16:0/2541397182 learned_addr learned my addr 172.21.15.16:0/2541397182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.821+0000 7f468f7fe640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901082a0 0x7f469019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f468effd640 1 -- 172.21.15.16:0/2541397182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901082a0 msgr2=0x7f469019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f468effd640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901082a0 0x7f469019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f468effd640 1 -- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901036e0 msgr2=0x7f4690198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f468effd640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901036e0 0x7f4690198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f468effd640 1 -- 172.21.15.16:0/2541397182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469019d520 con 0x7f4690102d30 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f468effd640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4690102d30 0x7f4690198570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4680009b50 tx=0x7f468000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f4677fff640 1 -- 172.21.15.16:0/2541397182 <== mon.2 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680009e50 con 0x7f4690102d30 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469019d810 con 0x7f4690102d30 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f4677fff640 1 -- 172.21.15.16:0/2541397182 <== mon.2 v2:172.21.15.59:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680005500 con 0x7f4690102d30 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f4677fff640 1 -- 172.21.15.16:0/2541397182 <== mon.2 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680019670 con 0x7f4690102d30 2024-09-15T22:49:41.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.822+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901008c0 con 0x7f4690102d30 2024-09-15T22:49:41.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.823+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469006a850 con 0x7f4690102d30 2024-09-15T22:49:41.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.824+0000 7f4677fff640 1 -- 172.21.15.16:0/2541397182 <== mon.2 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4680020020 con 0x7f4690102d30 2024-09-15T22:49:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.825+0000 7f4677fff640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a3550 0x7f46780a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.825+0000 7f468e7fc640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a3550 0x7f46780a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.825+0000 7f4677fff640 1 -- 172.21.15.16:0/2541397182 <== mon.2 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9467+0+0 (secure 0 0 0) 0x7f46800d3a20 con 0x7f4690102d30 2024-09-15T22:49:41.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.826+0000 7f468e7fc640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a3550 0x7f46780a5a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46900fede0 tx=0x7f4684005f90 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:41.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:41.828+0000 7f4677fff640 1 -- 172.21.15.16:0/2541397182 <== mon.2 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f468009fff0 con 0x7f4690102d30 2024-09-15T22:49:41.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:41 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:49:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.059+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4690103110 con 0x7f4690102d30 2024-09-15T22:49:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.067+0000 7f4677fff640 1 -- 172.21.15.16:0/2541397182 <== mon.2 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f468009f8f0 con 0x7f4690102d30 2024-09-15T22:49:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-15T22:49:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a3550 msgr2=0x7f46780a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a3550 0x7f46780a5a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f46900fede0 tx=0x7f4684005f90 comp rx=0 tx=0).stop 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4690102d30 msgr2=0x7f4690198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4690102d30 0x7f4690198570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4680009b50 tx=0x7f468000b690 comp rx=0 tx=0).stop 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 shutdown_connections 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f46780a3550 0x7f46780a5a10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f46901082a0 0x7f469019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46901036e0 0x7f4690198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 --2- 172.21.15.16:0/2541397182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4690102d30 0x7f4690198570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 >> 172.21.15.16:0/2541397182 conn(0x7f46900fc7d0 msgr2=0x7f4690105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 shutdown_connections 2024-09-15T22:49:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:42.069+0000 7f46955b9640 1 -- 172.21.15.16:0/2541397182 wait complete. 2024-09-15T22:49:42.442 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a[31488]: 2024-09-15T22:49:42.060+0000 7ff60a190640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:49:42.444 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: ignoring --setuser ceph since I am not root 2024-09-15T22:49:42.444 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: ignoring --setgroup ceph since I am not root 2024-09-15T22:49:42.444 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mds[65293]: -- 172.21.15.59:0/2968982535 <== mon.2 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ccfd0dd860 con 0x55ccfd0bb000 2024-09-15T22:49:42.444 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: starting mds.b at 2024-09-15T22:49:42.444 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h[64976]: ignoring --setuser ceph since I am not root 2024-09-15T22:49:42.444 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h[64976]: ignoring --setgroup ceph since I am not root 2024-09-15T22:49:42.444 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mds[64980]: -- 172.21.15.59:0/2379221495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610c0b55c20 con 0x5610c0b32800 2024-09-15T22:49:42.445 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h[64976]: starting mds.h at 2024-09-15T22:49:42.445 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e[64670]: ignoring --setuser ceph since I am not root 2024-09-15T22:49:42.445 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e[64670]: ignoring --setgroup ceph since I am not root 2024-09-15T22:49:42.445 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mds[64674]: -- 172.21.15.59:0/3991122200 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559698c49860 con 0x559698c27400 2024-09-15T22:49:42.445 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e[64670]: starting mds.e at 2024-09-15T22:49:42.529 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c[65634]: ignoring --setuser ceph since I am not root 2024-09-15T22:49:42.529 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c[65634]: ignoring --setgroup ceph since I am not root 2024-09-15T22:49:42.529 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mds[65638]: -- 172.21.15.130:0/1137964118 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c65515860 con 0x557c654f2800 2024-09-15T22:49:42.529 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c[65634]: starting mds.c at 2024-09-15T22:49:42.529 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f[65330]: ignoring --setuser ceph since I am not root 2024-09-15T22:49:42.529 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f[65330]: ignoring --setgroup ceph since I am not root 2024-09-15T22:49:42.530 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mds[65334]: -- 172.21.15.130:0/4206231520 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae2a405c20 con 0x55ae2a3e2800 2024-09-15T22:49:42.530 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f[65330]: starting mds.f at 2024-09-15T22:49:42.530 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i[65938]: ignoring --setuser ceph since I am not root 2024-09-15T22:49:42.530 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i[65938]: ignoring --setgroup ceph since I am not root 2024-09-15T22:49:42.530 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mds[65942]: -- 172.21.15.130:0/3022726623 <== mon.1 v2:172.21.15.130:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f28db0d860 con 0x55f28daeb400 2024-09-15T22:49:42.530 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i[65938]: starting mds.i at 2024-09-15T22:49:42.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: pgmap v351: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2541397182' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:49:43.029 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:49:43.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:49:43.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:49:43.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:49:43.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T22:49:43.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:49:43.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:49:43.030 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:42 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.042 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:43.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: pgmap v351: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2541397182' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:49:43.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:49:43.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:49:43.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:49:43.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T22:49:43.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:49:43.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:49:43.058 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:42 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: pgmap v351: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-15T22:49:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2541397182' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:49:43.194 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:49:43.195 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:42 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' 2024-09-15T22:49:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 -- 172.21.15.16:0/2753433177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c10c2f0 msgr2=0x7f228c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 --2- 172.21.15.16:0/2753433177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c10c2f0 0x7f228c10e790 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2284009ca0 tx=0x7f228402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:49:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 -- 172.21.15.16:0/2753433177 shutdown_connections 2024-09-15T22:49:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 --2- 172.21.15.16:0/2753433177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c10c2f0 0x7f228c10e790 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 --2- 172.21.15.16:0/2753433177 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f228c1037a0 0x7f228c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 --2- 172.21.15.16:0/2753433177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c102df0 0x7f228c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 -- 172.21.15.16:0/2753433177 >> 172.21.15.16:0/2753433177 conn(0x7f228c0fc7d0 msgr2=0x7f228c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 -- 172.21.15.16:0/2753433177 shutdown_connections 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.523+0000 7f22918d4640 1 -- 172.21.15.16:0/2753433177 wait complete. 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 Processor -- start 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 -- start start 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f228c102df0 0x7f228c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c1037a0 0x7f228c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c10c2f0 0x7f228c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1120f0 con 0x7f228c1037a0 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f228c111f70 con 0x7f228c10c2f0 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f22918d4640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f228c112270 con 0x7f228c102df0 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f2283fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c1037a0 0x7f228c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f2283fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c1037a0 0x7f228c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47882/0 (socket says 172.21.15.16:47882) 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f2283fff640 1 -- 172.21.15.16:0/2874124059 learned_addr learned my addr 172.21.15.16:0/2874124059 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f228b7fe640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c10c2f0 0x7f228c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.524+0000 7f228affd640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f228c102df0 0x7f228c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f2283fff640 1 -- 172.21.15.16:0/2874124059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f228c102df0 msgr2=0x7f228c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f2283fff640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f228c102df0 0x7f228c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f2283fff640 1 -- 172.21.15.16:0/2874124059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c10c2f0 msgr2=0x7f228c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f2283fff640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c10c2f0 0x7f228c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f2283fff640 1 -- 172.21.15.16:0/2874124059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c19d4c0 con 0x7f228c1037a0 2024-09-15T22:49:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f2283fff640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c1037a0 0x7f228c198a50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f227400c8c0 tx=0x7f227400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f22908d2640 1 -- 172.21.15.16:0/2874124059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274007bb0 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c19d7b0 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f22908d2640 1 -- 172.21.15.16:0/2874124059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2274007d50 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f22908d2640 1 -- 172.21.15.16:0/2874124059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274018830 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.525+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c1a5090 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.526+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228c1031d0 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.527+0000 7f22908d2640 1 -- 172.21.15.16:0/2874124059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f22740189f0 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.528+0000 7f22908d2640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f22780a3430 0x7f22780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.528+0000 7f228affd640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f22780a3430 0x7f22780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.528+0000 7f22908d2640 1 -- 172.21.15.16:0/2874124059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9637+0+0 (secure 0 0 0) 0x7f22740d0420 con 0x7f228c1037a0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.529+0000 7f228affd640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f22780a3430 0x7f22780a58f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f227c00bbe0 tx=0x7f227c00b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.531+0000 7f22908d2640 1 -- 172.21.15.16:0/2874124059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2274014030 con 0x7f228c1037a0 2024-09-15T22:49:43.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:43.745+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f228c1a52c0 con 0x7f228c1037a0 2024-09-15T22:49:44.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.069+0000 7f22908d2640 1 -- 172.21.15.16:0/2874124059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f227409ca50 con 0x7f228c1037a0 2024-09-15T22:49:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f22780a3430 msgr2=0x7f22780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f22780a3430 0x7f22780a58f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f227c00bbe0 tx=0x7f227c00b390 comp rx=0 tx=0).stop 2024-09-15T22:49:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c1037a0 msgr2=0x7f228c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c1037a0 0x7f228c198a50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f227400c8c0 tx=0x7f227400cd80 comp rx=0 tx=0).stop 2024-09-15T22:49:44.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 shutdown_connections 2024-09-15T22:49:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f228c10c2f0 0x7f228c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f228c1037a0 0x7f228c198a50 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f22780a3430 0x7f22780a58f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 --2- 172.21.15.16:0/2874124059 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f228c102df0 0x7f228c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 >> 172.21.15.16:0/2874124059 conn(0x7f228c0fc7d0 msgr2=0x7f228c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 shutdown_connections 2024-09-15T22:49:44.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:44.072+0000 7f22918d4640 1 -- 172.21.15.16:0/2874124059 wait complete. 2024-09-15T22:49:44.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6834/779262586,v1:172.21.15.130:6835/779262586] up:boot 2024-09-15T22:49:44.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6834/3353322754,v1:172.21.15.59:6835/3353322754] up:boot 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6837/1467344549,v1:172.21.15.59:6839/1467344549] up:boot 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.59:6836/4227845139,v1:172.21.15.59:6838/4227845139] up:boot 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6836/964869348,v1:172.21.15.130:6838/964869348] up:boot 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: mds.? [v2:172.21.15.130:6837/1171108522,v1:172.21.15.130:6839/1171108522] up:boot 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: pgmap v353: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-15T22:49:44.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:44 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2874124059' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:49:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6834/779262586,v1:172.21.15.130:6835/779262586] up:boot 2024-09-15T22:49:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6834/3353322754,v1:172.21.15.59:6835/3353322754] up:boot 2024-09-15T22:49:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6837/1467344549,v1:172.21.15.59:6839/1467344549] up:boot 2024-09-15T22:49:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.59:6836/4227845139,v1:172.21.15.59:6838/4227845139] up:boot 2024-09-15T22:49:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6836/964869348,v1:172.21.15.130:6838/964869348] up:boot 2024-09-15T22:49:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: mds.? [v2:172.21.15.130:6837/1171108522,v1:172.21.15.130:6839/1171108522] up:boot 2024-09-15T22:49:44.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: pgmap v353: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-15T22:49:44.445 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:44 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2874124059' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6834/779262586,v1:172.21.15.130:6835/779262586] up:boot 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6834/3353322754,v1:172.21.15.59:6835/3353322754] up:boot 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6837/1467344549,v1:172.21.15.59:6839/1467344549] up:boot 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.59:6836/4227845139,v1:172.21.15.59:6838/4227845139] up:boot 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6836/964869348,v1:172.21.15.130:6838/964869348] up:boot 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: mds.? [v2:172.21.15.130:6837/1171108522,v1:172.21.15.130:6839/1171108522] up:boot 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:49:44.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:49:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:49:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:49:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:49:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: from='mgr.15234 172.21.15.16:0/2368687743' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:49:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: pgmap v353: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-15T22:49:44.530 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:44 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2874124059' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:49:44.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-15T22:49:45.005 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:45.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:45 smithi016 ceph-mon[31492]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:49:45.404 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:45 smithi016 ceph-mon[31492]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:49:45.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:45 smithi016 ceph-mon[31492]: Cluster is now healthy 2024-09-15T22:49:45.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:45 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/2874124059' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:49:45.405 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:45 smithi016 ceph-mon[31492]: fsmap 12 up:standby 2024-09-15T22:49:45.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:45 smithi059 ceph-mon[34872]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:49:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:45 smithi059 ceph-mon[34872]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:49:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:45 smithi059 ceph-mon[34872]: Cluster is now healthy 2024-09-15T22:49:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:45 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/2874124059' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:49:45.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:45 smithi059 ceph-mon[34872]: fsmap 12 up:standby 2024-09-15T22:49:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 -- 172.21.15.16:0/1626599910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c106f10 msgr2=0x7f573c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 --2- 172.21.15.16:0/1626599910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c106f10 0x7f573c10ea90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5734009ca0 tx=0x7f573402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 -- 172.21.15.16:0/1626599910 shutdown_connections 2024-09-15T22:49:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 --2- 172.21.15.16:0/1626599910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c106f10 0x7f573c10ea90 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 --2- 172.21.15.16:0/1626599910 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f573c102790 0x7f573c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 --2- 172.21.15.16:0/1626599910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c101de0 0x7f573c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 -- 172.21.15.16:0/1626599910 >> 172.21.15.16:0/1626599910 conn(0x7f573c0fd8c0 msgr2=0x7f573c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 -- 172.21.15.16:0/1626599910 shutdown_connections 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 -- 172.21.15.16:0/1626599910 wait complete. 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 Processor -- start 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.490+0000 7f5742420640 1 -- start start 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5742420640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c101de0 0x7f573c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5742420640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f573c102790 0x7f573c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5742420640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c106f10 0x7f573c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5742420640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f573c111f70 con 0x7f573c101de0 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5742420640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f573c111df0 con 0x7f573c106f10 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5742420640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f573c1120f0 con 0x7f573c102790 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c101de0 0x7f573c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c101de0 0x7f573c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47904/0 (socket says 172.21.15.16:47904) 2024-09-15T22:49:45.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 -- 172.21.15.16:0/1168940030 learned_addr learned my addr 172.21.15.16:0/1168940030 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f57409bd640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c106f10 0x7f573c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5733fff640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f573c102790 0x7f573c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 -- 172.21.15.16:0/1168940030 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f573c102790 msgr2=0x7f573c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f573c102790 0x7f573c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 -- 172.21.15.16:0/1168940030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c106f10 msgr2=0x7f573c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c106f10 0x7f573c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 -- 172.21.15.16:0/1168940030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573c19d3a0 con 0x7f573c101de0 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f573bfff640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c101de0 0x7f573c1983f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d950 tx=0x7f572c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5739ffb640 1 -- 172.21.15.16:0/1168940030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c017030 con 0x7f573c101de0 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.491+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573c19d690 con 0x7f573c101de0 2024-09-15T22:49:45.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.492+0000 7f5739ffb640 1 -- 172.21.15.16:0/1168940030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f572c004020 con 0x7f573c101de0 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.492+0000 7f5739ffb640 1 -- 172.21.15.16:0/1168940030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572c0047a0 con 0x7f573c101de0 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.492+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f573c1a4f70 con 0x7f573c101de0 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.492+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573c1021c0 con 0x7f573c101de0 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.494+0000 7f5739ffb640 1 -- 172.21.15.16:0/1168940030 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f572c0041d0 con 0x7f573c101de0 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.495+0000 7f5739ffb640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f57280a33b0 0x7f57280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.495+0000 7f5733fff640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f57280a33b0 0x7f57280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.495+0000 7f5739ffb640 1 -- 172.21.15.16:0/1168940030 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9637+0+0 (secure 0 0 0) 0x7f572c0d0260 con 0x7f573c101de0 2024-09-15T22:49:45.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.495+0000 7f5733fff640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f57280a33b0 0x7f57280a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5724005fd0 tx=0x7f572400b510 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:45.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.498+0000 7f5739ffb640 1 -- 172.21.15.16:0/1168940030 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f572c09c780 con 0x7f573c101de0 2024-09-15T22:49:45.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:45 smithi130 ceph-mon[35742]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:49:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:45 smithi130 ceph-mon[35742]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:49:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:45 smithi130 ceph-mon[35742]: Cluster is now healthy 2024-09-15T22:49:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:45 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/2874124059' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:49:45.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:45 smithi130 ceph-mon[35742]: fsmap 12 up:standby 2024-09-15T22:49:45.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:45.675+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f573c10a150 con 0x7f573c101de0 2024-09-15T22:49:46.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.073+0000 7f5739ffb640 1 -- 172.21.15.16:0/1168940030 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f572c09c080 con 0x7f573c101de0 2024-09-15T22:49:46.075 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-15T22:49:46.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f57280a33b0 msgr2=0x7f57280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f57280a33b0 0x7f57280a5870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5724005fd0 tx=0x7f572400b510 comp rx=0 tx=0).stop 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c101de0 msgr2=0x7f573c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c101de0 0x7f573c1983f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d950 tx=0x7f572c00de10 comp rx=0 tx=0).stop 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 shutdown_connections 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f573c106f10 0x7f573c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f57280a33b0 0x7f57280a5870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f573c102790 0x7f573c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 --2- 172.21.15.16:0/1168940030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c101de0 0x7f573c1983f0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 >> 172.21.15.16:0/1168940030 conn(0x7f573c0fd8c0 msgr2=0x7f573c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 shutdown_connections 2024-09-15T22:49:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:46.077+0000 7f5742420640 1 -- 172.21.15.16:0/1168940030 wait complete. 2024-09-15T22:49:46.401 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:46 smithi016 ceph-mon[31492]: pgmap v354: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-15T22:49:46.401 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:46 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1168940030' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:46.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:46 smithi059 ceph-mon[34872]: pgmap v354: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-15T22:49:46.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:46 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1168940030' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:46 smithi130 ceph-mon[35742]: pgmap v354: 129 pgs: 129 active+clean; 17 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-15T22:49:46.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:46 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1168940030' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:46.744 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-15T22:49:47.006 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:47.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:47 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/1168940030' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:47.396 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:47 smithi016 ceph-mon[31492]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:49:47.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:47 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/1168940030' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:47.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:47 smithi059 ceph-mon[34872]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 -- 172.21.15.16:0/3521519469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 msgr2=0x7f2224106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 --2- 172.21.15.16:0/3521519469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f2224106e60 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2214009cd0 tx=0x7f221402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 -- 172.21.15.16:0/3521519469 shutdown_connections 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 --2- 172.21.15.16:0/3521519469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2224101420 0x7f222410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 --2- 172.21.15.16:0/3521519469 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2224100a80 0x7f2224100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 --2- 172.21.15.16:0/3521519469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f2224106e60 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 -- 172.21.15.16:0/3521519469 >> 172.21.15.16:0/3521519469 conn(0x7f22240fc7a0 msgr2=0x7f22240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 -- 172.21.15.16:0/3521519469 shutdown_connections 2024-09-15T22:49:47.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.481+0000 7f2228bec640 1 -- 172.21.15.16:0/3521519469 wait complete. 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.482+0000 7f2228bec640 1 Processor -- start 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.482+0000 7f2228bec640 1 -- start start 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2228bec640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2224100a80 0x7f22241984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2228bec640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2224101420 0x7f22241989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2228bec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f222419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2228bec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2224112040 con 0x7f2224106a80 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2228bec640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f222406b660 con 0x7f2224100a80 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2228bec640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f22241121c0 con 0x7f2224101420 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f222419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f222419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47926/0 (socket says 172.21.15.16:47926) 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 -- 172.21.15.16:0/3996961288 learned_addr learned my addr 172.21.15.16:0/3996961288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2221d9b640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2224101420 0x7f22241989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f222259c640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2224100a80 0x7f22241984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:47.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 -- 172.21.15.16:0/3996961288 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2224101420 msgr2=0x7f22241989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2224101420 0x7f22241989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 -- 172.21.15.16:0/3996961288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2224100a80 msgr2=0x7f22241984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2224100a80 0x7f22241984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 -- 172.21.15.16:0/3996961288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222419d460 con 0x7f2224106a80 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f222259c640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2224100a80 0x7f22241984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f2222d9d640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f222419cd80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c970 tx=0x7f221c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.483+0000 7f22137fe640 1 -- 172.21.15.16:0/3996961288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c007bb0 con 0x7f2224106a80 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.484+0000 7f22137fe640 1 -- 172.21.15.16:0/3996961288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c007d50 con 0x7f2224106a80 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.484+0000 7f22137fe640 1 -- 172.21.15.16:0/3996961288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c00f450 con 0x7f2224106a80 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.484+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222419d750 con 0x7f2224106a80 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.484+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241a5030 con 0x7f2224106a80 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.485+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2224100ee0 con 0x7f2224106a80 2024-09-15T22:49:47.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.486+0000 7f22137fe640 1 -- 172.21.15.16:0/3996961288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f221c004f00 con 0x7f2224106a80 2024-09-15T22:49:47.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.487+0000 7f22137fe640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f220c0a33b0 0x7f220c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:47.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.487+0000 7f222259c640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f220c0a33b0 0x7f220c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:47.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.487+0000 7f22137fe640 1 -- 172.21.15.16:0/3996961288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9112+0+0 (secure 0 0 0) 0x7f221c0d0030 con 0x7f2224106a80 2024-09-15T22:49:47.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.487+0000 7f222259c640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f220c0a33b0 0x7f220c0a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2214000f80 tx=0x7f2214031430 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:47.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.490+0000 7f22137fe640 1 -- 172.21.15.16:0/3996961288 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f221c014030 con 0x7f2224106a80 2024-09-15T22:49:47.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:47 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/1168940030' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:47.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:47 smithi130 ceph-mon[35742]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:49:47.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:47.665+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2224199810 con 0x7f2224106a80 2024-09-15T22:49:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.080+0000 7f22137fe640 1 -- 172.21.15.16:0/3996961288 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f2224199810 con 0x7f2224106a80 2024-09-15T22:49:48.082 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-15T22:49:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f220c0a33b0 msgr2=0x7f220c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f220c0a33b0 0x7f220c0a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2214000f80 tx=0x7f2214031430 comp rx=0 tx=0).stop 2024-09-15T22:49:48.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 msgr2=0x7f222419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f222419cd80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f221c00c970 tx=0x7f221c00ce30 comp rx=0 tx=0).stop 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 shutdown_connections 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2224106a80 0x7f222419cd80 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f220c0a33b0 0x7f220c0a5870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f2224101420 0x7f22241989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 --2- 172.21.15.16:0/3996961288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2224100a80 0x7f22241984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.084+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 >> 172.21.15.16:0/3996961288 conn(0x7f22240fc7a0 msgr2=0x7f22240fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.085+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 shutdown_connections 2024-09-15T22:49:48.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:48.085+0000 7f2228bec640 1 -- 172.21.15.16:0/3996961288 wait complete. 2024-09-15T22:49:48.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:48 smithi016 ceph-mon[31492]: pgmap v356: 97 pgs: 97 active+clean; 577 KiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:49:48.409 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:48 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3996961288' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:48.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:48 smithi059 ceph-mon[34872]: pgmap v356: 97 pgs: 97 active+clean; 577 KiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:49:48.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:48 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3996961288' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:48.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:48 smithi130 ceph-mon[35742]: pgmap v356: 97 pgs: 97 active+clean; 577 KiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:49:48.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:48 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3996961288' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:48.780 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid af43fc90-73b1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-15T22:49:49.049 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:49:49.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:49 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3996961288' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:49.337 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:49 smithi016 ceph-mon[31492]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:49:49.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:49 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3996961288' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:49.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:49 smithi059 ceph-mon[34872]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:49:49.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:49 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3996961288' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:49.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:49 smithi130 ceph-mon[35742]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:49:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 -- 172.21.15.16:0/2780455490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc10c180 msgr2=0x7f72dc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 --2- 172.21.15.16:0/2780455490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc10c180 0x7f72dc10e600 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009ca0 tx=0x7f72d402a950 comp rx=0 tx=0).stop 2024-09-15T22:49:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 -- 172.21.15.16:0/2780455490 shutdown_connections 2024-09-15T22:49:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 --2- 172.21.15.16:0/2780455490 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc10c180 0x7f72dc10e600 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 --2- 172.21.15.16:0/2780455490 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72dc1036c0 0x7f72dc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 --2- 172.21.15.16:0/2780455490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc102da0 0x7f72dc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 -- 172.21.15.16:0/2780455490 >> 172.21.15.16:0/2780455490 conn(0x7f72dc0fc7a0 msgr2=0x7f72dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 -- 172.21.15.16:0/2780455490 shutdown_connections 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.535+0000 7f72e1018640 1 -- 172.21.15.16:0/2780455490 wait complete. 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 Processor -- start 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 -- start start 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc102da0 0x7f72dc100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 --2- >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72dc1036c0 0x7f72dc100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc10c180 0x7f72dc101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc111f60 con 0x7f72dc102da0 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc111de0 con 0x7f72dc10c180 2024-09-15T22:49:49.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72e1018640 1 -- --> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] -- mon_getmap magic: 0 -- 0x7f72dc1120e0 con 0x7f72dc1036c0 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc102da0 0x7f72dc100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc102da0 0x7f72dc100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47950/0 (socket says 172.21.15.16:47950) 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72dad9d640 1 -- 172.21.15.16:0/3848128526 learned_addr learned my addr 172.21.15.16:0/3848128526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72db59e640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc10c180 0x7f72dc101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.536+0000 7f72d3fff640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72dc1036c0 0x7f72dc100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72dad9d640 1 -- 172.21.15.16:0/3848128526 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72dc1036c0 msgr2=0x7f72dc100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72dad9d640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72dc1036c0 0x7f72dc100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72dad9d640 1 -- 172.21.15.16:0/3848128526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc10c180 msgr2=0x7f72dc101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72dad9d640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc10c180 0x7f72dc101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72dad9d640 1 -- 172.21.15.16:0/3848128526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72dc1a6900 con 0x7f72dc102da0 2024-09-15T22:49:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72dad9d640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc102da0 0x7f72dc100a00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c890 tx=0x7f72cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72d8d99640 1 -- 172.21.15.16:0/3848128526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc004220 con 0x7f72dc102da0 2024-09-15T22:49:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72d8d99640 1 -- 172.21.15.16:0/3848128526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72cc0043c0 con 0x7f72dc102da0 2024-09-15T22:49:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72d8d99640 1 -- 172.21.15.16:0/3848128526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0049e0 con 0x7f72dc102da0 2024-09-15T22:49:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72dc1a6bf0 con 0x7f72dc102da0 2024-09-15T22:49:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.537+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72dc1a7130 con 0x7f72dc102da0 2024-09-15T22:49:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.538+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72dc103180 con 0x7f72dc102da0 2024-09-15T22:49:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.539+0000 7f72d8d99640 1 -- 172.21.15.16:0/3848128526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f72cc0079b0 con 0x7f72dc102da0 2024-09-15T22:49:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.540+0000 7f72d8d99640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f72c80a33b0 0x7f72c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.540+0000 7f72d3fff640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f72c80a33b0 0x7f72c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.540+0000 7f72d8d99640 1 -- 172.21.15.16:0/3848128526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8582+0+0 (secure 0 0 0) 0x7f72cc0cfd10 con 0x7f72dc102da0 2024-09-15T22:49:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.541+0000 7f72d3fff640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f72c80a33b0 0x7f72c80a5870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72dc102490 tx=0x7f72c400b390 comp rx=0 tx=0).ready entity=mgr.15234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:49.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.544+0000 7f72d8d99640 1 -- 172.21.15.16:0/3848128526 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72cc014030 con 0x7f72dc102da0 2024-09-15T22:49:49.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:49.722+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f72dc107220 con 0x7f72dc102da0 2024-09-15T22:49:50.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.087+0000 7f72d8d99640 1 -- 172.21.15.16:0/3848128526 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) ==== 178+0+0 (secure 0 0 0) 0x7f72cc09c060 con 0x7f72dc102da0 2024-09-15T22:49:50.089 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2024-09-15T22:49:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.091+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f72c80a33b0 msgr2=0x7f72c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.091+0000 7f72e1018640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f72c80a33b0 0x7f72c80a5870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f72dc102490 tx=0x7f72c400b390 comp rx=0 tx=0).stop 2024-09-15T22:49:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.091+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc102da0 msgr2=0x7f72dc100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:50.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.091+0000 7f72e1018640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc102da0 0x7f72dc100a00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c890 tx=0x7f72cc00cd50 comp rx=0 tx=0).stop 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 shutdown_connections 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72dc10c180 0x7f72dc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:6800/1157292381,v1:172.21.15.16:6801/1157292381] conn(0x7f72c80a33b0 0x7f72c80a5870 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.130:3300/0,v1:172.21.15.130:6789/0] conn(0x7f72dc1036c0 0x7f72dc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 --2- 172.21.15.16:0/3848128526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72dc102da0 0x7f72dc100a00 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 >> 172.21.15.16:0/3848128526 conn(0x7f72dc0fc7a0 msgr2=0x7f72dc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 shutdown_connections 2024-09-15T22:49:50.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-15T22:49:50.092+0000 7f72e1018640 1 -- 172.21.15.16:0/3848128526 wait complete. 2024-09-15T22:49:50.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:50 smithi016 ceph-mon[31492]: pgmap v358: 33 pgs: 33 active+clean; 577 KiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:49:50.417 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:50 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3848128526' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:50.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:50 smithi059 ceph-mon[34872]: pgmap v358: 33 pgs: 33 active+clean; 577 KiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:49:50.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:50 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3848128526' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:50.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:50 smithi130 ceph-mon[35742]: pgmap v358: 33 pgs: 33 active+clean; 577 KiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:49:50.529 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:50 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3848128526' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:49:50.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:49:50.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:49:50.813 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:49:50.845 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-15T22:49:50.845 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:49:50.862 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:49:50.880 DEBUG:teuthology.orchestra.run.smithi130:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:49:50.904 INFO:tasks.cephadm:Stopping all daemons... 2024-09-15T22:49:50.904 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-15T22:49:50.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a 2024-09-15T22:49:51.056 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:50 smithi016 systemd[1]: Stopping Ceph mon.a for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:49:51.443 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:51 smithi059 ceph-mon[34872]: from='client.? 172.21.15.16:0/3848128526' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:51.444 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:51 smithi059 ceph-mon[34872]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:49:51.478 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:51 smithi016 ceph-mon[31492]: from='client.? 172.21.15.16:0/3848128526' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:51.479 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:51 smithi016 ceph-mon[31492]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:49:51.479 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a[31488]: 2024-09-15T22:49:51.220+0000 7ff60f99b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:49:51.479 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a[31488]: 2024-09-15T22:49:51.220+0000 7ff60f99b640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-15T22:49:51.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:51 smithi130 ceph-mon[35742]: from='client.? 172.21.15.16:0/3848128526' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:49:51.528 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:51 smithi130 ceph-mon[35742]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:49:51.773 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:51 smithi016 podman[112980]: 2024-09-15 22:49:51.478215302 +0000 UTC m=+0.376632250 container died 7ea2261f5ae741770b0f78053f0a334e6f8726ac64b9ffa27081cc7d64a28aaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-15T22:49:52.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:51 smithi016 podman[112980]: 2024-09-15 22:49:51.863980961 +0000 UTC m=+0.762397908 container cleanup 7ea2261f5ae741770b0f78053f0a334e6f8726ac64b9ffa27081cc7d64a28aaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-15T22:49:52.057 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 15 22:49:51 smithi016 bash[112980]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-a 2024-09-15T22:49:52.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.a.service' 2024-09-15T22:49:52.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:49:52.363 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-15T22:49:52.363 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-15T22:49:52.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.b 2024-09-15T22:49:52.687 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:52 smithi059 systemd[1]: Stopping Ceph mon.b for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:49:52.943 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-b[34868]: 2024-09-15T22:49:52.686+0000 7feb8aab2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:49:52.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-b[34868]: 2024-09-15T22:49:52.686+0000 7feb8aab2640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-15T22:49:52.944 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:52 smithi059 podman[71145]: 2024-09-15 22:49:52.891285515 +0000 UTC m=+0.332163061 container died 69143470fcd6b33ff54ce1c3ac09321164ccd15286448671dbd8849031469c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-15T22:49:53.623 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:53 smithi059 podman[71145]: 2024-09-15 22:49:53.337216938 +0000 UTC m=+0.778094484 container cleanup 69143470fcd6b33ff54ce1c3ac09321164ccd15286448671dbd8849031469c75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:49:53.623 INFO:journalctl@ceph.mon.b.smithi059.stdout:Sep 15 22:49:53 smithi059 bash[71145]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-b 2024-09-15T22:49:53.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.b.service' 2024-09-15T22:49:53.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:49:53.825 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-15T22:49:53.825 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-15T22:49:53.825 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.c 2024-09-15T22:49:54.154 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:53 smithi130 systemd[1]: Stopping Ceph mon.c for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:49:54.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-c[35738]: 2024-09-15T22:49:54.153+0000 7f086f07a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:49:54.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-c[35738]: 2024-09-15T22:49:54.153+0000 7f086f07a640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-15T22:49:54.523 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:54 smithi130 podman[71875]: 2024-09-15 22:49:54.245711329 +0000 UTC m=+0.220155902 container died 4afbc3ac33c31cd138b154fa4c890259469ac1af0cc043e9e0ebc0848d3a8ce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-15T22:49:54.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:54 smithi130 podman[71875]: 2024-09-15 22:49:54.629697527 +0000 UTC m=+0.604142088 container cleanup 4afbc3ac33c31cd138b154fa4c890259469ac1af0cc043e9e0ebc0848d3a8ce7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T22:49:54.778 INFO:journalctl@ceph.mon.c.smithi130.stdout:Sep 15 22:49:54 smithi130 bash[71875]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mon-c 2024-09-15T22:49:55.061 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mon.c.service' 2024-09-15T22:49:55.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:49:55.109 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-15T22:49:55.109 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-15T22:49:55.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.x 2024-09-15T22:49:55.427 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:49:55 smithi016 systemd[1]: Stopping Ceph mgr.x for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:49:55.806 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:49:55 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:49:55.425+0000 7f4bad42d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:49:55.807 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:49:55 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x[31717]: 2024-09-15T22:49:55.425+0000 7f4bad42d640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-15T22:49:55.807 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:49:55 smithi016 podman[113104]: 2024-09-15 22:49:55.457599103 +0000 UTC m=+0.145974616 container died c77097c54c1fc68eab0a55343f83d3fc8f8652ccc242fafb54f2b4796cce87a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True) 2024-09-15T22:49:56.068 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:49:56 smithi016 podman[113104]: 2024-09-15 22:49:56.052538853 +0000 UTC m=+0.740914366 container cleanup c77097c54c1fc68eab0a55343f83d3fc8f8652ccc242fafb54f2b4796cce87a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-15T22:49:56.068 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 15 22:49:56 smithi016 bash[113104]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-x 2024-09-15T22:49:56.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-15T22:49:56.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:49:56.532 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-15T22:49:56.532 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-15T22:49:56.532 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.y 2024-09-15T22:49:56.845 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:49:56 smithi059 systemd[1]: Stopping Ceph mgr.y for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:49:57.117 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:49:56 smithi059 podman[71268]: 2024-09-15 22:49:56.880061684 +0000 UTC m=+0.149643508 container died ac1b67580f656d4c1bafc4f567f56e4d935e7e85fdb71c692f9ddb8d55002fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-15T22:49:57.443 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:49:57 smithi059 podman[71268]: 2024-09-15 22:49:57.292917903 +0000 UTC m=+0.562499716 container cleanup ac1b67580f656d4c1bafc4f567f56e4d935e7e85fdb71c692f9ddb8d55002fd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-15T22:49:57.443 INFO:journalctl@ceph.mgr.y.smithi059.stdout:Sep 15 22:49:57 smithi059 bash[71268]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-y 2024-09-15T22:49:57.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-15T22:49:57.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:49:57.754 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-15T22:49:57.754 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-15T22:49:57.754 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.z 2024-09-15T22:49:58.091 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:49:57 smithi130 systemd[1]: Stopping Ceph mgr.z for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:49:58.395 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:49:58 smithi130 podman[71996]: 2024-09-15 22:49:58.090535076 +0000 UTC m=+0.151006492 container died b016d80211b6dff4aed2eee43fbef03357bda04e80f49c3d970f35233c92bb76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:49:58.763 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:49:58 smithi130 podman[71996]: 2024-09-15 22:49:58.502628067 +0000 UTC m=+0.563099481 container cleanup b016d80211b6dff4aed2eee43fbef03357bda04e80f49c3d970f35233c92bb76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:49:58.763 INFO:journalctl@ceph.mgr.z.smithi130.stdout:Sep 15 22:49:58 smithi130 bash[71996]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mgr-z 2024-09-15T22:49:58.906 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-15T22:49:58.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:49:58.986 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-15T22:49:58.987 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-15T22:49:58.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.0 2024-09-15T22:49:59.299 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:49:59 smithi016 systemd[1]: Stopping Ceph osd.0 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:49:59.556 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:49:59 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0[45374]: 2024-09-15T22:49:59.298+0000 7f5cbbbcd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:49:59.557 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:49:59 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0[45374]: 2024-09-15T22:49:59.298+0000 7f5cbbbcd640 -1 osd.0 104 *** Got signal Terminated *** 2024-09-15T22:49:59.557 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:49:59 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0[45374]: 2024-09-15T22:49:59.298+0000 7f5cbbbcd640 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:05.057 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:04 smithi016 podman[113228]: 2024-09-15 22:50:04.725611509 +0000 UTC m=+5.536429020 container died 3752ea441a6058d9fd06134ee0341e6bc0df94d94be903a8e8cdb6978d1737b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-15T22:50:05.556 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:05 smithi016 podman[113228]: 2024-09-15 22:50:05.340748788 +0000 UTC m=+6.151566344 container cleanup 3752ea441a6058d9fd06134ee0341e6bc0df94d94be903a8e8cdb6978d1737b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:50:05.557 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:05 smithi016 bash[113228]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0 2024-09-15T22:50:06.026 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:05 smithi016 podman[113244]: 2024-09-15 22:50:05.600801699 +0000 UTC m=+0.873656793 container remove 3752ea441a6058d9fd06134ee0341e6bc0df94d94be903a8e8cdb6978d1737b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:50:06.306 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:06 smithi016 podman[113336]: 2024-09-15 22:50:05.927931841 +0000 UTC m=+0.016405616 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:50:06.307 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:06 smithi016 podman[113336]: 2024-09-15 22:50:06.085441761 +0000 UTC m=+0.173915571 container create 97dba2a6c0efadbd4086e00fb4307629ca2d622e37b8dc8d8e31b2fd27723d31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-15T22:50:06.807 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:06 smithi016 podman[113336]: 2024-09-15 22:50:06.454928369 +0000 UTC m=+0.543402135 container init 97dba2a6c0efadbd4086e00fb4307629ca2d622e37b8dc8d8e31b2fd27723d31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-15T22:50:06.807 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:06 smithi016 podman[113336]: 2024-09-15 22:50:06.458312101 +0000 UTC m=+0.546785875 container start 97dba2a6c0efadbd4086e00fb4307629ca2d622e37b8dc8d8e31b2fd27723d31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-15T22:50:06.807 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:06 smithi016 podman[113336]: 2024-09-15 22:50:06.679980599 +0000 UTC m=+0.768454381 container attach 97dba2a6c0efadbd4086e00fb4307629ca2d622e37b8dc8d8e31b2fd27723d31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-15T22:50:06.807 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 15 22:50:06 smithi016 podman[113336]: 2024-09-15 22:50:06.691734039 +0000 UTC m=+0.780207807 container died 97dba2a6c0efadbd4086e00fb4307629ca2d622e37b8dc8d8e31b2fd27723d31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-15T22:50:07.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.0.service' 2024-09-15T22:50:07.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:50:07.489 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-15T22:50:07.489 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-15T22:50:07.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.1 2024-09-15T22:50:07.891 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:07 smithi059 systemd[1]: Stopping Ceph osd.1 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:50:08.193 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1[39362]: 2024-09-15T22:50:07.889+0000 7f125464a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:50:08.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1[39362]: 2024-09-15T22:50:07.889+0000 7f125464a640 -1 osd.1 104 *** Got signal Terminated *** 2024-09-15T22:50:08.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1[39362]: 2024-09-15T22:50:07.889+0000 7f125464a640 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:13.675 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:13 smithi059 podman[71405]: 2024-09-15 22:50:13.351312336 +0000 UTC m=+5.653473186 container died 4322a1cd0886a271a92c97f9aa5167dbdb72d1c1ce27ab1e78dc737d336fe5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-15T22:50:13.944 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:13 smithi059 podman[71405]: 2024-09-15 22:50:13.773271585 +0000 UTC m=+6.075432416 container cleanup 4322a1cd0886a271a92c97f9aa5167dbdb72d1c1ce27ab1e78dc737d336fe5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:50:13.944 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:13 smithi059 bash[71405]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1 2024-09-15T22:50:14.443 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:14 smithi059 podman[71420]: 2024-09-15 22:50:14.049733592 +0000 UTC m=+0.687860807 container remove 4322a1cd0886a271a92c97f9aa5167dbdb72d1c1ce27ab1e78dc737d336fe5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-15T22:50:14.836 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:14 smithi059 podman[71511]: 2024-09-15 22:50:14.451166434 +0000 UTC m=+0.015988350 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:50:14.837 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:14 smithi059 podman[71511]: 2024-09-15 22:50:14.575461822 +0000 UTC m=+0.140283737 container create c18382c4c076a9965b7366c7fea01adfb381424acbc96681d15212fe4cf1026c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-15T22:50:14.837 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:14 smithi059 podman[71511]: 2024-09-15 22:50:14.836367575 +0000 UTC m=+0.401189500 container init c18382c4c076a9965b7366c7fea01adfb381424acbc96681d15212fe4cf1026c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-15T22:50:15.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:14 smithi059 podman[71511]: 2024-09-15 22:50:14.83979107 +0000 UTC m=+0.404612988 container start c18382c4c076a9965b7366c7fea01adfb381424acbc96681d15212fe4cf1026c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-15T22:50:15.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:14 smithi059 podman[71511]: 2024-09-15 22:50:14.936540833 +0000 UTC m=+0.501362745 container attach c18382c4c076a9965b7366c7fea01adfb381424acbc96681d15212fe4cf1026c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:50:15.194 INFO:journalctl@ceph.osd.1.smithi059.stdout:Sep 15 22:50:15 smithi059 podman[71511]: 2024-09-15 22:50:15.030846345 +0000 UTC m=+0.595668267 container died c18382c4c076a9965b7366c7fea01adfb381424acbc96681d15212fe4cf1026c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-15T22:50:15.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.1.service' 2024-09-15T22:50:15.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:50:15.765 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-15T22:50:15.765 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-15T22:50:15.765 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.2 2024-09-15T22:50:16.100 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:15 smithi130 systemd[1]: Stopping Ceph osd.2 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:50:16.528 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2[40701]: 2024-09-15T22:50:16.098+0000 7fe68a090640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:50:16.528 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2[40701]: 2024-09-15T22:50:16.098+0000 7fe68a090640 -1 osd.2 104 *** Got signal Terminated *** 2024-09-15T22:50:16.529 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2[40701]: 2024-09-15T22:50:16.098+0000 7fe68a090640 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:21.751 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:21 smithi130 podman[72133]: 2024-09-15 22:50:21.456540669 +0000 UTC m=+5.488949189 container died 3bee4528cb3faebfb6fd7e91410eff71fa0936cdf5cca94c7aca1e29ccc081bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T22:50:22.028 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:21 smithi130 podman[72133]: 2024-09-15 22:50:21.840822982 +0000 UTC m=+5.873231503 container cleanup 3bee4528cb3faebfb6fd7e91410eff71fa0936cdf5cca94c7aca1e29ccc081bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-15T22:50:22.029 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:21 smithi130 bash[72133]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2 2024-09-15T22:50:22.460 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:22 smithi130 podman[72149]: 2024-09-15 22:50:22.050509282 +0000 UTC m=+0.593650695 container remove 3bee4528cb3faebfb6fd7e91410eff71fa0936cdf5cca94c7aca1e29ccc081bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-15T22:50:22.755 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:22 smithi130 podman[72239]: 2024-09-15 22:50:22.362657406 +0000 UTC m=+0.016083212 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:50:22.755 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:22 smithi130 podman[72239]: 2024-09-15 22:50:22.493263763 +0000 UTC m=+0.146689564 container create 965291b88520e3c82fce31baf00ab74ed04a40bdc64e930d0d6dd5e04c5a6020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-15T22:50:23.029 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:22 smithi130 podman[72239]: 2024-09-15 22:50:22.754503122 +0000 UTC m=+0.407928923 container init 965291b88520e3c82fce31baf00ab74ed04a40bdc64e930d0d6dd5e04c5a6020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-15T22:50:23.029 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:22 smithi130 podman[72239]: 2024-09-15 22:50:22.758274679 +0000 UTC m=+0.411700475 container start 965291b88520e3c82fce31baf00ab74ed04a40bdc64e930d0d6dd5e04c5a6020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-15T22:50:23.029 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:22 smithi130 podman[72239]: 2024-09-15 22:50:22.846229806 +0000 UTC m=+0.499655616 container attach 965291b88520e3c82fce31baf00ab74ed04a40bdc64e930d0d6dd5e04c5a6020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T22:50:23.029 INFO:journalctl@ceph.osd.2.smithi130.stdout:Sep 15 22:50:22 smithi130 podman[72239]: 2024-09-15 22:50:22.946270977 +0000 UTC m=+0.599696772 container died 965291b88520e3c82fce31baf00ab74ed04a40bdc64e930d0d6dd5e04c5a6020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T22:50:23.625 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.2.service' 2024-09-15T22:50:23.664 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:50:23.664 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-15T22:50:23.665 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-15T22:50:23.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.3 2024-09-15T22:50:23.988 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:23 smithi016 systemd[1]: Stopping Ceph osd.3 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:50:24.306 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:50:23.986+0000 7f06a6a62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:50:24.307 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:50:23.986+0000 7f06a6a62640 -1 osd.3 104 *** Got signal Terminated *** 2024-09-15T22:50:24.307 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:23 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:50:23.986+0000 7f06a6a62640 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:27.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:26 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:26.864+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:28.057 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:27 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:50:27.723+0000 7f069f070640 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:49:59.125410+0000 front 2024-09-15T22:49:59.125318+0000 (oldest deadline 2024-09-15T22:50:26.726126+0000) 2024-09-15T22:50:28.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:27 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:27.886+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:28.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:28.195+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:29.056 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:28 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3[51833]: 2024-09-15T22:50:28.766+0000 7f069f070640 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:49:59.125410+0000 front 2024-09-15T22:49:59.125318+0000 (oldest deadline 2024-09-15T22:50:26.726126+0000) 2024-09-15T22:50:29.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:28 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:28.843+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:29.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:29.200+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:29.647 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:29 smithi016 podman[113437]: 2024-09-15 22:50:29.307319583 +0000 UTC m=+5.444139321 container died 48b6a25f6d03c289ac309d3b97936ac4c22c06357a9b3d463b355b391ca8129d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T22:50:30.056 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:29 smithi016 podman[113437]: 2024-09-15 22:50:29.844964949 +0000 UTC m=+5.981784685 container cleanup 48b6a25f6d03c289ac309d3b97936ac4c22c06357a9b3d463b355b391ca8129d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-15T22:50:30.056 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:29 smithi016 bash[113437]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3 2024-09-15T22:50:30.188 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:29.847+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:30.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:29 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:29.849+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:30.306 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:30 smithi016 podman[113453]: 2024-09-15 22:50:30.055013429 +0000 UTC m=+0.746372335 container remove 48b6a25f6d03c289ac309d3b97936ac4c22c06357a9b3d463b355b391ca8129d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-15T22:50:30.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:30.187+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:30.618 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:30 smithi016 podman[113540]: 2024-09-15 22:50:30.375455171 +0000 UTC m=+0.016500922 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:50:30.619 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:30 smithi016 podman[113540]: 2024-09-15 22:50:30.4893881 +0000 UTC m=+0.130433848 container create d9355c5940e99492e2da0a6eaeb0ec004d14dc0b0677e10c1297a123231ff4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:50:30.619 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:30 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:30.322+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:30.942 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:30 smithi016 podman[113540]: 2024-09-15 22:50:30.742099143 +0000 UTC m=+0.383144899 container init d9355c5940e99492e2da0a6eaeb0ec004d14dc0b0677e10c1297a123231ff4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:50:30.942 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:30 smithi016 podman[113540]: 2024-09-15 22:50:30.745808 +0000 UTC m=+0.386853754 container start d9355c5940e99492e2da0a6eaeb0ec004d14dc0b0677e10c1297a123231ff4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:50:30.942 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:30 smithi016 podman[113540]: 2024-09-15 22:50:30.833580302 +0000 UTC m=+0.474626059 container attach d9355c5940e99492e2da0a6eaeb0ec004d14dc0b0677e10c1297a123231ff4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:50:31.166 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:30.849+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:31.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:30 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:30.883+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:31.306 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 15 22:50:30 smithi016 podman[113540]: 2024-09-15 22:50:30.942474166 +0000 UTC m=+0.583519918 container died d9355c5940e99492e2da0a6eaeb0ec004d14dc0b0677e10c1297a123231ff4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-15T22:50:31.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:31.164+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:31.696 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:31 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:31.360+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:31.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.3.service' 2024-09-15T22:50:31.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:50:31.772 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-15T22:50:31.772 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-15T22:50:31.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.4 2024-09-15T22:50:32.111 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:31 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:31.864+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:32.111 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:31 smithi059 systemd[1]: Stopping Ceph osd.4 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:50:32.202 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:31.852+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:32.444 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:32 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4[46256]: 2024-09-15T22:50:32.109+0000 7fd43878f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:50:32.444 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:32 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4[46256]: 2024-09-15T22:50:32.109+0000 7fd43878f640 -1 osd.4 104 *** Got signal Terminated *** 2024-09-15T22:50:32.444 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:32 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4[46256]: 2024-09-15T22:50:32.109+0000 7fd43878f640 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:32.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:32.201+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:32.806 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:32 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:32.360+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:33.155 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:32.829+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:33.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:32 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:32.887+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:33.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:33 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:33.154+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:33.806 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:33 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:33.366+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:34.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:33 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:33.828+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:34.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:33 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:33.664+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:34.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:33 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:33.909+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:34.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:34 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:34.159+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:34.806 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:34 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:34.409+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:35.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:34 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:34.673+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:35.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:34 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:34.779+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:35.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:34 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:34.881+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:35.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:35 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:35.195+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:35.806 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:35 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:35.429+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:36.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:35 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:35.676+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:36.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:35 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:35.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:36.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:35 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:35.845+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:36.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:36 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:36.217+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:36.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:36 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:36.217+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:36.806 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:36 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:36.478+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:37.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:36 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:36.661+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:37.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:36 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:36.757+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:37.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:36 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:36.845+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:37.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:36 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:36.845+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:37.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:37 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:37.214+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:37.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:37 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:37.214+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:37.702 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:37 smithi059 podman[71631]: 2024-09-15 22:50:37.442596432 +0000 UTC m=+5.466110369 container died e2d27bc2b9853c8769f469fc52fc01a053c905d6e6d2e7675295c735e0a6d247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-15T22:50:37.702 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:37 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:37.338+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:37.808 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:37 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:37.501+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:37.808 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:37 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:37.501+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:38.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:37 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:37.630+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:38.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:37 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:37.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:38.096 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:37 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:37.862+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:38.096 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:37 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:37.862+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:38.096 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:37 smithi059 podman[71631]: 2024-09-15 22:50:37.885500711 +0000 UTC m=+5.909014636 container cleanup e2d27bc2b9853c8769f469fc52fc01a053c905d6e6d2e7675295c735e0a6d247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T22:50:38.096 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:37 smithi059 bash[71631]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4 2024-09-15T22:50:38.444 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:38 smithi059 podman[71648]: 2024-09-15 22:50:38.095133148 +0000 UTC m=+0.652630361 container remove e2d27bc2b9853c8769f469fc52fc01a053c905d6e6d2e7675295c735e0a6d247 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T22:50:38.444 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:38 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:38.327+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:38.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:38 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:38.247+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:38.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:38 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:38.247+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:38.547 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:38 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:38.072+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:38.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:38 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:38.546+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:38.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:38 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:38.546+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:38.807 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:38 smithi059 podman[71733]: 2024-09-15 22:50:38.40607559 +0000 UTC m=+0.015835951 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:50:38.808 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:38 smithi059 podman[71733]: 2024-09-15 22:50:38.537737054 +0000 UTC m=+0.147497451 container create db6648a06c77f95aaca4d633d9cb4020dd8fe60c74738d0bd6956a462e570d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-15T22:50:38.808 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:38 smithi059 podman[71733]: 2024-09-15 22:50:38.806884484 +0000 UTC m=+0.416644842 container init db6648a06c77f95aaca4d633d9cb4020dd8fe60c74738d0bd6956a462e570d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-15T22:50:39.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:38 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:38.639+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:39.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:38 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:38.742+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:39.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:38 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:38.742+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:39.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:38 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:38.878+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:39.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:38 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:38.878+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:39.194 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:38 smithi059 podman[71733]: 2024-09-15 22:50:38.810384635 +0000 UTC m=+0.420144992 container start db6648a06c77f95aaca4d633d9cb4020dd8fe60c74738d0bd6956a462e570d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-15T22:50:39.194 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:38 smithi059 podman[71733]: 2024-09-15 22:50:38.890090669 +0000 UTC m=+0.499851042 container attach db6648a06c77f95aaca4d633d9cb4020dd8fe60c74738d0bd6956a462e570d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:50:39.194 INFO:journalctl@ceph.osd.4.smithi059.stdout:Sep 15 22:50:39 smithi059 podman[71733]: 2024-09-15 22:50:39.002226088 +0000 UTC m=+0.611986448 container died db6648a06c77f95aaca4d633d9cb4020dd8fe60c74738d0bd6956a462e570d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-15T22:50:39.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:39 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:39.042+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:39.685 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:39 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:39.292+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:39.685 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:39 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:39.292+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:39.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:39 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:39.321+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:39.782 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.4.service' 2024-09-15T22:50:39.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:50:39.828 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-15T22:50:39.829 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-15T22:50:39.829 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.5 2024-09-15T22:50:40.029 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:39 smithi130 systemd[1]: Stopping Ceph osd.5 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:50:40.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:39 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:39.684+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:40.030 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:39 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:39.698+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:40.030 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:39 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:39.698+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:40.043 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:39 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:39.586+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:40.043 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:39 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:39.586+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:40.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:39 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:39.922+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:40.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:39 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:39.922+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:40.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:40 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:40.042+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:40.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:40.153+0000 7f142b77d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:50:40.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:40.153+0000 7f142b77d640 -1 osd.5 104 *** Got signal Terminated *** 2024-09-15T22:50:40.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:40.153+0000 7f142b77d640 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:40.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:40.248+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:40.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:40.248+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:40.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:40 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:40.320+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:41.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:40.678+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:41.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:40.676+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:41.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:40 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:40.676+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:41.039 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:40 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:40.593+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:41.039 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:40 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:40.593+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:41.304 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:40 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:40.972+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:41.305 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:40 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:40.972+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:41.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:41.038+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:41.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:41 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:41.237+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:41.529 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:41 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:41.237+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:41.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:41 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:41.303+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:42.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:41 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:41.646+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:42.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:41 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:41.659+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:42.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:41 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:41.659+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:42.034 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:41.590+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:42.034 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:41.590+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:42.269 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:41 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:41.960+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:42.269 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:41 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:41.960+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:42.269 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:41 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:41.960+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:42.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:42.033+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:42.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:42.251+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:42.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:42.251+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:42.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:42.266+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:43.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:42.636+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:43.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:42 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:42.636+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:43.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:42.657+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:43.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:42.696+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:43.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:42 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:42.696+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:43.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:42.939+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:43.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:42.939+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:43.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:42 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:42.939+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:43.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:43.014+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:43.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:43.014+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:43.682 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:43.286+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:43.682 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:43.286+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:43.682 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:43.286+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.046648+0000 front 2024-09-15T22:50:19.046670+0000 (oldest deadline 2024-09-15T22:50:42.546800+0000) 2024-09-15T22:50:43.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:43 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:43.315+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:43.968 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:43.648+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:43.969 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:43.648+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:44.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:43.680+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:44.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:43.680+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:44.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:43.718+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:44.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:43 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:43.718+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:44.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:43 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:43.926+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:44.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:43 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:43.926+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:44.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:43 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:43.926+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:44.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:43.968+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:44.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:43 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:43.968+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:44.657 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:44.318+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.844096+0000 front 2024-09-15T22:50:02.843949+0000 (oldest deadline 2024-09-15T22:50:28.143704+0000) 2024-09-15T22:50:44.657 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:44.318+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.745384+0000 front 2024-09-15T22:50:12.745428+0000 (oldest deadline 2024-09-15T22:50:35.645314+0000) 2024-09-15T22:50:44.658 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5[47627]: 2024-09-15T22:50:44.318+0000 7f1427d93640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.046648+0000 front 2024-09-15T22:50:19.046670+0000 (oldest deadline 2024-09-15T22:50:42.546800+0000) 2024-09-15T22:50:44.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:44 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:44.299+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:44.943 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:44.645+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:44.943 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:44.645+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:44.943 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:44.645+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:45.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:44.656+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:45.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:44.656+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:45.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:44.676+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:45.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:44.676+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:45.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:44 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:44.676+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:45.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:44 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:44.917+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:45.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:44 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:44.917+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:45.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:44 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:44.917+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:45.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:44.942+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:45.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:44 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:44.942+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:45.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:45 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:45.308+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:45.821 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:45 smithi130 podman[72353]: 2024-09-15 22:50:45.547439609 +0000 UTC m=+5.516166175 container died 5a8f98d84e1f635cd72e930ea1fbdb84d7fddd89693ab96c5436344bed660cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T22:50:45.822 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:45 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:45.644+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:45.822 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:45 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:45.644+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:45.822 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:45 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:45.714+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:45.822 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:45 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:45.714+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:45.822 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:45 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:45.714+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:45.923 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:45.618+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:45.923 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:45.618+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:45.923 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:45.618+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:46.189 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:45 smithi130 podman[72353]: 2024-09-15 22:50:45.920474085 +0000 UTC m=+5.889200641 container cleanup 5a8f98d84e1f635cd72e930ea1fbdb84d7fddd89693ab96c5436344bed660cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-15T22:50:46.189 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:45 smithi130 bash[72353]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5 2024-09-15T22:50:46.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:45 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:45.883+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:46.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:45 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:45.883+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:46.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:45 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:45.883+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:46.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:45.922+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:46.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:45.922+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:46.528 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:46 smithi130 podman[72371]: 2024-09-15 22:50:46.188545486 +0000 UTC m=+0.638093921 container remove 5a8f98d84e1f635cd72e930ea1fbdb84d7fddd89693ab96c5436344bed660cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T22:50:46.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:46 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:46.303+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:46.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:46 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:46.303+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:46.825 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:46 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:46.668+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:46.825 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:46 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:46.668+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:46.826 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:46 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:46.759+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:46.826 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:46 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:46.759+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:46.826 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:46 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:46.759+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:46.826 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:46 smithi130 podman[72459]: 2024-09-15 22:50:46.431589706 +0000 UTC m=+0.015798458 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:50:46.826 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:46 smithi130 podman[72459]: 2024-09-15 22:50:46.564531812 +0000 UTC m=+0.148740560 container create bfad7be0ad2764f26074b16c6a7eccc71b0a5b4f9b0f9b31334150e63bd258e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-15T22:50:46.826 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:46 smithi130 podman[72459]: 2024-09-15 22:50:46.825319335 +0000 UTC m=+0.409528080 container init bfad7be0ad2764f26074b16c6a7eccc71b0a5b4f9b0f9b31334150e63bd258e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-15T22:50:46.950 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:46 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:46.668+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:46.951 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:46 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:46.668+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:46.951 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:46 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:46.668+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:47.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:46 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:46.853+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:47.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:46 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:46.853+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:47.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:46 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:46.853+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:47.278 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:46 smithi130 podman[72459]: 2024-09-15 22:50:46.828583909 +0000 UTC m=+0.412792658 container start bfad7be0ad2764f26074b16c6a7eccc71b0a5b4f9b0f9b31334150e63bd258e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-15T22:50:47.279 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:46 smithi130 podman[72459]: 2024-09-15 22:50:46.933767147 +0000 UTC m=+0.517975902 container attach bfad7be0ad2764f26074b16c6a7eccc71b0a5b4f9b0f9b31334150e63bd258e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-15T22:50:47.279 INFO:journalctl@ceph.osd.5.smithi130.stdout:Sep 15 22:50:47 smithi130 podman[72459]: 2024-09-15 22:50:47.020410594 +0000 UTC m=+0.604619348 container died bfad7be0ad2764f26074b16c6a7eccc71b0a5b4f9b0f9b31334150e63bd258e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-15T22:50:47.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:46 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:46.949+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:47.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:46 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:46.949+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:47.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:47 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:47.259+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:47.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:47 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:47.259+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:47.700 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.5.service' 2024-09-15T22:50:47.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:50:47.739 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-15T22:50:47.739 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-15T22:50:47.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.6 2024-09-15T22:50:47.985 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:47 smithi016 systemd[1]: Stopping Ceph osd.6 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:50:47.985 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:47 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:47.640+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:47.985 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:47 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:47.640+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:47.985 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:47 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:47.640+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:48.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:47 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:47.649+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:48.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:47 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:47.649+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:48.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:47 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:47.725+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:48.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:47 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:47.725+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:48.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:47 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:47.725+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:48.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:47 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:47.814+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:48.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:47 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:47.814+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:48.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:47 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:47.814+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:48.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:47 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:47.984+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:48.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:47 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:47.984+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:48.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:48 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:48.073+0000 7f76be062640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:50:48.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:48 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:48.073+0000 7f76be062640 -1 osd.6 104 *** Got signal Terminated *** 2024-09-15T22:50:48.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:48 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:48.073+0000 7f76be062640 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:48.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:48 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:48.299+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:48.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:48 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:48.299+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:49.017 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:48 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:48.600+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:49.017 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:48 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:48.600+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:49.017 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:48 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:48.600+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:49.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:48 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:48.608+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:49.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:48 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:48.608+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:49.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:48 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:48.775+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:49.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:48 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:48.775+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:49.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:48 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:48.775+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:49.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:48 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:48.856+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:49.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:48 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:48.856+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:49.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:48 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:48.856+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:49.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:49.015+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:49.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:49.015+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:49.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:49 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:49.332+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:49.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:49 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:49.332+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:49.993 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:49.647+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:49.994 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:49.647+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:49.994 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:49.647+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:50.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:49 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:49.582+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:50.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:49 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:49.582+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:50.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:49 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:49.775+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:50.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:49 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:49.775+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:50.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:49 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:49.775+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:50.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:49 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:49.881+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:50.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:49 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:49.881+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:50.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:49 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:49.881+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:50.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:49.992+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:50.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:49.992+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:50.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:50 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:50.292+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:50.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:50 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:50.292+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:50.959 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:50 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:50.638+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:50.959 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:50 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:50.638+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:50.959 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:50 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:50.638+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:51.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:50 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:50.558+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:51.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:50 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:50.558+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:51.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:50 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:50.558+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:51.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:50 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:50.805+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:51.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:50 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:50.805+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:51.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:50 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:50.805+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:51.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:50 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:50.862+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:51.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:50 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:50.862+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:51.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:50 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:50.862+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:51.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:50 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:50.862+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:51.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:50 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:50.958+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:51.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:50 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:50.958+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:51.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:51.306+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:51.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:51.306+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:51.778 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:51 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:51.520+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:51.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:51 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:51.520+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:51.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:51 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:51.520+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:51.938 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:51.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:51.939 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:51.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:51.939 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:51.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:52.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:51.880+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:52.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:51.880+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:52.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:51.880+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:52.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:51 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:51.880+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:52.278 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:51 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:51.826+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:52.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:51 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:51.826+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:52.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:51 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:51.826+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:52.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:51.938+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:52.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:51.938+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:52.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:51 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:51.938+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114465+0000 front 2024-09-15T22:50:26.114453+0000 (oldest deadline 2024-09-15T22:50:51.314758+0000) 2024-09-15T22:50:52.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:52.333+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:52.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:52.333+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:52.801 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:52 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:52.549+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:52.801 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:52 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:52.549+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:52.801 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:52 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:52.549+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:52.945 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:52.606+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:52.945 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:52.606+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:52.945 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:52.606+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:52.945 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:52.606+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:50:53.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:52.852+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:53.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:52.852+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:53.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:52.852+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:53.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:52 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:52.852+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:53.278 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:52 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:52.800+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:53.278 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:52 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:52.800+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:53.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:52 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:52.800+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:53.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:52.944+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.712513+0000 front 2024-09-15T22:50:12.712657+0000 (oldest deadline 2024-09-15T22:50:38.012423+0000) 2024-09-15T22:50:53.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:52.944+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.713358+0000 front 2024-09-15T22:50:19.713408+0000 (oldest deadline 2024-09-15T22:50:42.613359+0000) 2024-09-15T22:50:53.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:52 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6[58299]: 2024-09-15T22:50:52.944+0000 7f76b6670640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114465+0000 front 2024-09-15T22:50:26.114453+0000 (oldest deadline 2024-09-15T22:50:51.314758+0000) 2024-09-15T22:50:53.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:53.361+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:53.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:53.361+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:53.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:53.361+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:50:53.807 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:53 smithi016 podman[113642]: 2024-09-15 22:50:53.539504461 +0000 UTC m=+5.590470340 container died 841debfe2272d0fa3f0a399ee00893fd84133ec30f9c626ff953dd18ed0b7607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-15T22:50:53.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:53 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:53.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:53.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:53 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:53.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:53.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:53 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:53.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:53.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:53 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:53.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:50:54.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:53 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:53.547+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:54.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:53 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:53.547+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:54.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:53 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:53.547+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:54.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:53 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:53.752+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:54.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:53 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:53.752+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:54.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:53 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:53.752+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:54.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:53.864+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:54.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:53.864+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:54.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:53.864+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:54.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:53.864+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:54.225 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:53 smithi016 podman[113642]: 2024-09-15 22:50:53.956872549 +0000 UTC m=+6.007838460 container cleanup 841debfe2272d0fa3f0a399ee00893fd84133ec30f9c626ff953dd18ed0b7607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T22:50:54.225 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:53 smithi016 bash[113642]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6 2024-09-15T22:50:54.556 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:54 smithi016 podman[113660]: 2024-09-15 22:50:54.224929744 +0000 UTC m=+0.684193415 container remove 841debfe2272d0fa3f0a399ee00893fd84133ec30f9c626ff953dd18ed0b7607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-15T22:50:54.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:54 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:54.349+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:54.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:54 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:54.349+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:54.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:54 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:54.349+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:50:55.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:54.772+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:55.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:54.772+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:55.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:54.772+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:55.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:54.569+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:55.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:54.569+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:55.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:54 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:54.569+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:55.031 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:54 smithi016 podman[113749]: 2024-09-15 22:50:54.541273642 +0000 UTC m=+0.016335394 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:50:55.031 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:54 smithi016 podman[113749]: 2024-09-15 22:50:54.783932088 +0000 UTC m=+0.258993840 container create 6ecc911a231dc12327e59c391db0a50ba94f710d26b3063729e18269ec9a6846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:50:55.031 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:54 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:54.558+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:55.031 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:54 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:54.558+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:55.031 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:54 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:54.558+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:55.032 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:54 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:54.558+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:50:55.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:54 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:54.899+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:55.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:54 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:54.899+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:55.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:54 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:54.899+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:55.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:54 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:54.899+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:55.306 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:55 smithi016 podman[113749]: 2024-09-15 22:50:55.161424386 +0000 UTC m=+0.636486137 container init 6ecc911a231dc12327e59c391db0a50ba94f710d26b3063729e18269ec9a6846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:50:55.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:55 smithi016 podman[113749]: 2024-09-15 22:50:55.164967096 +0000 UTC m=+0.640028848 container start 6ecc911a231dc12327e59c391db0a50ba94f710d26b3063729e18269ec9a6846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:50:55.307 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:55 smithi016 podman[113749]: 2024-09-15 22:50:55.24499149 +0000 UTC m=+0.720053258 container attach 6ecc911a231dc12327e59c391db0a50ba94f710d26b3063729e18269ec9a6846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e) 2024-09-15T22:50:55.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:55 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:55.310+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:55.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:55 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:55.310+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:55.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:55 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:55.310+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:50:55.787 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 15 22:50:55 smithi016 podman[113749]: 2024-09-15 22:50:55.357690535 +0000 UTC m=+0.832752287 container died 6ecc911a231dc12327e59c391db0a50ba94f710d26b3063729e18269ec9a6846 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-15T22:50:55.787 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:55 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:55.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:55.787 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:55 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:55.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:55.787 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:55 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:55.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:55.788 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:55 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:55.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:50:56.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:55 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:55.754+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:56.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:55 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:55.754+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:56.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:55 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:55.754+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:56.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:55 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:55.588+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:56.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:55 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:55.588+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:56.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:55 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:55.588+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:56.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.6.service' 2024-09-15T22:50:56.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:50:56.149 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-15T22:50:56.150 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-15T22:50:56.150 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.7 2024-09-15T22:50:56.177 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:55 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:55.873+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:56.177 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:55 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:55.873+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:56.177 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:55 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:55.873+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:56.177 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:55 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:55.873+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:56.444 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:56 smithi059 systemd[1]: Stopping Ceph osd.7 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:50:56.444 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:56.308+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:56.444 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:56.308+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:56.444 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:56.308+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:50:56.904 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:56.495+0000 7fe21fcca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:50:56.904 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:56.495+0000 7fe21fcca640 -1 osd.7 104 *** Got signal Terminated *** 2024-09-15T22:50:56.904 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:56.495+0000 7fe21fcca640 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:50:57.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:56 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:56.544+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:57.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:56 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:56.544+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:57.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:56 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:56.544+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:57.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:56 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:56.785+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:57.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:56 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:56.785+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:57.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:56 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:56.785+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:57.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:56 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:56.607+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:57.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:56 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:56.607+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:57.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:56 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:56.607+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:57.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:56 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:56.607+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:50:57.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:56.902+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:57.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:56.902+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:57.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:56.902+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:57.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:56 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:56.902+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:57.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:57.291+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:57.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:57.291+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:57.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:57.291+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:50:58.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:57 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:57.574+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:58.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:57 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:57.574+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:58.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:57 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:57.574+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:58.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:57 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:57.800+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:58.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:57 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:57.800+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:58.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:57 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:57.800+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:58.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:57 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:57.638+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:58.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:57 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:57.638+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:58.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:57 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:57.638+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:58.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:57 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:57.638+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:50:58.196 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:57.891+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:58.196 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:57.891+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:58.196 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:57.891+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:58.196 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:57.891+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:58.197 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:57 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:57.891+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:50:58.693 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:58.269+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:58.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:58.269+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:58.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:58.269+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:50:58.778 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:58.525+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:58.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:58.525+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:58.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:58.525+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:50:58.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:58.769+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:50:58.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:58.769+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:50:58.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:58.769+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:50:58.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:58 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:58.769+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:50:59.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:58 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:58.615+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:50:59.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:58 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:58.615+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:50:59.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:58 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:58.615+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:50:59.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:58 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:58.615+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:50:59.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:58.841+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:50:59.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:58.841+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:50:59.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:58.841+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:50:59.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:58.841+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:50:59.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:58.841+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:50:59.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:59.236+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:50:59.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:59.236+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:50:59.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:50:59.236+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:50:59.766 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:59 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:59.505+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:50:59.767 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:59 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:59.505+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:50:59.767 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:50:59 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:50:59.505+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:00.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:59 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:59.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:00.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:59 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:59.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:00.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:59 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:59.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:00.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:50:59 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:50:59.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:00.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:59 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:59.603+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:00.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:59 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:59.603+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:00.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:59 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:59.603+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:00.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:50:59 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:50:59.603+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:00.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:59.793+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:00.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:59.793+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:00.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:59.793+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:00.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:59.793+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:00.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:50:59 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:50:59.793+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:00.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:00.281+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:51:00.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:00.281+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:51:00.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:00.281+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:51:00.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:00.281+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.875339+0000 front 2024-09-15T22:50:35.875316+0000 (oldest deadline 2024-09-15T22:50:59.975635+0000) 2024-09-15T22:51:00.749 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:00.465+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:00.750 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:00.465+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:00.750 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:00.465+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:01.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:00.748+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:01.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:00.748+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:01.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:00.748+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:01.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:00 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:00.748+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:01.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:00.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:01.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:00.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:01.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:00.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:01.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:00 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:00.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:01.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:00.785+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:01.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:00.785+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:01.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:00.785+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:01.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:00.785+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:01.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:00 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:00.785+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:01.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:01.310+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.371718+0000 front 2024-09-15T22:50:11.371934+0000 (oldest deadline 2024-09-15T22:50:36.671925+0000) 2024-09-15T22:51:01.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:01.311+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.573105+0000 front 2024-09-15T22:50:19.573044+0000 (oldest deadline 2024-09-15T22:50:45.473076+0000) 2024-09-15T22:51:01.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:01.311+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:27.174072+0000 front 2024-09-15T22:50:27.174126+0000 (oldest deadline 2024-09-15T22:50:52.474033+0000) 2024-09-15T22:51:01.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7[52943]: 2024-09-15T22:51:01.311+0000 7fe2182d8640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.875339+0000 front 2024-09-15T22:50:35.875316+0000 (oldest deadline 2024-09-15T22:50:59.975635+0000) 2024-09-15T22:51:01.746 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:01.480+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:01.747 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:01.480+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:01.747 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:01.480+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:01.747 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:01.480+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:02.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:01.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:02.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:01.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:02.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:01.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:02.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:01 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:01.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:02.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:01 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:01.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:02.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:01 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:01.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:02.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:01 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:01.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:02.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:01 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:01.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:02.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:01 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:01.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:02.194 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:01 smithi059 podman[71836]: 2024-09-15 22:51:01.95430709 +0000 UTC m=+5.594276521 container died 5b647a19f940e2a816184d4b80f088062c81d380ddf0528a2d505fe343311bcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-15T22:51:02.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:01.779+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:02.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:01.779+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:02.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:01.779+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:02.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:01.779+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:02.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:01 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:01.779+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:02.623 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:02 smithi059 podman[71836]: 2024-09-15 22:51:02.321918702 +0000 UTC m=+5.961888132 container cleanup 5b647a19f940e2a816184d4b80f088062c81d380ddf0528a2d505fe343311bcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True) 2024-09-15T22:51:02.623 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:02 smithi059 bash[71836]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7 2024-09-15T22:51:02.778 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:02.730+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:02.778 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:02.730+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:02.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:02.730+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:02.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:02.730+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:02.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:02.509+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:02.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:02.509+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:02.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:02.509+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:02.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:02 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:02.509+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:02.937 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:02 smithi059 podman[71863]: 2024-09-15 22:51:02.623106616 +0000 UTC m=+0.662166324 container remove 5b647a19f940e2a816184d4b80f088062c81d380ddf0528a2d505fe343311bcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-15T22:51:02.937 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:02 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:02.738+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:02.938 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:02 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:02.738+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:02.938 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:02 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:02.738+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:02.938 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:02 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:02.738+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:02.938 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:02 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:02.738+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:03.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:02 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:02.634+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:03.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:02 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:02.634+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:03.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:02 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:02.634+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:03.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:02 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:02.634+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:03.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:02 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:02.634+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:03.193 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:02 smithi059 podman[71950]: 2024-09-15 22:51:02.839550785 +0000 UTC m=+0.015998538 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:51:03.194 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:02 smithi059 podman[71950]: 2024-09-15 22:51:02.949125407 +0000 UTC m=+0.125573149 container create b4fb6d2b50abb78f16d879e519591676dbbaf9de535b8ad991a780ce5f3866b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-15T22:51:03.194 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:03 smithi059 podman[71950]: 2024-09-15 22:51:03.176646954 +0000 UTC m=+0.353094696 container init b4fb6d2b50abb78f16d879e519591676dbbaf9de535b8ad991a780ce5f3866b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-15T22:51:03.194 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:03 smithi059 podman[71950]: 2024-09-15 22:51:03.179899253 +0000 UTC m=+0.356346995 container start b4fb6d2b50abb78f16d879e519591676dbbaf9de535b8ad991a780ce5f3866b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:51:03.694 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:03 smithi059 podman[71950]: 2024-09-15 22:51:03.276668607 +0000 UTC m=+0.453116352 container attach b4fb6d2b50abb78f16d879e519591676dbbaf9de535b8ad991a780ce5f3866b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T22:51:04.064 INFO:journalctl@ceph.osd.7.smithi059.stdout:Sep 15 22:51:03 smithi059 podman[71950]: 2024-09-15 22:51:03.36984921 +0000 UTC m=+0.546296947 container died b4fb6d2b50abb78f16d879e519591676dbbaf9de535b8ad991a780ce5f3866b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:03 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:03.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:03 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:03.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:03 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:03.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:03 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:03.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:03 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:03.626+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:03.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:03.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:04.065 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:03.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:03.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:03.715+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:03.715+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:03.715+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:03.715+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:03.715+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:03.527+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:03.527+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:03.527+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:04.066 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:03 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:03.527+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:04.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.7.service' 2024-09-15T22:51:04.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:04.149 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-15T22:51:04.149 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-15T22:51:04.149 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.8 2024-09-15T22:51:04.466 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 systemd[1]: Stopping Ceph osd.8 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:04.735 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:04.464+0000 7f4c55386640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:04.735 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:04.464+0000 7f4c55386640 -1 osd.8 104 *** Got signal Terminated *** 2024-09-15T22:51:04.735 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:04.465+0000 7f4c55386640 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:51:04.736 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:04.487+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:04.736 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:04.487+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:04.736 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:04.487+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:04.736 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:04.487+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:04.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:04 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:04.679+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:04.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:04 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:04.679+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:04.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:04 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:04.679+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:04.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:04 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:04.679+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:04.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:04 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:04.679+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:05.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:04.734+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:05.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:04.734+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:05.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:04.734+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:05.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:04 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:04.734+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:05.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:04 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:04.580+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:05.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:04 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:04.580+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:05.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:04 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:04.580+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:05.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:04 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:04.580+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:05.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:04 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:04.580+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:06.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:05.531+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:06.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:05.531+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:06.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:05.531+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:06.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:05.531+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:06.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:05.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:06.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:05.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:06.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:05.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:06.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:05 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:05.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:06.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:05.614+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:06.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:05.614+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:06.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:05.614+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:06.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:05.614+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:06.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:05.614+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:06.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:05 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:05.614+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:06.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:05 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:05.701+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:06.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:05 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:05.701+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:06.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:05 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:05.701+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:06.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:05 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:05.701+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:06.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:05 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:05.701+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:06.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:06.500+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:06.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:06.500+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:06.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:06.500+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:06.780 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:06.500+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:06.780 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:06.702+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:06.780 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:06.702+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:06.781 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:06.702+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:06.781 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:06.702+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:06.781 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:06 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:06.702+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:07.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:06.592+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:07.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:06.592+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:07.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:06.592+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:07.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:06.592+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:07.058 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:06.592+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:07.058 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:06 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:06.592+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:07.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:06 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:06.729+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:07.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:06 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:06.729+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:07.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:06 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:06.729+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:07.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:06 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:06.729+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:07.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:06 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:06.729+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:07.778 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:07.511+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:07.511+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:07.511+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:07.511+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:07.511+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.216406+0000 front 2024-09-15T22:50:41.216378+0000 (oldest deadline 2024-09-15T22:51:06.516455+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:07.722+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:07.722+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:07.722+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:07.722+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:07.779 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:07 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:07.722+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:07.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:07.687+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:07.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:07.687+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:07.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:07.687+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:07.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:07.687+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:07.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:07.687+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:08.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:07.588+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:08.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:07.588+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:08.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:07.588+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:08.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:07.588+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:08.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:07.588+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:08.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:07 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:07.588+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:09.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:08.537+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:11.110584+0000 front 2024-09-15T22:50:11.110655+0000 (oldest deadline 2024-09-15T22:50:33.410689+0000) 2024-09-15T22:51:09.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:08.537+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:19.212263+0000 front 2024-09-15T22:50:19.212289+0000 (oldest deadline 2024-09-15T22:50:42.712420+0000) 2024-09-15T22:51:09.029 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:08.537+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.114095+0000 front 2024-09-15T22:50:26.114131+0000 (oldest deadline 2024-09-15T22:50:50.214065+0000) 2024-09-15T22:51:09.030 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:08.537+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:36.615823+0000 front 2024-09-15T22:50:36.615938+0000 (oldest deadline 2024-09-15T22:51:01.215990+0000) 2024-09-15T22:51:09.030 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8[54102]: 2024-09-15T22:51:08.537+0000 7f4c4d994640 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.216406+0000 front 2024-09-15T22:50:41.216378+0000 (oldest deadline 2024-09-15T22:51:06.516455+0000) 2024-09-15T22:51:09.030 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:08.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:09.030 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:08.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:09.031 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:08.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:09.031 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:08.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:09.031 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:08 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:08.765+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:09.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:08.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:09.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:08.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:09.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:08.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:09.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:08.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:09.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:08.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:09.058 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:08 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:08.595+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:09.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:08 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:08.728+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:09.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:08 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:08.728+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:09.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:08 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:08.728+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:09.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:08 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:08.728+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:09.195 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:08 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:08.728+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:09.195 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:08 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:08.728+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:10.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:09 smithi130 podman[72578]: 2024-09-15 22:51:09.797739682 +0000 UTC m=+5.454965822 container died d0e784dd317a42617ab45fc7968af86ef3c6a9a5cd449abbe269d1ba6579ab3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-15T22:51:10.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:09.768+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:10.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:09.768+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:10.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:09.768+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:10.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:09.768+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:10.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:09 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:09.768+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:10.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:09.575+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:10.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:09.575+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:10.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:09.575+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:10.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:09.575+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:10.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:09.575+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:10.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:09 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:09.575+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:10.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:09 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:09.713+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:10.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:09 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:09.713+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:10.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:09 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:09.713+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:10.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:09 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:09.713+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:10.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:09 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:09.713+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:10.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:09 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:09.713+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:10.459 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:10 smithi130 podman[72578]: 2024-09-15 22:51:10.190427382 +0000 UTC m=+5.847653519 container cleanup d0e784dd317a42617ab45fc7968af86ef3c6a9a5cd449abbe269d1ba6579ab3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-15T22:51:10.459 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:10 smithi130 bash[72578]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8 2024-09-15T22:51:10.746 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:10 smithi130 podman[72595]: 2024-09-15 22:51:10.458735872 +0000 UTC m=+0.657916541 container remove d0e784dd317a42617ab45fc7968af86ef3c6a9a5cd449abbe269d1ba6579ab3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:51:10.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:10.691+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:10.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:10.691+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:10.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:10.691+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:10.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:10.691+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:10.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:10.691+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:10.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:10 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:10.691+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:11.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:10 smithi130 podman[72686]: 2024-09-15 22:51:10.705548636 +0000 UTC m=+0.015465747 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:51:11.028 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:10 smithi130 podman[72686]: 2024-09-15 22:51:10.834815247 +0000 UTC m=+0.144732347 container create 3d4c144e57d7f6fb03e22521912cb1b8073e1877bba2146dd490bb7adca3dead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-15T22:51:11.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:10.744+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:11.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:10.744+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:11.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:10.744+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:11.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:10.744+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:11.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:10.744+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:11.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:10 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:10.610+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:11.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:10 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:10.610+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:11.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:10 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:10.610+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:11.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:10 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:10.610+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:11.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:10 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:10.610+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:11.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:10 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:10.610+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:11.529 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:11 smithi130 podman[72686]: 2024-09-15 22:51:11.062344038 +0000 UTC m=+0.372261135 container init 3d4c144e57d7f6fb03e22521912cb1b8073e1877bba2146dd490bb7adca3dead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-15T22:51:11.529 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:11 smithi130 podman[72686]: 2024-09-15 22:51:11.065633939 +0000 UTC m=+0.375551042 container start 3d4c144e57d7f6fb03e22521912cb1b8073e1877bba2146dd490bb7adca3dead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-15T22:51:11.529 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:11 smithi130 podman[72686]: 2024-09-15 22:51:11.162332972 +0000 UTC m=+0.472250072 container attach 3d4c144e57d7f6fb03e22521912cb1b8073e1877bba2146dd490bb7adca3dead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-15T22:51:11.529 INFO:journalctl@ceph.osd.8.smithi130.stdout:Sep 15 22:51:11 smithi130 podman[72686]: 2024-09-15 22:51:11.251393584 +0000 UTC m=+0.561310696 container died 3d4c144e57d7f6fb03e22521912cb1b8073e1877bba2146dd490bb7adca3dead (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T22:51:11.900 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:11.717+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:11.900 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:11.717+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:11.900 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:11.717+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:11.901 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:11.717+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:11.901 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:11 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:11.717+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:11.920 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.8.service' 2024-09-15T22:51:11.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:11.664+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:11.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:11.664+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:11.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:11.664+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:11.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:11.664+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:11.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:11.664+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:11.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:11 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:11.664+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:11.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:11.958 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-15T22:51:11.958 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-15T22:51:11.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.9 2024-09-15T22:51:11.986 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:11 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:11.649+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:11.986 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:11 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:11.649+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:11.986 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:11 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:11.649+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:11.986 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:11 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:11.649+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:11.986 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:11 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:11.649+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:11.986 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:11 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:11.649+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:12.277 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 systemd[1]: Stopping Ceph osd.9 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:12.556 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.276+0000 7fd0e2a01640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:12.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.276+0000 7fd0e2a01640 -1 osd.9 104 *** Got signal Terminated *** 2024-09-15T22:51:12.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.276+0000 7fd0e2a01640 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:51:12.943 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:12.649+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:12.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:12.649+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:12.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:12.649+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:12.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:12.649+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:12.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:12.649+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:12.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:12 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:12.649+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:13.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:12.682+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:13.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:12.682+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:13.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:12.682+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:13.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:12.682+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:13.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:12 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:12.682+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:13.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.620+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:13.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.620+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:13.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.620+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:13.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.620+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:13.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.620+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:13.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:12 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:12.620+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:13.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:13.626+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:13.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:13.626+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:13.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:13.626+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:13.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:13.626+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:13.945 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:13.626+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:13.945 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:13 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:13.626+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:14.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:13 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:13.680+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:14.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:13 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:13.680+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:14.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:13 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:13.680+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:14.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:13 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:13.680+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:14.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:13 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:13.680+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:14.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:13.616+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:14.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:13.616+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:14.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:13.616+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:14.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:13.616+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:14.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:13.616+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:14.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:13 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:13.616+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:14.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:14.595+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:14.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:14.595+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:14.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:14.595+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:14.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:14.595+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:14.945 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:14.595+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:14.945 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:14 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:14.595+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:15.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:14 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:14.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:15.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:14 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:14.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:15.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:14 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:14.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:15.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:14 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:14.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:15.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:14 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:14.703+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:15.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:14 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:14.635+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:15.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:14 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:14.635+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:15.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:14 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:14.635+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:15.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:14 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:14.635+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:15.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:14 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:14.635+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:15.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:14 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:14.635+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:15.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:15.619+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:15.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:15.619+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:15.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:15.619+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:15.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:15.619+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:15.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:15.619+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:15.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:15.620+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:15.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:15 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:15.620+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:16.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:15.699+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:16.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:15.699+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:16.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:15.699+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:16.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:15.699+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:16.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:15.699+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:16.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:15.699+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:16.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:15 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:15.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:16.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:15 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:15.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:16.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:15 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:15.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:16.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:15 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:15.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:16.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:15 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:15.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:16.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:15 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:15.651+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:16.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:16 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:16.596+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:16.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:16 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:16.596+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:16.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:16 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:16.596+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:16.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:16 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:16.596+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:16.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:16 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:16.596+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:16.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:16 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:16.596+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:16.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:16 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:16.596+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:17.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:16.716+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:17.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:16.716+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:17.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:16.716+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:17.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:16.716+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:17.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:16.716+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:17.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:16 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:16.716+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:17.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:16 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:16.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:04.081324+0000 front 2024-09-15T22:50:04.081387+0000 (oldest deadline 2024-09-15T22:50:29.981545+0000) 2024-09-15T22:51:17.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:16 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:16.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.183396+0000 front 2024-09-15T22:50:12.183589+0000 (oldest deadline 2024-09-15T22:50:36.883323+0000) 2024-09-15T22:51:17.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:16 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:16.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:17.984432+0000 front 2024-09-15T22:50:17.984547+0000 (oldest deadline 2024-09-15T22:50:43.884452+0000) 2024-09-15T22:51:17.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:16 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:16.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:26.185481+0000 front 2024-09-15T22:50:26.185494+0000 (oldest deadline 2024-09-15T22:50:52.085779+0000) 2024-09-15T22:51:17.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:16 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:16.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:35.586932+0000 front 2024-09-15T22:50:35.586832+0000 (oldest deadline 2024-09-15T22:51:01.486689+0000) 2024-09-15T22:51:17.057 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:16 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9[65279]: 2024-09-15T22:51:16.623+0000 7fd0db00f640 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:41.487182+0000 front 2024-09-15T22:50:41.487232+0000 (oldest deadline 2024-09-15T22:51:05.587165+0000) 2024-09-15T22:51:17.943 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:17.611+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:17.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:17.611+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:17.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:17.611+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:17.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:17.611+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:17.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:17.611+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:17.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:17.611+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:17.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:17 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:17.611+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:18.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:17 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:17.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:18.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:17 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:17.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:18.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:17 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:17.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:18.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:17 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:17.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:18.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:17 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:17.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:18.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:17 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:17.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:18.056 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:17 smithi016 podman[113852]: 2024-09-15 22:51:17.766783042 +0000 UTC m=+5.603326592 container died b17d9657ce21ba07981da914c0d756c2e3087b072f16a6cbedd1522333caef4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:51:18.503 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:18 smithi016 podman[113852]: 2024-09-15 22:51:18.243003105 +0000 UTC m=+6.079546638 container cleanup b17d9657ce21ba07981da914c0d756c2e3087b072f16a6cbedd1522333caef4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-15T22:51:18.503 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:18 smithi016 bash[113852]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9 2024-09-15T22:51:18.807 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:18 smithi016 podman[113867]: 2024-09-15 22:51:18.50264716 +0000 UTC m=+0.730011044 container remove b17d9657ce21ba07981da914c0d756c2e3087b072f16a6cbedd1522333caef4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-15T22:51:18.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:18 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:18.615+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:18.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:18 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:18.615+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:18.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:18 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:18.615+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:18.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:18 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:18.615+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:18.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:18 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:18.615+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:18.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:18 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:18.615+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:18.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:18 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:18.615+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:19.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:18 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:18.747+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:19.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:18 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:18.747+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:19.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:18 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:18.747+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:19.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:18 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:18.747+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:19.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:18 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:18.747+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:19.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:18 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:18.747+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:19.156 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:18 smithi016 podman[113959]: 2024-09-15 22:51:18.721581997 +0000 UTC m=+0.016555516 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:51:19.156 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:18 smithi016 podman[113959]: 2024-09-15 22:51:18.828331108 +0000 UTC m=+0.123304652 container create 61117f73825659732bd2f7f76d5344355dc4fa5b378d19720fe743beeaa46910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:51:19.157 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:19 smithi016 podman[113959]: 2024-09-15 22:51:19.155577715 +0000 UTC m=+0.450551213 container init 61117f73825659732bd2f7f76d5344355dc4fa5b378d19720fe743beeaa46910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-15T22:51:19.556 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:19 smithi016 podman[113959]: 2024-09-15 22:51:19.159034207 +0000 UTC m=+0.454007706 container start 61117f73825659732bd2f7f76d5344355dc4fa5b378d19720fe743beeaa46910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-15T22:51:19.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:19 smithi016 podman[113959]: 2024-09-15 22:51:19.238978029 +0000 UTC m=+0.533951540 container attach 61117f73825659732bd2f7f76d5344355dc4fa5b378d19720fe743beeaa46910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:51:19.557 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:19 smithi016 podman[113959]: 2024-09-15 22:51:19.362011368 +0000 UTC m=+0.656984867 container died 61117f73825659732bd2f7f76d5344355dc4fa5b378d19720fe743beeaa46910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-15T22:51:19.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:19.590+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:19.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:19.590+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:19.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:19.590+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:19.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:19.590+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:19.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:19.590+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:19.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:19.590+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:19.945 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:19 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:19.590+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:19.986 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 15 22:51:19 smithi016 podman[113983]: 2024-09-15 22:51:19.967741457 +0000 UTC m=+0.599554249 container remove 61117f73825659732bd2f7f76d5344355dc4fa5b378d19720fe743beeaa46910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:51:19.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.9.service' 2024-09-15T22:51:20.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:19.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:20.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:19.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:20.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:19.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:20.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:19.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:20.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:19.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:20.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:19.719+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:20.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:20.075 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-15T22:51:20.075 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-15T22:51:20.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.10 2024-09-15T22:51:20.367 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 systemd[1]: Stopping Ceph osd.10 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:20.629 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.365+0000 7f9204065640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:20.629 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.365+0000 7f9204065640 -1 osd.10 104 *** Got signal Terminated *** 2024-09-15T22:51:20.629 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.365+0000 7f9204065640 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:51:20.630 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.628+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:20.630 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.628+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:20.630 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.628+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:20.630 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.628+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:20.630 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.628+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:20.630 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.628+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:20.630 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:20 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:20.628+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:21.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:20.740+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:21.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:20.740+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:21.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:20.740+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:21.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:20.740+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:21.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:20.740+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:21.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:20 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:20.740+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:21.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:21.644+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:21.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:21.644+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:21.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:21.644+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:21.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:21.644+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:21.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:21.644+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:21.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:21.644+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:21.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:21 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:21.644+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:22.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:21.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:22.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:21.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:22.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:21.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:22.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:21.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:22.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:21.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:22.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:21 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:21.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:22.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:22.663+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:22.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:22.663+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:22.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:22.663+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:22.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:22.663+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:22.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:22.663+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:22.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:22.663+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:22.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:22 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:22.663+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:23.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:22.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:23.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:22.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:23.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:22.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:23.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:22.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:23.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:22.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:23.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:22.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:23.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:22 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:22.745+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:23.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:23.647+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:23.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:23.647+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:23.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:23.647+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:23.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:23.647+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:23.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:23.647+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:23.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:23.647+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:23.944 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:23 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:23.647+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:24.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:23.741+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:24.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:23.741+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:24.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:23.741+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:24.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:23.741+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:24.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:23.741+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:24.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:23.741+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:24.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:23.741+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:25.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:24.705+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:25.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:24.705+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:25.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:24.705+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:25.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:24.705+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:25.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:24.705+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:25.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:24.705+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:25.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:24 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:24.705+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:25.193 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:24.694+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:02.130409+0000 front 2024-09-15T22:50:02.130301+0000 (oldest deadline 2024-09-15T22:50:26.230061+0000) 2024-09-15T22:51:25.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:24.694+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:12.432938+0000 front 2024-09-15T22:50:12.433079+0000 (oldest deadline 2024-09-15T22:50:36.533084+0000) 2024-09-15T22:51:25.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:24.694+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:16.533399+0000 front 2024-09-15T22:50:16.533520+0000 (oldest deadline 2024-09-15T22:50:41.833504+0000) 2024-09-15T22:51:25.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:24.694+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-15T22:50:24.134599+0000 front 2024-09-15T22:50:24.134680+0000 (oldest deadline 2024-09-15T22:50:50.034366+0000) 2024-09-15T22:51:25.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:24.694+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.535922+0000 front 2024-09-15T22:50:34.535963+0000 (oldest deadline 2024-09-15T22:50:57.435996+0000) 2024-09-15T22:51:25.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:24.694+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:44.437162+0000 front 2024-09-15T22:50:44.437175+0000 (oldest deadline 2024-09-15T22:51:07.937146+0000) 2024-09-15T22:51:25.194 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:24 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10[59631]: 2024-09-15T22:51:24.694+0000 7f920067b640 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.937567+0000 front 2024-09-15T22:50:47.937635+0000 (oldest deadline 2024-09-15T22:51:15.537917+0000) 2024-09-15T22:51:26.000 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:25 smithi059 podman[72055]: 2024-09-15 22:51:25.738071424 +0000 UTC m=+5.464696265 container died 83570af48b2f5a304f62a0913f1fdded9e6fcffef7520f8273deced9e74ed4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-15T22:51:26.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:25 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:25.729+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:26.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:25 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:25.729+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:26.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:25 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:25.729+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:26.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:25 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:25.729+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:26.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:25 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:25.729+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:26.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:25 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:25.729+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:26.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:25 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:25.729+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:26.383 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:26 smithi059 podman[72055]: 2024-09-15 22:51:26.098593482 +0000 UTC m=+5.825218307 container cleanup 83570af48b2f5a304f62a0913f1fdded9e6fcffef7520f8273deced9e74ed4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:51:26.384 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:26 smithi059 bash[72055]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10 2024-09-15T22:51:26.693 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:26 smithi059 podman[72072]: 2024-09-15 22:51:26.383020494 +0000 UTC m=+0.638503733 container remove 83570af48b2f5a304f62a0913f1fdded9e6fcffef7520f8273deced9e74ed4c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-15T22:51:27.020 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:26 smithi059 podman[72162]: 2024-09-15 22:51:26.626398397 +0000 UTC m=+0.015893266 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:51:27.020 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:26 smithi059 podman[72162]: 2024-09-15 22:51:26.725247335 +0000 UTC m=+0.114742188 container create ddad92b54c32a3971670e66187a0e5e0b40ef0746ff2fd10e314e6c75071005e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-15T22:51:27.020 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:26 smithi059 podman[72162]: 2024-09-15 22:51:26.944469557 +0000 UTC m=+0.333964413 container init ddad92b54c32a3971670e66187a0e5e0b40ef0746ff2fd10e314e6c75071005e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-15T22:51:27.020 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:26 smithi059 podman[72162]: 2024-09-15 22:51:26.947726847 +0000 UTC m=+0.337221701 container start ddad92b54c32a3971670e66187a0e5e0b40ef0746ff2fd10e314e6c75071005e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T22:51:27.020 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:27 smithi059 podman[72162]: 2024-09-15 22:51:27.019365276 +0000 UTC m=+0.408860148 container attach ddad92b54c32a3971670e66187a0e5e0b40ef0746ff2fd10e314e6c75071005e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-15T22:51:27.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:26.733+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:27.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:26.733+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:27.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:26.733+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:27.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:26.733+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:27.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:26.733+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:27.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:26.733+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:27.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:26 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:26.733+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:27.444 INFO:journalctl@ceph.osd.10.smithi059.stdout:Sep 15 22:51:27 smithi059 podman[72162]: 2024-09-15 22:51:27.142671002 +0000 UTC m=+0.532165873 container died ddad92b54c32a3971670e66187a0e5e0b40ef0746ff2fd10e314e6c75071005e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-15T22:51:27.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.10.service' 2024-09-15T22:51:27.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:27.979 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-15T22:51:27.979 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-15T22:51:27.979 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.11 2024-09-15T22:51:28.006 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:27 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:27.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:28.006 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:27 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:27.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:28.006 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:27 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:27.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:28.006 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:27 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:27.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:28.006 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:27 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:27.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:28.006 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:27 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:27.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:28.006 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:27 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:27.728+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:28.278 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 systemd[1]: Stopping Ceph osd.11 for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:28.531 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.443+0000 7fa85e75e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:28.531 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.443+0000 7fa85e75e640 -1 osd.11 104 *** Got signal Terminated *** 2024-09-15T22:51:28.532 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.443+0000 7fa85e75e640 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:51:29.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:29.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:29.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:29.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:29.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:29.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:29.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:28 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:28.739+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:30.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:29.755+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:30.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:29.755+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:30.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:29.755+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:30.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:29.755+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:30.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:29.755+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:30.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:29.755+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:30.029 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:29 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:29.755+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:31.278 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:30.799+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:31.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:30.799+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:31.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:30.799+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:31.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:30.799+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:31.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:30.799+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:31.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:30.799+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:31.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:30 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:30.799+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:32.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:31 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:31.816+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-09-15T22:51:07.361823+0000 front 2024-09-15T22:51:07.361842+0000 (oldest deadline 2024-09-15T22:51:31.461809+0000) 2024-09-15T22:51:33.278 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-15T22:50:03.754236+0000 front 2024-09-15T22:50:03.754217+0000 (oldest deadline 2024-09-15T22:50:29.054266+0000) 2024-09-15T22:51:33.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6804 osd.1 since back 2024-09-15T22:50:09.054960+0000 front 2024-09-15T22:50:09.054770+0000 (oldest deadline 2024-09-15T22:50:38.455314+0000) 2024-09-15T22:51:33.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6804 osd.2 since back 2024-09-15T22:50:18.956267+0000 front 2024-09-15T22:50:18.956402+0000 (oldest deadline 2024-09-15T22:50:44.256318+0000) 2024-09-15T22:51:33.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6812 osd.4 since back 2024-09-15T22:50:34.158231+0000 front 2024-09-15T22:50:34.158243+0000 (oldest deadline 2024-09-15T22:50:58.258068+0000) 2024-09-15T22:51:33.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6812 osd.5 since back 2024-09-15T22:50:39.958944+0000 front 2024-09-15T22:50:39.958999+0000 (oldest deadline 2024-09-15T22:51:05.858867+0000) 2024-09-15T22:51:33.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-15T22:50:47.460277+0000 front 2024-09-15T22:50:47.460316+0000 (oldest deadline 2024-09-15T22:51:15.060431+0000) 2024-09-15T22:51:33.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.59:6820 osd.7 since back 2024-09-15T22:50:56.161242+0000 front 2024-09-15T22:50:56.161216+0000 (oldest deadline 2024-09-15T22:51:22.061170+0000) 2024-09-15T22:51:33.279 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:32 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11[60809]: 2024-09-15T22:51:32.853+0000 7fa85ad74640 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.130:6820 osd.8 since back 2024-09-15T22:51:07.361823+0000 front 2024-09-15T22:51:07.361842+0000 (oldest deadline 2024-09-15T22:51:31.461809+0000) 2024-09-15T22:51:34.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:33 smithi130 podman[72789]: 2024-09-15 22:51:33.767749083 +0000 UTC m=+5.595695729 container died e9d309afaa3a10f17fe79ff8f06efa1d6e94e5713ab3d7ac093c9ecb893c6c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-15T22:51:34.378 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:34 smithi130 podman[72789]: 2024-09-15 22:51:34.126406857 +0000 UTC m=+5.954353562 container cleanup e9d309afaa3a10f17fe79ff8f06efa1d6e94e5713ab3d7ac093c9ecb893c6c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-15T22:51:34.540 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:34 smithi130 bash[72789]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11 2024-09-15T22:51:34.686 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:34 smithi130 podman[72815]: 2024-09-15 22:51:34.377575581 +0000 UTC m=+0.608683797 container remove e9d309afaa3a10f17fe79ff8f06efa1d6e94e5713ab3d7ac093c9ecb893c6c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:51:35.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:34 smithi130 podman[72903]: 2024-09-15 22:51:34.588390881 +0000 UTC m=+0.015711601 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:51:35.028 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:34 smithi130 podman[72903]: 2024-09-15 22:51:34.695033862 +0000 UTC m=+0.122354629 container create 35fdd576c57fb0fa917387f8fc164c67dcc22109d28fe101ed29e860530226c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-15T22:51:35.529 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:35 smithi130 podman[72903]: 2024-09-15 22:51:35.058860517 +0000 UTC m=+0.486181248 container init 35fdd576c57fb0fa917387f8fc164c67dcc22109d28fe101ed29e860530226c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-15T22:51:35.529 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:35 smithi130 podman[72903]: 2024-09-15 22:51:35.062199319 +0000 UTC m=+0.489520040 container start 35fdd576c57fb0fa917387f8fc164c67dcc22109d28fe101ed29e860530226c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-15T22:51:35.530 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:35 smithi130 podman[72903]: 2024-09-15 22:51:35.139300017 +0000 UTC m=+0.566620747 container attach 35fdd576c57fb0fa917387f8fc164c67dcc22109d28fe101ed29e860530226c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T22:51:35.530 INFO:journalctl@ceph.osd.11.smithi130.stdout:Sep 15 22:51:35 smithi130 podman[72903]: 2024-09-15 22:51:35.248689708 +0000 UTC m=+0.676010440 container died 35fdd576c57fb0fa917387f8fc164c67dcc22109d28fe101ed29e860530226c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:51:35.816 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@osd.11.service' 2024-09-15T22:51:35.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:35.897 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-15T22:51:35.897 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-15T22:51:35.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.a 2024-09-15T22:51:36.195 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:35 smithi016 systemd[1]: Stopping Ceph mds.a for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:36.556 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:36 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a[72072]: 2024-09-15T22:51:36.194+0000 7f53aafa8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:36.557 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:36 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a[72072]: 2024-09-15T22:51:36.194+0000 7f53aafa8640 -1 mds.a *** got signal Terminated *** 2024-09-15T22:51:40.057 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:39 smithi016 podman[114064]: 2024-09-15 22:51:39.722053193 +0000 UTC m=+3.619070442 container died f678fa8d9cad7b606d746657064fef4fed528b33028cce78057c950ab11bbb34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-15T22:51:40.438 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:40 smithi016 podman[114064]: 2024-09-15 22:51:40.26879878 +0000 UTC m=+4.165816029 container cleanup f678fa8d9cad7b606d746657064fef4fed528b33028cce78057c950ab11bbb34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T22:51:40.438 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:40 smithi016 bash[114064]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a 2024-09-15T22:51:40.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.a.service' 2024-09-15T22:51:40.710 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:40 smithi016 podman[114076]: 2024-09-15 22:51:40.436957673 +0000 UTC m=+0.708121362 container remove f678fa8d9cad7b606d746657064fef4fed528b33028cce78057c950ab11bbb34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True) 2024-09-15T22:51:40.710 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:40 smithi016 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.a.service: Deactivated successfully. 2024-09-15T22:51:40.710 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:40 smithi016 systemd[1]: Stopped Ceph mds.a for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:51:40.710 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 15 22:51:40 smithi016 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.a.service: Consumed 1.112s CPU time. 2024-09-15T22:51:40.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:40.723 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-15T22:51:40.723 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-15T22:51:40.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.d 2024-09-15T22:51:41.032 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:40 smithi016 systemd[1]: Stopping Ceph mds.d for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:41.306 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d[70877]: 2024-09-15T22:51:41.030+0000 7f07f4da1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:41.307 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:41 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d[70877]: 2024-09-15T22:51:41.030+0000 7f07f4da1640 -1 mds.d *** got signal Terminated *** 2024-09-15T22:51:44.350 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:44 smithi016 podman[114184]: 2024-09-15 22:51:44.086106135 +0000 UTC m=+3.141462519 container died 0d040ce8a24674f66e816fb38cec170db585454e10006d5e17fbfa2ff3c77053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-15T22:51:44.691 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:44 smithi016 podman[114184]: 2024-09-15 22:51:44.523125846 +0000 UTC m=+3.578482227 container cleanup 0d040ce8a24674f66e816fb38cec170db585454e10006d5e17fbfa2ff3c77053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e) 2024-09-15T22:51:44.692 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:44 smithi016 bash[114184]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d 2024-09-15T22:51:44.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.d.service' 2024-09-15T22:51:44.959 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:44 smithi016 podman[114208]: 2024-09-15 22:51:44.691094045 +0000 UTC m=+0.607985320 container remove 0d040ce8a24674f66e816fb38cec170db585454e10006d5e17fbfa2ff3c77053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-15T22:51:44.959 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:44 smithi016 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-15T22:51:44.959 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:44 smithi016 systemd[1]: Stopped Ceph mds.d for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:51:44.959 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 15 22:51:44 smithi016 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.d.service: Consumed 1.077s CPU time. 2024-09-15T22:51:44.974 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:44.974 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-15T22:51:44.974 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-15T22:51:44.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.g 2024-09-15T22:51:45.307 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:51:45 smithi016 systemd[1]: Stopping Ceph mds.g for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:45.807 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:51:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g[71672]: 2024-09-15T22:51:45.435+0000 7f3284a6d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:45.807 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:51:45 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g[71672]: 2024-09-15T22:51:45.435+0000 7f3284a6d640 -1 mds.g *** got signal Terminated *** 2024-09-15T22:51:48.238 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:51:47 smithi016 podman[114317]: 2024-09-15 22:51:47.962004335 +0000 UTC m=+2.761471271 container died 7f183bd1c09f7869abea84e24e13ce8cfe5a903a7be225f35fa19d11e7c633f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T22:51:48.806 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:51:48 smithi016 podman[114317]: 2024-09-15 22:51:48.535646085 +0000 UTC m=+3.335113019 container cleanup 7f183bd1c09f7869abea84e24e13ce8cfe5a903a7be225f35fa19d11e7c633f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-15T22:51:48.807 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:51:48 smithi016 bash[114317]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g 2024-09-15T22:51:48.807 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 15 22:51:48 smithi016 podman[114329]: 2024-09-15 22:51:48.77853355 +0000 UTC m=+0.809230276 container remove 7f183bd1c09f7869abea84e24e13ce8cfe5a903a7be225f35fa19d11e7c633f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-15T22:51:48.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.g.service' 2024-09-15T22:51:48.966 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:48.967 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-15T22:51:48.967 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-15T22:51:48.967 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.j 2024-09-15T22:51:49.282 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:51:49 smithi016 systemd[1]: Stopping Ceph mds.j for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:49.556 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:51:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j[70485]: 2024-09-15T22:51:49.280+0000 7fec58727640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:49.557 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:51:49 smithi016 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j[70485]: 2024-09-15T22:51:49.280+0000 7fec58727640 -1 mds.j *** got signal Terminated *** 2024-09-15T22:51:52.557 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:51:52 smithi016 podman[114437]: 2024-09-15 22:51:52.289372425 +0000 UTC m=+3.101600513 container died d1044db680afc0c190ce363d435c3ae4f3c6b53319428bc62b266f85dab99385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-15T22:51:52.999 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:51:52 smithi016 podman[114437]: 2024-09-15 22:51:52.747634836 +0000 UTC m=+3.559862930 container cleanup d1044db680afc0c190ce363d435c3ae4f3c6b53319428bc62b266f85dab99385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-15T22:51:52.999 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 15 22:51:52 smithi016 bash[114437]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-j 2024-09-15T22:51:53.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.j.service' 2024-09-15T22:51:53.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:53.157 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-15T22:51:53.158 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-15T22:51:53.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.b 2024-09-15T22:51:53.443 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:51:53 smithi059 systemd[1]: Stopping Ceph mds.b for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:53.943 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:51:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: 2024-09-15T22:51:53.447+0000 7f9ec8518640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:53.944 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:51:53 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b[65289]: 2024-09-15T22:51:53.447+0000 7f9ec8518640 -1 mds.b *** got signal Terminated *** 2024-09-15T22:51:57.748 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:51:57 smithi059 podman[72264]: 2024-09-15 22:51:57.454377018 +0000 UTC m=+4.102712104 container died 6d8fe8545d836e87531673f6d0c54260c9a02d00d84309bbf24c1a6ed31bd239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-15T22:51:58.034 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:51:57 smithi059 podman[72264]: 2024-09-15 22:51:57.74752385 +0000 UTC m=+4.395858935 container cleanup 6d8fe8545d836e87531673f6d0c54260c9a02d00d84309bbf24c1a6ed31bd239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-15T22:51:58.034 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:51:57 smithi059 bash[72264]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b 2024-09-15T22:51:58.311 INFO:journalctl@ceph.mds.b.smithi059.stdout:Sep 15 22:51:58 smithi059 podman[72276]: 2024-09-15 22:51:58.140497598 +0000 UTC m=+0.680093519 container remove 6d8fe8545d836e87531673f6d0c54260c9a02d00d84309bbf24c1a6ed31bd239 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:51:58.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.b.service' 2024-09-15T22:51:58.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:51:58.364 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-15T22:51:58.364 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-15T22:51:58.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.e 2024-09-15T22:51:58.693 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:51:58 smithi059 systemd[1]: Stopping Ceph mds.e for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:51:59.193 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:51:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e[64670]: 2024-09-15T22:51:58.717+0000 7f7598242640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:51:59.194 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:51:58 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e[64670]: 2024-09-15T22:51:58.717+0000 7f7598242640 -1 mds.e *** got signal Terminated *** 2024-09-15T22:52:02.419 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:52:02 smithi059 podman[72396]: 2024-09-15 22:52:02.12885781 +0000 UTC m=+3.524978237 container died 02dbb5fb62823d08472024fc191bd83f044a77677f721cfb3b52e16674cd8422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:52:02.693 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:52:02 smithi059 podman[72396]: 2024-09-15 22:52:02.509489303 +0000 UTC m=+3.905609729 container cleanup 02dbb5fb62823d08472024fc191bd83f044a77677f721cfb3b52e16674cd8422 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-15T22:52:02.694 INFO:journalctl@ceph.mds.e.smithi059.stdout:Sep 15 22:52:02 smithi059 bash[72396]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-e 2024-09-15T22:52:02.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.e.service' 2024-09-15T22:52:02.992 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:52:02.993 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-15T22:52:02.993 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-15T22:52:02.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.h 2024-09-15T22:52:03.314 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:52:03 smithi059 systemd[1]: Stopping Ceph mds.h for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:52:03.693 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:52:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h[64976]: 2024-09-15T22:52:03.312+0000 7f85a7f8a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:52:03.694 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:52:03 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h[64976]: 2024-09-15T22:52:03.312+0000 7f85a7f8a640 -1 mds.h *** got signal Terminated *** 2024-09-15T22:52:06.397 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:52:06 smithi059 podman[72519]: 2024-09-15 22:52:06.132612018 +0000 UTC m=+2.922032153 container died 55618dd24db2f05dc7c381d4fa80af4a9272ab8997686b7d8be0c4f6a3c77882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T22:52:06.693 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:52:06 smithi059 podman[72519]: 2024-09-15 22:52:06.479764013 +0000 UTC m=+3.269184145 container cleanup 55618dd24db2f05dc7c381d4fa80af4a9272ab8997686b7d8be0c4f6a3c77882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e) 2024-09-15T22:52:06.694 INFO:journalctl@ceph.mds.h.smithi059.stdout:Sep 15 22:52:06 smithi059 bash[72519]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-h 2024-09-15T22:52:07.073 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.h.service' 2024-09-15T22:52:07.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:52:07.106 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-15T22:52:07.107 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-15T22:52:07.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.k 2024-09-15T22:52:07.434 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:52:07 smithi059 systemd[1]: Stopping Ceph mds.k for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:52:07.693 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:52:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k[64352]: 2024-09-15T22:52:07.432+0000 7f8eeae3a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:52:07.694 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:52:07 smithi059 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k[64352]: 2024-09-15T22:52:07.432+0000 7f8eeae3a640 -1 mds.k *** got signal Terminated *** 2024-09-15T22:52:09.193 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:52:09 smithi059 podman[72641]: 2024-09-15 22:52:09.020837303 +0000 UTC m=+1.695525350 container died d51f617d1f26764f330030bed888ff2fbe5099d0cd91b9ad3eca08e9a27d285f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-15T22:52:09.634 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:52:09 smithi059 podman[72641]: 2024-09-15 22:52:09.374492099 +0000 UTC m=+2.049180134 container cleanup d51f617d1f26764f330030bed888ff2fbe5099d0cd91b9ad3eca08e9a27d285f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:52:09.634 INFO:journalctl@ceph.mds.k.smithi059.stdout:Sep 15 22:52:09 smithi059 bash[72641]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-k 2024-09-15T22:52:09.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.k.service' 2024-09-15T22:52:09.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:52:09.840 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-15T22:52:09.840 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-15T22:52:09.840 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.c 2024-09-15T22:52:10.209 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:52:09 smithi130 systemd[1]: Stopping Ceph mds.c for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:52:10.528 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:52:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c[65634]: 2024-09-15T22:52:10.207+0000 7f699827f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:52:10.529 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:52:10 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c[65634]: 2024-09-15T22:52:10.207+0000 7f699827f640 -1 mds.c *** got signal Terminated *** 2024-09-15T22:52:14.509 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:52:14 smithi130 podman[73004]: 2024-09-15 22:52:14.136998108 +0000 UTC m=+4.109656585 container died 830d2a77a2bcfd8859aae2ff8ab04d0d25c70245762d6e12e4e7692b10899149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-15T22:52:14.778 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:52:14 smithi130 podman[73004]: 2024-09-15 22:52:14.599790287 +0000 UTC m=+4.572448762 container cleanup 830d2a77a2bcfd8859aae2ff8ab04d0d25c70245762d6e12e4e7692b10899149 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-15T22:52:14.778 INFO:journalctl@ceph.mds.c.smithi130.stdout:Sep 15 22:52:14 smithi130 bash[73004]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-c 2024-09-15T22:52:15.032 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.c.service' 2024-09-15T22:52:15.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:52:15.067 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-15T22:52:15.067 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-15T22:52:15.067 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.f 2024-09-15T22:52:15.396 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:52:15 smithi130 systemd[1]: Stopping Ceph mds.f for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:52:15.778 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:52:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f[65330]: 2024-09-15T22:52:15.395+0000 7f11894e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:52:15.778 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:52:15 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f[65330]: 2024-09-15T22:52:15.395+0000 7f11894e9640 -1 mds.f *** got signal Terminated *** 2024-09-15T22:52:18.405 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:52:18 smithi130 podman[73138]: 2024-09-15 22:52:18.1196569 +0000 UTC m=+2.834869101 container died ef1819beb068ccd49f3afadcd52590c5f7af638d41aea26344182d37d55fd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:52:18.738 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:52:18 smithi130 podman[73138]: 2024-09-15 22:52:18.495545892 +0000 UTC m=+3.210758082 container cleanup ef1819beb068ccd49f3afadcd52590c5f7af638d41aea26344182d37d55fd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:52:18.739 INFO:journalctl@ceph.mds.f.smithi130.stdout:Sep 15 22:52:18 smithi130 bash[73138]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-f 2024-09-15T22:52:18.893 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.f.service' 2024-09-15T22:52:18.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:52:18.926 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-15T22:52:18.926 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-15T22:52:18.926 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.i 2024-09-15T22:52:19.278 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:19 smithi130 systemd[1]: Stopping Ceph mds.i for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:52:19.278 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i[65938]: 2024-09-15T22:52:19.232+0000 7ff2ce4d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:52:19.279 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:19 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i[65938]: 2024-09-15T22:52:19.232+0000 7ff2ce4d2640 -1 mds.i *** got signal Terminated *** 2024-09-15T22:52:22.278 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:22 smithi130 podman[73262]: 2024-09-15 22:52:22.134547748 +0000 UTC m=+2.997068964 container died e5b9681439796e97fb7d0237243a4e711f389b24aa7dc2a45d32c66e84836603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T22:52:22.742 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:22 smithi130 podman[73262]: 2024-09-15 22:52:22.549141232 +0000 UTC m=+3.411662452 container cleanup e5b9681439796e97fb7d0237243a4e711f389b24aa7dc2a45d32c66e84836603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-09-15T22:52:22.742 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:22 smithi130 bash[73262]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i 2024-09-15T22:52:22.972 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.i.service' 2024-09-15T22:52:22.993 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:22 smithi130 podman[73274]: 2024-09-15 22:52:22.74201644 +0000 UTC m=+0.603548677 container remove e5b9681439796e97fb7d0237243a4e711f389b24aa7dc2a45d32c66e84836603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T22:52:22.993 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:22 smithi130 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-15T22:52:22.993 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:22 smithi130 systemd[1]: Stopped Ceph mds.i for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:52:22.993 INFO:journalctl@ceph.mds.i.smithi130.stdout:Sep 15 22:52:22 smithi130 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.i.service: Consumed 1.950s CPU time. 2024-09-15T22:52:23.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:52:23.004 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-15T22:52:23.004 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-15T22:52:23.005 DEBUG:teuthology.orchestra.run.smithi130:> sudo systemctl stop ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.l 2024-09-15T22:52:23.278 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:23 smithi130 systemd[1]: Stopping Ceph mds.l for af43fc90-73b1-11ef-bceb-c7b262605968... 2024-09-15T22:52:23.778 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l[65029]: 2024-09-15T22:52:23.335+0000 7ffb06056640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:52:23.779 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:23 smithi130 ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l[65029]: 2024-09-15T22:52:23.335+0000 7ffb06056640 -1 mds.l *** got signal Terminated *** 2024-09-15T22:52:26.529 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:26 smithi130 podman[73382]: 2024-09-15 22:52:26.225600923 +0000 UTC m=+2.996907199 container died ab862784bd7aea40f5e6336af15c70107b2faa70330278aab21eee7a09189b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:52:26.937 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:26 smithi130 podman[73382]: 2024-09-15 22:52:26.761203153 +0000 UTC m=+3.532509428 container cleanup ab862784bd7aea40f5e6336af15c70107b2faa70330278aab21eee7a09189b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:52:26.938 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:26 smithi130 bash[73382]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l 2024-09-15T22:52:27.168 DEBUG:teuthology.orchestra.run.smithi130:> sudo pkill -f 'journalctl -f -n 0 -u ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.l.service' 2024-09-15T22:52:27.190 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:26 smithi130 podman[73395]: 2024-09-15 22:52:26.937109946 +0000 UTC m=+0.708262157 container remove ab862784bd7aea40f5e6336af15c70107b2faa70330278aab21eee7a09189b29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-af43fc90-73b1-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T22:52:27.190 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:27 smithi130 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-15T22:52:27.190 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:27 smithi130 systemd[1]: Stopped Ceph mds.l for af43fc90-73b1-11ef-bceb-c7b262605968. 2024-09-15T22:52:27.190 INFO:journalctl@ceph.mds.l.smithi130.stdout:Sep 15 22:52:27 smithi130 systemd[1]: ceph-af43fc90-73b1-11ef-bceb-c7b262605968@mds.l.service: Consumed 1.085s CPU time. 2024-09-15T22:52:27.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:52:27.200 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-15T22:52:27.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af43fc90-73b1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:52:27.411 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:52:29.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af43fc90-73b1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:52:30.031 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:52:32.481 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af43fc90-73b1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:52:32.681 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:52:35.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:52:35.110 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:52:35.143 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:52:35.170 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-15T22:52:35.171 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi016/crash 2024-09-15T22:52:35.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:52:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:52:35.214 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:52:35.216 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi059/crash 2024-09-15T22:52:35.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:52:35.260 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:52:35.261 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:52:35.262 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi130/crash 2024-09-15T22:52:35.264 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:52:35.298 INFO:teuthology.orchestra.run.smithi130.stderr:tar: /var/lib/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:52:35.298 INFO:teuthology.orchestra.run.smithi130.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:52:35.299 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-15T22:52:35.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-15T22:52:35.339 INFO:tasks.cephadm:Compressing logs... 2024-09-15T22:52:35.340 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:52:35.383 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:52:35.385 DEBUG:teuthology.orchestra.run.smithi130:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:52:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-15T22:52:35.408 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-09-15T22:52:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-15T22:52:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-15T22:52:35.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.log: /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.a.log: /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.a.log: 84.5% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-15T22:52:35.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:52:35.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:52:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-15T22:52:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log: 94.5% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-15T22:52:35.417 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-15T22:52:35.420 INFO:teuthology.orchestra.run.smithi130.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:52:35.420 INFO:teuthology.orchestra.run.smithi130.stderr:: No such file or directory 2024-09-15T22:52:35.420 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:52:35.420 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose --/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr: 82.0% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.2.log: --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.l.log: 87.6% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.8.log: 94.6% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-15T22:52:35.421 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-15T22:52:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-15T22:52:35.422 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-15T22:52:35.423 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:52:35.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-15T22:52:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-15T22:52:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:: No such file or directory 2024-09-15T22:52:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-15T22:52:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr: 94.4% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.y.log: 82.1% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log: 87.7% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:52:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-15T22:52:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-15T22:52:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:52:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.3.log: 94.7% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-15T22:52:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: 90.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:52:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.h.log: 90.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:52:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-15T22:52:35.431 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.b.log: 91.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-15T22:52:35.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:52:35.432 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-15T22:52:35.432 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.c.log: 92.0% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-15T22:52:35.432 INFO:teuthology.orchestra.run.smithi130.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:52:35.435 INFO:teuthology.orchestra.run.smithi016.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:52:35.441 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:52:35.448 INFO:teuthology.orchestra.run.smithi130.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:52:35.453 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:52:35.482 INFO:teuthology.orchestra.run.smithi130.stderr: 92.4% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-15T22:52:35.496 INFO:teuthology.orchestra.run.smithi059.stderr: 92.4% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-15T22:52:35.498 INFO:teuthology.orchestra.run.smithi130.stderr: 94.4% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-15T22:52:35.511 INFO:teuthology.orchestra.run.smithi059.stderr: 94.6% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-15T22:52:35.798 INFO:teuthology.orchestra.run.smithi016.stderr: 90.2% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-15T22:52:35.822 INFO:teuthology.orchestra.run.smithi130.stderr: 91.8% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-15T22:52:35.855 INFO:teuthology.orchestra.run.smithi130.stderr: 95.7% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-15T22:52:35.859 INFO:teuthology.orchestra.run.smithi059.stderr: 91.7% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-15T22:52:36.718 INFO:teuthology.orchestra.run.smithi016.stderr: 90.5% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-15T22:52:37.170 INFO:teuthology.orchestra.run.smithi130.stderr: 92.0% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-15T22:52:37.452 INFO:teuthology.orchestra.run.smithi130.stderr: 93.0% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-15T22:52:37.599 INFO:teuthology.orchestra.run.smithi059.stderr: 92.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-15T22:52:37.643 INFO:teuthology.orchestra.run.smithi016.stderr: 92.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-15T22:52:37.876 INFO:teuthology.orchestra.run.smithi016.stderr: 92.8% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-15T22:52:37.890 INFO:teuthology.orchestra.run.smithi059.stderr: 92.0% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-15T22:52:38.039 INFO:teuthology.orchestra.run.smithi059.stderr: 92.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-15T22:52:38.174 INFO:teuthology.orchestra.run.smithi016.stderr: 92.7% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-15T22:52:38.295 INFO:teuthology.orchestra.run.smithi059.stderr: 92.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-15T22:52:38.454 INFO:teuthology.orchestra.run.smithi130.stderr: 93.6% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-15T22:52:38.771 INFO:teuthology.orchestra.run.smithi016.stderr: 92.7% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-15T22:52:38.773 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-15T22:52:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m3.376s 2024-09-15T22:52:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m12.138s 2024-09-15T22:52:38.774 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.426s 2024-09-15T22:52:39.835 INFO:teuthology.orchestra.run.smithi059.stderr: 93.4% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-15T22:52:39.836 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-09-15T22:52:39.837 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m4.436s 2024-09-15T22:52:39.837 INFO:teuthology.orchestra.run.smithi059.stderr:user 0m14.729s 2024-09-15T22:52:39.837 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m0.470s 2024-09-15T22:52:40.966 INFO:teuthology.orchestra.run.smithi130.stderr: 94.9% -- replaced with /var/log/ceph/af43fc90-73b1-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-15T22:52:40.968 INFO:teuthology.orchestra.run.smithi130.stderr: 2024-09-15T22:52:40.968 INFO:teuthology.orchestra.run.smithi130.stderr:real 0m5.567s 2024-09-15T22:52:40.968 INFO:teuthology.orchestra.run.smithi130.stderr:user 0m12.947s 2024-09-15T22:52:40.968 INFO:teuthology.orchestra.run.smithi130.stderr:sys 0m0.460s 2024-09-15T22:52:40.968 INFO:tasks.cephadm:Archiving logs... 2024-09-15T22:52:40.968 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi016/log 2024-09-15T22:52:40.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T22:52:42.833 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi059/log 2024-09-15T22:52:42.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T22:52:44.776 DEBUG:teuthology.misc:Transferring archived files from smithi130:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi130/log 2024-09-15T22:52:44.777 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T22:52:46.604 INFO:tasks.cephadm:Removing cluster... 2024-09-15T22:52:46.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af43fc90-73b1-11ef-bceb-c7b262605968 --force 2024-09-15T22:52:46.816 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:52:47.090 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af43fc90-73b1-11ef-bceb-c7b262605968 --force 2024-09-15T22:52:47.301 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:52:47.568 DEBUG:teuthology.orchestra.run.smithi130:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid af43fc90-73b1-11ef-bceb-c7b262605968 --force 2024-09-15T22:52:47.770 INFO:teuthology.orchestra.run.smithi130.stdout:Deleting cluster with fsid: af43fc90-73b1-11ef-bceb-c7b262605968 2024-09-15T22:52:48.038 INFO:tasks.cephadm:Removing cephadm ... 2024-09-15T22:52:48.039 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T22:52:48.058 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T22:52:48.075 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T22:52:48.091 INFO:tasks.cephadm:Teardown complete 2024-09-15T22:52:48.091 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T22:52:48.102 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T22:52:48.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T22:52:48.106 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T22:52:48.118 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T22:52:48.230 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T22:52:48.241 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T22:52:48.241 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:52:48.244 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:52:48.246 DEBUG:teuthology.orchestra.run.smithi130:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:52:48.259 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:52:48.259 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:52:48.260 INFO:teuthology.orchestra.run.smithi130.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:52:48.263 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:52:48.263 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-09-15T22:52:48.263 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 +1146us[+1142us] +/- 59ms 2024-09-15T22:52:48.263 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1008us[-1011us] +/- 57ms 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 -108us[ -111us] +/- 80ms 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 7 377 7 +1090us[+1082us] +/- 59ms 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -1097us[-1097us] +/- 57ms 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -181us[ -181us] +/- 80ms 2024-09-15T22:52:48.264 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:52:48.266 INFO:teuthology.orchestra.run.smithi130.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:52:48.266 INFO:teuthology.orchestra.run.smithi130.stdout:=============================================================================== 2024-09-15T22:52:48.266 INFO:teuthology.orchestra.run.smithi130.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +1124us[+1134us] +/- 59ms 2024-09-15T22:52:48.266 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1031us[-1031us] +/- 57ms 2024-09-15T22:52:48.266 INFO:teuthology.orchestra.run.smithi130.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -129us[ -119us] +/- 80ms 2024-09-15T22:52:48.266 INFO:teuthology.orchestra.run.smithi130.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:52:48.266 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T22:52:48.279 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T22:52:48.279 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T22:52:48.298 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T22:52:48.400 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T22:52:48.471 DEBUG:teuthology.orchestra.run.smithi130:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T22:52:48.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:52:48.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:52:48.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:52:48.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:52:48.600 DEBUG:teuthology.orchestra.run.smithi130:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:52:48.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:52:48.627 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T22:52:48.639 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T22:52:48.675 INFO:teuthology.task.internal:Duration was 1873.099569 seconds 2024-09-15T22:52:48.675 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T22:52:48.718 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T22:52:48.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T22:52:48.721 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T22:52:48.723 DEBUG:teuthology.orchestra.run.smithi130:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T22:52:48.760 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:52:48.761 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:52:48.762 INFO:teuthology.orchestra.run.smithi130.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:52:49.240 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T22:52:49.240 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-15T22:52:49.240 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T22:52:49.261 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-09-15T22:52:49.262 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T22:52:49.302 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi130.front.sepia.ceph.com 2024-09-15T22:52:49.303 DEBUG:teuthology.orchestra.run.smithi130:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T22:52:49.331 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T22:52:49.331 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T22:52:49.333 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T22:52:49.345 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T22:52:49.400 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T22:52:49.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T22:52:49.403 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T22:52:49.412 DEBUG:teuthology.orchestra.run.smithi130:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T22:52:50.634 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T22:52:50.646 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T22:52:50.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T22:52:50.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T22:52:50.702 DEBUG:teuthology.orchestra.run.smithi130:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T22:52:50.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T22:52:50.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:52:50.746 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:52:50.748 DEBUG:teuthology.orchestra.run.smithi130:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:52:50.768 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-15T22:52:50.773 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-09-15T22:52:50.798 INFO:teuthology.orchestra.run.smithi130.stdout:kernel.core_pattern = core 2024-09-15T22:52:50.823 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:52:50.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:52:50.838 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:52:50.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:52:50.869 DEBUG:teuthology.orchestra.run.smithi130:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:52:50.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:52:50.885 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T22:52:50.897 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T22:52:50.898 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi016 2024-09-15T22:52:50.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T22:52:50.961 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi059 2024-09-15T22:52:50.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T22:52:51.000 DEBUG:teuthology.misc:Transferring archived files from smithi130:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906253/remote/smithi130 2024-09-15T22:52:51.000 DEBUG:teuthology.orchestra.run.smithi130:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T22:52:51.038 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T22:52:51.039 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T22:52:51.041 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T22:52:51.044 DEBUG:teuthology.orchestra.run.smithi130:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T22:52:51.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T22:52:51.103 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T22:52:51.103 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T22:52:51.118 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T22:52:51.118 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T22:52:51.121 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T22:52:51.123 DEBUG:teuthology.orchestra.run.smithi130:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T22:52:51.133 INFO:teuthology.orchestra.run.smithi016.stdout: 265602 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 22:52 /home/ubuntu/cephtest 2024-09-15T22:52:51.137 INFO:teuthology.orchestra.run.smithi059.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 22:52 /home/ubuntu/cephtest 2024-09-15T22:52:51.145 INFO:teuthology.orchestra.run.smithi130.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 22:52 /home/ubuntu/cephtest 2024-09-15T22:52:51.146 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T22:52:51.158 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T22:52:51.183 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/direct_io}} duration: 1873.0995688438416 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T22:52:51.183 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T22:52:51.269 INFO:teuthology.run:pass